Freight vehicle weighing method and system based on image and laser scanning

Abstract

The invention discloses a freight vehicle weighing method and system based on image and laser scanning, and relates to the technical field of machine vision. Image data and three-dimensional point cloud data of a freight vehicle are collected, and basic information of the freight vehicle is retrieved and obtained from a freight vehicle management database; based on the image data, the three-dimensional point cloud data and the basic information, extracting key features of the freight vehicle, based on feature types of the key features, constructing a weighing prediction model, and training the weighing prediction model by using a pre-collected weighing sample data set, based on the feature vectors of the key features and the weighing prediction model, generating the weight of the freight vehicle, associating the weight of the freight vehicle with freight vehicle identification information, and outputting final freight vehicle weight data; therefore, non-contact and non-intrusive freight vehicle weighing is realized.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a freight vehicle weighing method and system based on image and laser scanning. Background Art

[0002] Traditional methods for weighing freight vehicles rely primarily on static weighing platforms or dynamic weighing bridges, which have significant limitations. Static weighing requires the vehicle to come to a complete stop on the weighing platform, which not only reduces logistics and transportation efficiency but also easily causes traffic congestion. While dynamic weighing allows vehicles to pass at low speeds, it still requires specialized weighing facilities and is restricted to specific weighing areas. These traditional methods not only have high infrastructure costs and maintenance expenses, but the weighing process often interrupts the normal logistics and transportation process, reducing overall transportation efficiency. In addition, traditional weighing methods are difficult to adapt to the modern logistics industry's demand for efficient, convenient, and accurate weighing, especially in areas with frequent freight vehicle traffic such as highways, ports, and logistics centers. Traditional weighing methods have become a bottleneck restricting the improvement of logistics efficiency.

[0003] The rapid development of computer vision, deep learning, and laser scanning technologies has made contactless, non-intrusive freight vehicle weighing possible. This new weighing technology eliminates the need for vehicles to stop or slow down to pass through a specific platform; it can be performed while the vehicle is in motion, significantly improving logistics and transportation efficiency while reducing infrastructure investment costs.

[0004] To this end, the present invention proposes a freight vehicle weighing method and system based on image and laser scanning. Summary of the Invention

[0005] The present invention aims to address at least one of the technical problems existing in the prior art. To this end, the present invention proposes a freight vehicle weighing method and system based on image and laser scanning, enabling non-contact and non-invasive freight vehicle weighing, eliminating the inconvenience of requiring freight vehicles to stop or pass through a specific weighing platform.

[0006] To achieve the above objectives, a freight vehicle weighing method based on image and laser scanning is proposed, which includes the following steps: Step 1: Collect image data of freight vehicles through high-resolution cameras; Step 2: Scan the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle; Step 3: Identify the license plate number of the freight vehicle through license plate recognition technology, and retrieve the basic information of the freight vehicle from the freight vehicle management database; Step 4: extracting key features of the freight vehicle based on the image data, the three-dimensional point cloud data and the basic information; Step 5: Based on the feature types of the key features, a weighing prediction model is constructed, which consists of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork. The weighing prediction model is trained using a pre-collected weighing sample dataset. Step 6: Based on the feature vector of the key feature and the weighing prediction model, generate the freight vehicle weight, associate the freight vehicle weight with the freight vehicle identification information, and output the final freight vehicle weight data.

[0007] The method of collecting image data of freight vehicles by using a high-resolution camera includes the following steps: Step 11: Arrange multiple high-resolution cameras to capture images of freight vehicles from different angles; Step 12: When a freight vehicle is detected entering the collection area, the high-resolution camera is triggered to continuously collect an image sequence of the freight vehicle at a preset frame rate; Step 13: Using image processing technology to detect and track freight vehicles in the acquired image sequence to determine a complete freight vehicle image; Scanning the freight vehicle using a laser scanning device comprises the following steps: Step 21: Deploy multiple laser scanning devices in the area where freight vehicles pass through, and scan freight vehicles from different angles; Step 22: When a freight vehicle is detected entering the scanning area, the laser scanning device emits a laser beam and receives a reflected signal, and records the emission angle, reception angle, and flight time of the laser beam; Step 23: Calculate the three-dimensional coordinates of each reflection point on the surface of the freight vehicle based on the emission angle, reception angle, and flight time of the laser beam to generate three-dimensional point cloud data; The process of identifying the license plate number of a freight vehicle by using license plate recognition technology and retrieving basic information of the freight vehicle from a freight vehicle management database includes the following steps: Step 31: Using the image acquired by the image acquisition module, identify the license plate number of the freight vehicle through the license plate recognition algorithm; Step 32: Using the identified license plate number as an index, query the freight vehicle management database; Step 33: Retrieve basic information of the freight vehicle corresponding to the license plate number from the database; Step 34: Convert the retrieved basic information of the freight vehicle into a structured data format for subsequent processing; The receiving of the image data, the three-dimensional point cloud data, and the basic information, and extracting key features of the freight vehicle comprises the following steps: Step 41: extracting visual features and road environment data from the image data; The extraction of visual features comprises the following sub-steps: Use semantic segmentation algorithms to identify key parts from image data; Calculate the tire-ground contact area and deformation degree from the image data; Analyze the vehicle body posture from the image data, including the lateral tilt angle and longitudinal tilt angle of the vehicle body; Extracting road surface environmental data from image data, including road surface type, road surface flatness and stiffness, road surface inclination angle and slope; Step 42: Extract geometric features from 3D point cloud data: The extraction of geometric features comprises: Measure the length, width, height, and wheelbase of freight vehicles from 3D point cloud data; Calculate the volume and surface area of ​​each part of the freight vehicle from 3D point cloud data; Analyze the deformation degree of the freight vehicle suspension system from 3D point cloud data; Measure the distance change between the chassis of a freight vehicle and the ground from 3D point cloud data; Step 43: Calculate composite features based on visual features, road environment data, and geometric features combined with basic information of the freight vehicle; The calculation of composite features specifically includes: Calculate tire compression ratio based on tire specifications and tire deformation degree; Estimating load distribution based on freight vehicle weight and body posture; Calculate the deviation of the center of gravity of the freight vehicle relative to the standard state; Analyze the relationship between suspension system deformation and load; Step 44: The visual features, road environment data, geometric features, freight vehicle basic information and composite features constitute key features; The image feature processing subnetwork uses a deep convolutional neural network structure to specifically process visual features and extract visual representations of the freight vehicle's appearance, tire deformation, and load distribution; The point cloud feature processing sub-network adopts a voxelized convolutional network structure to specifically process three-dimensional point cloud data and extract the geometric shape and spatial structure features of freight vehicles; The environmental feature processing sub-network processes environmental factors such as road surface type and road surface condition to generate an environmental compensation factor; The freight vehicle parameter processing subnetwork processes the freight vehicle basic information to generate a freight vehicle characteristic representation; The multimodal feature fusion sub-network adopts the attention mechanism and cross-modal transformer structure to adaptively fuse the output features of each sub-network to generate a comprehensive feature representation; The weight prediction sub-network adopts a multi-layer perceptron structure to map the fusion features into the total weight and load capacity of the freight vehicle; The training process of the weighing prediction model includes the following steps: Step 51: Construct a weighing sample dataset, which includes several freight vehicle samples with known weights; Step 52: Based on the weighing sample dataset, a phased training strategy is used to pre-train the image feature processing subnetwork, point cloud feature processing subnetwork, environment feature processing subnetwork, and freight vehicle parameter processing subnetwork: The pre-training process includes: Use the image data and partial weight labels in the weighing sample dataset to pre-train the image feature processing sub-network; Use the 3D point cloud data and some weight labels in the weighing sample dataset to pre-train the point cloud feature processing sub-network; Use the environmental parameters and correction coefficients in the weighing sample dataset to pre-train the environmental feature processing subnetwork; Use the basic information of freight vehicles and load-carrying relationships in the weighing sample dataset to pre-train the freight vehicle parameter processing subnetwork; Step 53: Using an end-to-end joint training method, optimize the multimodal feature fusion subnetwork and the weight prediction subnetwork to complete the training of the weight prediction model; Outputting the final freight vehicle weight data comprises the following steps: Step 61: Based on the feature vector of the key feature and the weighing prediction model, generate a predicted total weight of the freight vehicle through a weight prediction subnetwork of the weighing prediction model; Step 62: Calculate the freight vehicle load capacity: the load capacity is the predicted total weight minus the freight vehicle's own weight; Step 63: Combine the final freight vehicle load and freight vehicle identification information to output freight vehicle weight data.

[0008] A freight vehicle weighing system based on image and laser scanning is proposed, which includes an image acquisition module, a laser scanning module, a freight vehicle information acquisition module, a feature extraction module, a weighing prediction module, and a weight output module. The modules are electrically connected to each other. An image acquisition module collects image data of the freight vehicle through a high-resolution camera and sends the image data to a feature extraction module; A laser scanning module scans the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle, and sends the three-dimensional point cloud data to the feature extraction module; The freight vehicle information acquisition module uses license plate recognition technology to identify the license plate number of the freight vehicle, retrieves the basic information of the freight vehicle from the freight vehicle management database, and sends the basic information to the feature extraction module and the weighing prediction module; a feature extraction module that receives the image data, the three-dimensional point cloud data, and the basic information, and extracts key features of the freight vehicle; the feature extraction module sends the extracted key features to the weighing prediction module and the weight output module; A weight prediction module, based on the feature types of the key features, constructs a weight prediction model consisting of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork, and trains the weight prediction model using a pre-collected weighing sample data set; the weight prediction module sends the trained model to the weight output module; The weight output module generates the weight of the freight vehicle based on the feature vector of the key feature and the weighing prediction model, associates the weight of the freight vehicle with the freight vehicle identification information, and outputs the final freight vehicle weight data.

[0009] Compared with the prior art, the present invention has the following beneficial effects: First, high-resolution cameras are used to collect image data of freight vehicles. At the same time, laser scanning equipment performs three-dimensional scanning on freight vehicles to obtain three-dimensional point cloud data. Secondly, license plate recognition technology is used to identify the license plate number of the freight vehicle, and the basic information of the vehicle is retrieved from the freight vehicle management database. Based on the collected image data, three-dimensional point cloud data and vehicle basic information, the key features of the freight vehicle are extracted. These features include vehicle geometric dimensions, vehicle deformation degree, tire compression status, suspension system status, cargo distribution and other features related to vehicle weight. A weighing prediction model is constructed. The weighing prediction model consists of multiple specialized sub-networks: the image feature processing sub-network is responsible for processing visual features extracted from the image; the point cloud feature processing sub-network processes the geometric features in the three-dimensional point cloud data; the environmental feature processing sub-network considers the road conditions; the freight vehicle parameter processing sub-network processes the vehicle's own parameter information; the multimodal feature fusion sub-network effectively fuses various features; finally, the weight prediction sub-network generates the final weight prediction result based on the fused features. Finally, the system inputs the extracted key features into a trained weight prediction model to generate a predicted freight vehicle weight. This weight is then correlated with the vehicle's identification information to output the final freight vehicle weight data. This enables non-contact, non-invasive freight vehicle weighing based on image and laser scanning. Vehicles can be weighed while driving normally, without having to stop or slow down to pass through a specific weighing platform, significantly improving logistics and transportation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Flowchart of a freight vehicle weighing method based on image and laser scanning in Example 1 of the present invention; Figure 2 This is a module connection diagram of the freight vehicle weighing system based on image and laser scanning in Example 2 of the present invention. DETAILED DESCRIPTION

[0011] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention. Example 1

[0012] like Figure 1 As shown, the freight vehicle weighing method based on image and laser scanning includes the following steps: Step 1: Collect image data of freight vehicles through high-resolution cameras; Step 2: Scan the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle; Step 3: Identify the license plate number of the freight vehicle through license plate recognition technology, and retrieve the basic information of the freight vehicle from the freight vehicle management database; Step 4: extracting key features of the freight vehicle based on the image data, the three-dimensional point cloud data and the basic information; Step 5: Based on the feature types of the key features, a weighing prediction model is constructed, which consists of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork. The weighing prediction model is trained using a pre-collected weighing sample dataset. Step 6: Based on the feature vector of the key feature and the weighing prediction model, generate the freight vehicle weight, associate the freight vehicle weight with the freight vehicle identification information, and output the final freight vehicle weight data.

[0013] Specifically, the method of collecting image data of freight vehicles using a high-resolution camera includes the following steps: Step 11: Place several high-resolution cameras on the roads where freight vehicles pass, and capture images of freight vehicles from different angles; The method of arranging several high-resolution cameras is as follows: Brackets are set up on both sides of the road where freight vehicles pass, and high-resolution cameras are installed on the brackets. The high-resolution cameras include a front-view camera, a side-view camera, and a top-view camera to form an image acquisition system with multi-angle coverage. The front-view camera is installed on a bracket in front of the freight vehicle in the direction of travel to capture the front image of the freight vehicle; the side-view camera is installed on a bracket on both sides of the road to capture the side image of the freight vehicle; the top-view camera is installed on a bridge across the road to capture the top image of the freight vehicle. Each camera is equipped with an anti-shake device and an autofocus function to ensure clear images under various lighting conditions and weather environments. The installation height and angle of the camera are precisely adjusted to ensure that freight vehicles in the acquisition area can be fully captured.

[0014] Step 12: When a freight vehicle is detected entering the collection area, the high-resolution camera is triggered to continuously collect an image sequence of the freight vehicle at a preset frame rate; The method for triggering the high-resolution camera to continuously capture an image sequence of freight vehicles at a preset frame rate is as follows: a freight vehicle detection device is set at the entrance of the collection area, and the freight vehicle detection device includes a ground sensor coil and an infrared sensor. When a freight vehicle passes the ground sensor coil, the coil generates an electromagnetic induction signal; at the same time, the infrared sensor detects that the freight vehicle blocks the infrared light and generates a trigger signal. After the electromagnetic induction signal and the two trigger signals are triggered simultaneously, it is determined that a freight vehicle has entered the collection area, and the system background sends a start instruction to all high-resolution cameras. After receiving the start instruction, the high-resolution camera begins to continuously capture images at a preset frame rate. The preset frame rate is automatically adjusted according to the speed of the freight vehicle to ensure that a sufficiently dense image sequence can be obtained during the process of the freight vehicle passing through the entire collection area. Preferably, a freight vehicle exit detection device can also be provided, which automatically stops image acquisition when it detects that the freight vehicle has completely left the collection area to save storage space and computing resources. It can be understood that since the height of a freight vehicle is higher than that of a small family car, the infrared sensor can be installed at a high position to avoid false detection of a family car; Step 13: Using image processing technology to detect and track freight vehicles in the acquired image sequence, determine complete freight vehicle images, and form a complete freight vehicle image set as image data; In a specific embodiment, the method for detecting and tracking freight vehicles is: The system uses a deep learning target detection algorithm to process the collected image sequence. First, a target detection model based on a convolutional neural network, such as the improved Faster R-CNN or YOLO model, is used to detect freight vehicles in each frame of the image, identify the position of the freight vehicle in the image, and generate a bounding box. Then, a multi-target tracking algorithm, such as DeepSORT or IoU tracker, is used to establish a correspondence between freight vehicles between consecutive frames in the image sequence to form a freight vehicle trajectory. Preferably, for the case of partial occlusion by other freight vehicles, spatiotemporal context information is used to compensate to ensure that a complete freight vehicle image sequence is obtained. Finally, each frame of the freight vehicle image is intercepted from the freight vehicle trajectory to form a complete freight vehicle image set as image data; Furthermore, the scanning of the freight vehicle by the laser scanning device includes the following steps: Step 21: Deploy multiple laser scanning devices in the area where freight vehicles pass through, and scan freight vehicles from different angles; In an embodiment of the present invention, the step of arranging multiple laser scanning devices to scan the freight vehicle from different angles specifically includes: Several laser scanning devices are deployed on both sides of the freight vehicle traffic area. Some are mounted on a support structure above the ground, scanning freight vehicles from the sides and top. Others are embedded in the road surface, scanning the freight vehicle chassis from the bottom. The laser scanning devices consist of a laser transmitter, a receiver, and a signal processing unit. The laser transmitter emits a laser beam of a specific wavelength, the receiver captures the laser signal reflected from the freight vehicle surface, and the signal processing unit performs preliminary processing of the received reflected signal.

[0015] Step 22: When a freight vehicle is detected entering the scanning area, the laser scanning device emits a laser beam and receives a reflected signal; When the electromagnetic induction signal and the two trigger signals are triggered simultaneously, the laser scanning device is activated.

[0016] Once activated, the laser scanning device operates according to a pre-set scanning pattern, which includes scanning frequency, scanning density, and scanning sequence. The laser beam emitted by the laser transmitter follows a specific scanning path, covering various surfaces of the freight vehicle. When the laser beam strikes the surface of the freight vehicle, some of the light is reflected back to the receiver. The receiver captures these reflected signals and records the time difference between the laser beam's emission and reception times, as well as the emission and reception angles of the laser beam.

[0017] Step 23: Calculate the three-dimensional coordinates of each point on the surface of the freight vehicle based on the emission angle and flight time of the laser beam, and generate three-dimensional point cloud data; In an embodiment of the present invention, calculating the three-dimensional coordinates of each point on the surface of the freight vehicle based on the emission angle and flight time of the laser beam to generate three-dimensional point cloud data specifically includes: Based on the principle of the speed of light, the round-trip distance of the laser beam is calculated by the flight time. Combined with the emission angle, the three-dimensional coordinates of each reflection point on the surface of the freight vehicle are calculated using the triangulation principle.

[0018] For each reflection point, its X, Y, and Z coordinates and reflection intensity are recorded to form the initial point cloud data. This initial point cloud data is then filtered for noise to remove outliers caused by environmental interference or multiple reflections. A spatial clustering algorithm is then used to segment the filtered point cloud, identifying and extracting the subset of point clouds belonging to the freight vehicle while excluding point clouds from the ground and surrounding environment.

[0019] Finally, to improve point cloud quality, the segmented 3D point cloud data is registered and fused, unifying the 3D point cloud data from different laser scanners into a common coordinate system and eliminating redundant points in overlapping areas. The resulting high-precision 3D point cloud contains detailed geometric information about the freight vehicle's exterior surface, providing the foundational data for subsequent feature extraction and weight prediction.

[0020] Furthermore, the process of identifying the license plate number of a freight vehicle by using license plate recognition technology and retrieving basic information of the freight vehicle from a freight vehicle management database includes the following steps: Step 31: Using the image acquired by the image acquisition module, identify the license plate number of the freight vehicle through the license plate recognition algorithm; It is understandable that the existing OCR technology can accurately identify the license plate number of a freight vehicle, which is a conventional method in the field and will not be described in detail in the present invention. Step 32: Using the identified license plate number as an index, query the freight vehicle management database. It is understood that overweight freight vehicles may affect public transportation safety. Therefore, the freight vehicle management database can be derived from the freight vehicle management unit. Step 33: Retrieve basic information of the freight vehicle corresponding to the license plate number from the database. The basic information includes parameters such as the freight vehicle model, tire specifications, freight vehicle weight, rated load capacity, and freight vehicle dimensions. It is understood that reading data from the database is a conventional method in the art and will not be described in detail herein. Furthermore, the receiving of the image data, the three-dimensional point cloud data, and the basic information, and extracting key features of the freight vehicle includes the following steps: Step 41: extracting visual features and road environment data from the image data; Specifically, the extraction of visual features includes the following sub-steps: A semantic segmentation algorithm is used to identify key features of freight vehicles, such as their outline, tire locations, and compartment areas, from image data. Specifically, the algorithm employs a deep learning-based U-Net network structure, consisting of an encoder and a decoder. The encoder extracts multi-scale features from each frame of the image data through several convolutional and pooling layers, while the decoder restores the spatial resolution of these features through several deconvolutional layers and upsampling operations. In practical applications, images captured by a high-resolution camera are fed into a trained semantic segmentation network, which outputs a probability map of each pixel belonging to each key feature. Thresholding is then used to generate a segmentation mask, thereby locating key features of the freight vehicle.

[0021] The tire's contact area and deformation are calculated from the image data. Specifically, an edge detection algorithm is used to extract the tire's outline from the tire area identified by the semantic segmentation algorithm. The tire's aspect ratio and deformation are calculated to analyze the shape changes in the tire's contact area. This involves first extracting the ideal circular outline of the tire as a reference. Next, the deviation between the actual tire outline and the ideal outline is measured, and the vertical compression deformation of the tire is calculated as the deformation degree. Finally, the actual contact area of ​​the tire is estimated based on the number of pixels in the contact area, combined with camera calibration parameters.

[0022] Analyze the vehicle body posture from the image data, including the lateral and longitudinal tilt angles. Specifically, the three-dimensional posture of the vehicle body is analyzed by detecting the spatial positional relationship of the vehicle body feature points. First, based on the semantic segmentation results, key points of the vehicle body frame are extracted, such as the front, rear, and four corners of the vehicle body. Then, through perspective transformation and camera calibration parameters, the feature points in the two-dimensional image are mapped to three-dimensional space. Next, the angle between the vehicle body plane and the horizontal plane is calculated to obtain the lateral and longitudinal tilt angles of the vehicle body. Road surface environmental data, including road surface type, smoothness and stiffness, and road inclination and slope, is extracted from the image data. Specifically, the system analyzes the texture and geometric characteristics of the road surface surrounding the freight vehicle in the image data to extract these road surface environmental parameters. First, a texture classification algorithm is used to identify the road surface type, classifying it into different categories, such as asphalt, concrete, and gravel. Next, the system assesses road surface smoothness by analyzing grayscale variations and texture complexity within the road surface image. Next, the system estimates the road surface stiffness based on the relative motion between the freight vehicle and the road surface. Finally, the system calculates the road surface inclination and slope by analyzing the road surface's geometry and gradient. The system uses these extracted road surface environmental parameters as correction factors during the weighing process, minimizing the impact of environmental factors on weighing accuracy.

[0023] Step 42: Extract geometric features from 3D point cloud data: In this embodiment, extracting geometric features includes: The length, width, height, and wheelbase of freight vehicles are measured from 3D point cloud data. Specifically, based on 3D point cloud data acquired by laser scanning equipment, point cloud segmentation and feature extraction algorithms are used to measure the geometric dimensions of freight vehicles. First, a ground detection algorithm is used to identify and remove ground point clouds, isolating the freight vehicle point cloud. Then, a Euclidean clustering algorithm is used to segment the freight vehicle point cloud and identify the main structure of the freight vehicle. Next, the maximum distance of the freight vehicle point cloud along the three main axes is calculated to obtain the length, width, and height of the freight vehicle. Finally, by identifying the point cloud density characteristics at the axle positions, the distance between adjacent axles is measured to obtain the freight vehicle's wheelbase information. The system compares the measured geometric dimensions with standard parameters in the freight vehicle management database to verify the freight vehicle type and provide basic data for subsequent load estimation.

[0024] Calculate the volume and surface area of ​​each part of a freight vehicle from 3D point cloud data; Calculate the volume and surface area of ​​each part of a freight vehicle from 3D point cloud data; Specifically, based on the segmented point cloud of the freight vehicle, 3D reconstruction and volume calculation algorithms are used to estimate the volume and surface area of ​​each part of the freight vehicle. First, the freight vehicle point cloud is divided into functional areas such as the front, compartment, and chassis. Then, the point cloud of each area is triangulated to generate a 3D surface model of each part of the freight vehicle. Next, the surface area of ​​each part is calculated based on the 3D model. Finally, the volume of each part is calculated using voxelization or convex hull algorithms.

[0025] Analyze the deformation degree of the freight vehicle suspension system from 3D point cloud data; Specifically, the deformation of the suspension system is analyzed by tracking the position changes of key components of the freight vehicle's suspension system. First, the key components of the freight vehicle, such as the frame, axle, suspension springs, and shock absorbers, are identified from the 3D point cloud data. Then, the relative positions of these components in the vertical direction are measured. Next, the compression deformation of the suspension system, including the spring compression length and shock absorber compression degree, is calculated. Measure the distance change between the chassis of a freight vehicle and the ground from 3D point cloud data; Specifically, the load status of a freight vehicle is assessed by analyzing the change in the distance between the truck chassis and the ground. First, a point cloud plane representing the ground is extracted from the 3D point cloud data. Then, the lowest points of the truck chassis, including the bottom of the frame, anti-collision beams, and exhaust system, are identified. The vertical distance from these lowest points to the ground plane is calculated to obtain the truck's ground clearance. The ground clearance at each time point is then sorted by time to form the distance change. Step 43: Calculate composite features based on visual features, road environment data, and geometric features combined with basic information of the freight vehicle; The calculation of composite features specifically includes: The tire compression ratio is calculated based on tire specifications and actual deformation. Specifically, the tire compression ratio is calculated based on tire deformation information extracted from image data and tire specification parameters obtained from basic vehicle information. First, the standard tire specifications of the vehicle, including tire diameter, width, and sidewall height, are retrieved from the vehicle management database. Then, combined with the actual tire deformation data obtained from image analysis, the vertical compression deformation of the tire is calculated. Next, the actual deformation is compared with the standard tire height to calculate the tire compression ratio. Finally, a functional relationship between the tire compression ratio and load is established, taking into account the tire's elastic properties and air pressure.

[0026] The vehicle's load distribution is estimated based on its own weight and body posture. Specifically, the vehicle's own weight data, included in the basic vehicle information, is combined with body posture information extracted from image data to estimate the vehicle's load distribution. First, the vehicle's own weight and standard center of gravity are obtained from the vehicle management database. Then, based on the body posture analysis results, the vehicle's sinkage at different axle positions is calculated. Next, a mapping relationship between changes in body posture and axle load is established based on the vehicle's suspension characteristics and axle load distribution patterns. Finally, the load distribution on each axle is estimated by solving the mechanical equilibrium equations.

[0027] Calculate the offset of the vehicle's center of gravity relative to its standard state. Specifically, this offset is calculated based on the vehicle's body posture and load distribution information. First, the center of gravity coordinates in the standard state are obtained from the vehicle's basic information. Then, based on the vehicle's body posture analysis and load distribution estimation, the center of gravity position in the current state is calculated. Next, the three-dimensional offset vector between the current center of gravity position and the standard center of gravity position is calculated. Finally, the relationship between the center of gravity offset and load changes is analyzed to establish a corresponding relationship between the center of gravity offset and the total load.

[0028] Analyze the relationship between suspension system deformation and load. Specifically, based on the suspension system deformation information extracted from 3D point cloud data, the quantitative relationship between suspension deformation and load is analyzed. First, the suspension system type and parameters, including spring stiffness, shock absorber characteristics, and suspension structure, are obtained from basic vehicle information. Then, based on the suspension system deformation analysis results, the load acting on the suspension system is calculated. Next, based on the mechanical model of the suspension system, a functional relationship between suspension deformation and load is established. Step 44: The visual features, road environment data, geometric features, freight vehicle basic information and composite features constitute key features; Furthermore, the image feature processing subnetwork uses a deep convolutional neural network structure to specifically process visual features and extract visual representations of the freight vehicle's appearance, tire deformation, and load distribution; The point cloud feature processing sub-network adopts a voxelized convolutional network structure to specifically process three-dimensional point cloud data and extract the geometric shape and spatial structure features of freight vehicles; The environmental feature processing sub-network processes environmental factors such as road surface type and road surface condition to generate an environmental compensation factor; The freight vehicle parameter processing subnetwork processes the freight vehicle basic information to generate a freight vehicle characteristic representation; The multimodal feature fusion sub-network adopts the attention mechanism and cross-modal transformer structure to adaptively fuse the output features of each sub-network to generate a comprehensive feature representation; The weight prediction sub-network adopts a multi-layer perceptron structure to map the fusion features into the total weight and load capacity of the freight vehicle; Furthermore, the training process of the weight prediction model includes the following steps: Step 51: Pre-collect a weighing sample dataset, which includes several freight vehicle samples with known weights. Each freight vehicle sample includes image data, 3D point cloud data, basic information about the freight vehicle, environmental parameters, and actual weight labels. The weighing sample data set consists of samples of various types of freight vehicles, including sample data of different vehicle models, different load states and different environmental conditions.

[0029] Among them, each freight vehicle sample contains at least the following information: multi-angle image data collected by high-resolution cameras, recording image data of the vehicle's appearance, tire deformation and load distribution; three-dimensional point cloud data obtained by laser scanning equipment, recording the precise geometric shape and spatial structure of the vehicle; basic vehicle information retrieved from the vehicle management database, including parameters such as vehicle model, number of axles, wheelbase, rated load and dead weight; environmental parameters at the time of collection, including road surface type, road surface flatness, temperature and humidity, etc.; and actual weight labels measured by standard static weighing equipment as supervision signals for training weighing prediction models.

[0030] In the implementation of this invention, a stratified sampling strategy was employed to construct the weighing sample dataset, ensuring a balanced distribution across vehicle types and load states. For each vehicle type, samples were collected under different load states, including empty, partially loaded, and fully loaded. For each load state, samples were collected under varying road conditions and environmental parameters to enhance the model's adaptability to environmental variations.

[0031] The image data, three-dimensional point cloud data, basic vehicle information, environmental parameters and actual weight label of each sample are converted into visual features, road environment data, geometric features and key feature extraction of composite features through steps 41 to 44 to form a structured freight vehicle sample for model training.

[0032] Step 52: Based on the weighing sample dataset, a phased training strategy is used to pre-train the image feature processing subnetwork, point cloud feature processing subnetwork, environment feature processing subnetwork, and freight vehicle parameter processing subnetwork: Specifically, the pre-training process includes: To pre-train the image feature processing subnetwork, visual features and corresponding weight labels are first extracted from a dataset of weighing samples. This pre-training utilizes a transfer learning approach, using a pre-trained computer vision model as initial weights and fine-tuning the network to adapt it to the freight vehicle weighing task. During training, the image data undergoes data augmentation, including random cropping, rotation, and brightness and contrast adjustments, to enhance the model's generalization capabilities. A mean squared error loss is used as the loss function, optimizing network parameters to minimize the difference between the predicted weight and the partial weight labels.

[0033] For pre-training of the point cloud feature processing subnetwork, geometric features and corresponding weight labels are extracted from a dataset of weighing samples. During pre-training of the point cloud feature processing subnetwork, the point cloud data is voxelized and converted into a regular 3D grid representation. A voxelized convolutional network architecture is employed to progressively extract geometric features of the point cloud through 3D convolutional layers. Point cloud data augmentation techniques, including random rotation, translation, and point density changes, are used during training to enhance the model's robustness to point cloud variations. Mean squared error loss is also used as the loss function to optimize network parameters.

[0034] For the pre-training of the environmental feature processing sub-network, the road environment data and the corresponding correction coefficients in the weighing sample data set are used. The sub-network learns the influence of environmental factors on the weighing results and generates environmental compensation factors. During the training of the environmental feature processing sub-network, the environmental parameters are standardized and input into the multi-layer perceptron structure. The loss function is designed as the mean square error between the predicted correction coefficient and the true correction coefficient, and the network parameters are optimized by back propagation. The correction coefficient refers to a numerical parameter used to adjust the influence of environmental factors on the weighing results. For example, when the road surface has a certain degree of inclination, it will cause the vehicle weight to be unevenly distributed among the tires, affecting the image-based tire deformation analysis results; the true correction coefficient is calculated by comparing the weighing results under different environmental conditions with the weighing results under standard conditions, and the degree of influence of the environmental parameter on the weighing results is formed to form a corresponding correction coefficient; The vehicle parameter processing subnetwork pre-trains the basic information and load relationships of freight vehicles from the weighing sample dataset. This subnetwork learns how different vehicle models and structural parameters affect load capacity. During pre-training, basic vehicle information undergoes feature encoding and embedding, then is input into a deep neural network. The loss function is designed to be the mean squared error between the predicted and actual load relationships, and the network parameters are optimized using gradient descent.

[0035] After the pre-training of each sub-network is completed, its parameters are frozen as the basis for subsequent end-to-end joint training.

[0036] Step 53: Using an end-to-end joint training method, optimize the multimodal feature fusion subnetwork and the weight prediction subnetwork to complete the training of the weight prediction model; The method of optimizing the multimodal feature fusion subnetwork and the weight prediction subnetwork is as follows: The joint training phase first loads the pre-trained and frozen parameters of the image feature processing subnetwork, point cloud feature processing subnetwork, environmental feature processing subnetwork, and vehicle parameter processing subnetwork. Then, a complete end-to-end model is constructed, including the image feature processing subnetwork, point cloud feature processing subnetwork, environmental feature processing subnetwork, vehicle parameter processing subnetwork, multimodal feature fusion subnetwork, and weight prediction subnetwork.

[0037] The multimodal feature fusion subnetwork uses an attention mechanism and a cross-modal transformer structure to adaptively fuse features from different subnetworks. Specifically, it first calculates the attention weights for each modal feature, reflecting its importance in the current sample. Then, through the cross-modal transformer structure, it establishes associations between features from different modalities, capturing their complementary information. Finally, based on the attention weights and cross-modal associations, it adaptively fuses the features from each modality to generate a comprehensive feature representation.

[0038] The weight prediction subnetwork uses a multi-layer perceptron architecture to map the comprehensive feature representation to the total weight of the freight vehicle. The loss function designed for the weight prediction subnetwork consists of two parts: a main loss term, the mean squared error (MSE) between the predicted and actual total weights, reflecting prediction accuracy; and a regularization term, the L2 norm of the model parameters, to prevent overfitting. The loss function is formulated as the mean squared error between the predicted and actual total weights plus the L2 norm of the model parameters multiplied by the regularization coefficient.

[0039] The training process uses mini-batch stochastic gradient descent. Each batch is randomly sampled from the training dataset. The batch loss is calculated and the network parameters are updated. The learning rate uses a cosine annealing strategy, with a higher learning rate used early in training for rapid convergence and a smaller learning rate used later for fine-tuning. An early stopping strategy is also used, terminating training when performance on the validation set no longer improves to prevent overfitting.

[0040] Furthermore, the outputting of the final freight vehicle weight data comprises the following steps: Step 61: Based on the feature vector of the key feature and the weighing prediction model, generate a predicted total weight of the freight vehicle through a weight prediction subnetwork of the weighing prediction model; Specifically, the method for generating the predicted total weight of the freight vehicle is: The feature vectors of key features are input into the weighing prediction model, and the corresponding processed features are output through the image feature processing subnetwork, point cloud feature processing subnetwork, environmental feature processing subnetwork and freight vehicle parameter processing subnetwork respectively. The processed features are then passed through the multimodal feature fusion subnetwork and the weight prediction subnetwork to output the predicted total weight of the freight vehicle; Step 62: Calculate the freight vehicle load capacity: the load capacity is the predicted total weight minus the freight vehicle's own weight; Step 63: Combine the final freight vehicle load and freight vehicle identification information to output freight vehicle weight data. Example 2

[0041] like Figure 2 As shown, the freight vehicle weighing system based on image and laser scanning includes an image acquisition module, a laser scanning module, a freight vehicle information acquisition module, a feature extraction module, a weighing prediction module, and a weight output module; wherein each module is electrically connected; An image acquisition module collects image data of the freight vehicle through a high-resolution camera and sends the image data to a feature extraction module; A laser scanning module scans the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle, and sends the three-dimensional point cloud data to the feature extraction module; The freight vehicle information acquisition module uses license plate recognition technology to identify the license plate number of the freight vehicle, retrieves the basic information of the freight vehicle from the freight vehicle management database, and sends the basic information to the feature extraction module and the weighing prediction module; a feature extraction module that receives the image data, the three-dimensional point cloud data, and the basic information, and extracts key features of the freight vehicle; the feature extraction module sends the extracted key features to the weighing prediction module and the weight output module; A weight prediction module, based on the feature types of the key features, constructs a weight prediction model consisting of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork, and trains the weight prediction model using a pre-collected weighing sample data set; the weight prediction module sends the trained model to the weight output module; The weight output module generates the weight of the freight vehicle based on the feature vector of the key feature and the weighing prediction model, associates the weight of the freight vehicle with the freight vehicle identification information, and outputs the final freight vehicle weight data.

[0042] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A freight vehicle weighing method based on image and laser scanning, characterized in that: The following steps are involved: Step 1: Collect image data of freight vehicles through high-resolution cameras; Step 2: Scan the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle; Step 3: Identify the license plate number of the freight vehicle through license plate recognition technology, and retrieve the basic information of the freight vehicle from the freight vehicle management database; Step 4: extracting key features of the freight vehicle based on the image data, the three-dimensional point cloud data and the basic information; Step 5: Based on the feature types of the key features, a weighing prediction model is constructed, which consists of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork. The weighing prediction model is trained using a pre-collected weighing sample dataset. Step 6: Based on the feature vector of the key feature and the weighing prediction model, generate the freight vehicle weight, associate the freight vehicle weight with the freight vehicle identification information, and output the final freight vehicle weight data.

2. The freight vehicle weighing method based on image and laser scanning according to claim 1, characterized in that: The method of collecting image data of freight vehicles by using a high-resolution camera includes the following steps: Step 11: Arrange multiple high-resolution cameras to capture images of freight vehicles from different angles; Step 12: When a freight vehicle is detected entering the collection area, the high-resolution camera is triggered to continuously collect an image sequence of the freight vehicle at a preset frame rate; Step 13: Use image processing technology to detect and track freight vehicles in the collected image sequence to determine a complete freight vehicle image.

3. The freight vehicle weighing method based on image and laser scanning according to claim 2, characterized in that: Scanning the freight vehicle using a laser scanning device comprises the following steps: Step 21: Deploy multiple laser scanning devices in the area where freight vehicles pass through, and scan freight vehicles from different angles; Step 22: When a freight vehicle is detected entering the scanning area, the laser scanning device emits a laser beam and receives a reflected signal, and records the emission angle, reception angle, and flight time of the laser beam; Step 23: Based on the emission angle, reception angle, and flight time of the laser beam, the three-dimensional coordinates of each reflection point on the surface of the freight vehicle are calculated to generate three-dimensional point cloud data.

4. The freight vehicle weighing method based on image and laser scanning according to claim 3, characterized in that: The process of identifying the license plate number of a freight vehicle by using license plate recognition technology and retrieving basic information of the freight vehicle from a freight vehicle management database includes the following steps: Step 31: Using the image acquired by the image acquisition module, identify the license plate number of the freight vehicle through the license plate recognition algorithm; Step 32: Using the identified license plate number as an index, query the freight vehicle management database; Step 33: Retrieve basic information of the freight vehicle corresponding to the license plate number from the database; Step 34: Convert the retrieved basic information of the freight vehicle into a structured data format for subsequent processing.

5. The freight vehicle weighing method based on image and laser scanning according to claim 4, characterized in that: Extracting key features of a freight vehicle from the image data, the three-dimensional point cloud data, and the basic information comprises the following steps: Step 41: extracting visual features and road environment data from the image data; Step 42: Extracting geometric features from the 3D point cloud data; Step 43: Calculate composite features based on visual features, road environment data, and geometric features combined with basic information of the freight vehicle; Step 44: The visual features, road environment data, geometric features, freight vehicle basic information and composite features constitute key features.

6. The freight vehicle weighing method based on image and laser scanning according to claim 5, characterized in that: The extraction of visual features comprises the following sub-steps: Use semantic segmentation algorithms to identify key parts from image data; Calculate the tire-ground contact area and deformation degree from the image data; Analyze the vehicle body posture from the image data, including the lateral tilt angle and longitudinal tilt angle of the vehicle body; Extracting road surface environmental data from image data, including road surface type, road surface flatness and stiffness, road surface inclination angle and slope; The extraction of geometric features comprises: Measure the length, width, height, and wheelbase of freight vehicles from 3D point cloud data; Calculate the volume and surface area of ​​each part of the freight vehicle from 3D point cloud data; Analyze the deformation degree of the freight vehicle suspension system from 3D point cloud data; Measure the distance change between the chassis of a freight vehicle and the ground from 3D point cloud data; The calculation of composite features specifically includes: Calculate tire compression ratio based on tire specifications and tire deformation degree; Estimating load distribution based on freight vehicle weight and body posture; Calculate the deviation of the center of gravity of the freight vehicle relative to the standard state; Analyze the relationship between suspension system deformation and load.

7. The freight vehicle weighing method based on image and laser scanning according to claim 6, characterized in that: The training process of the weighing prediction model includes the following steps: Step 51: Construct a weighing sample dataset, which includes several freight vehicle samples with known weights; Step 52: Based on the weighing sample dataset, a phased training strategy is adopted to pre-train the image feature processing subnetwork, the point cloud feature processing subnetwork, the environment feature processing subnetwork, and the freight vehicle parameter processing subnetwork; Step 53: Use an end-to-end joint training method to optimize the multimodal feature fusion subnetwork and the weight prediction subnetwork to complete the training of the weight prediction model.

8. The freight vehicle weighing method based on image and laser scanning according to claim 7, characterized in that: The pre-training process includes: Use the image data and partial weight labels in the weighing sample dataset to pre-train the image feature processing sub-network; Use the 3D point cloud data and some weight labels in the weighing sample dataset to pre-train the point cloud feature processing sub-network; Use the environmental parameters and correction coefficients in the weighing sample dataset to pre-train the environmental feature processing sub-network; The freight vehicle parameter processing subnetwork is pre-trained using the freight vehicle basic information and load relationship in the weighing sample dataset.

9. The freight vehicle weighing method based on image and laser scanning according to claim 8, characterized in that: Outputting the final freight vehicle weight data comprises the following steps: Step 61: Based on the feature vector of the key feature and the weighing prediction model, generate a predicted total weight of the freight vehicle through a weight prediction subnetwork of the weighing prediction model; Step 62: Calculate the freight vehicle load capacity: the load capacity is the predicted total weight minus the freight vehicle's own weight; Step 63: Combine the final freight vehicle load and freight vehicle identification information to output freight vehicle weight data.

10. A freight vehicle weighing system based on image and laser scanning, which is used to implement the freight vehicle weighing method based on image and laser scanning according to any one of claims 1 to 9, characterized in that: It includes an image acquisition module, a laser scanning module, a freight vehicle information acquisition module, a feature extraction module, a weighing prediction module, and a weight output module; wherein each module is electrically connected; An image acquisition module collects image data of the freight vehicle through a high-resolution camera and sends the image data to a feature extraction module; A laser scanning module scans the freight vehicle using a laser scanning device to obtain three-dimensional point cloud data of the freight vehicle, and sends the three-dimensional point cloud data to the feature extraction module; The freight vehicle information acquisition module uses license plate recognition technology to identify the license plate number of the freight vehicle, retrieves the basic information of the freight vehicle from the freight vehicle management database, and sends the basic information to the feature extraction module and the weighing prediction module; a feature extraction module that receives the image data, the three-dimensional point cloud data, and the basic information, and extracts key features of the freight vehicle; the feature extraction module sends the extracted key features to the weighing prediction module and the weight output module; A weight prediction module, based on the feature types of the key features, constructs a weight prediction model consisting of an image feature processing subnetwork, a point cloud feature processing subnetwork, an environmental feature processing subnetwork, a freight vehicle parameter processing subnetwork, a multimodal feature fusion subnetwork, and a weight prediction subnetwork, and trains the weight prediction model using a pre-collected weighing sample data set; the weight prediction module sends the trained model to the weight output module; The weight output module generates the weight of the freight vehicle based on the feature vector of the key feature and the weighing prediction model, associates the weight of the freight vehicle with the freight vehicle identification information, and outputs the final freight vehicle weight data.

Images