Intelligent traffic management method and system based on multi-view perception fusion

Abstract

The invention provides a multi-view perception fusion intelligent traffic management method and system, and relates to the field of intelligent traffic management, and the method comprises the steps: carrying out the contour feature extraction of a carriage image set and a tire image set, and obtaining a carriage contour set and a tire contour set; respectively carrying out point cloud fitting on the compartment point cloud set and the tire point cloud set by utilizing the compartment contour set and the tire contour set, constructing a compartment three-dimensional model and a plurality of tire three-dimensional models according to a point cloud fitting result, and carrying out analysis based on the plurality of tire three-dimensional models to obtain a tire deformation degree; if the compartment three-dimensional model exceeds the standard compartment size or the tire deformation degree is larger than the preset deformation index, overload early warning is conducted on the target truck. The overload detection method aims at solving the technical problem that a traditional overload detection method is insufficient in detection accuracy and real-time performance, the overload detection precision can be effectively improved through data fusion of multiple sensors, and therefore the real-time performance and accuracy of overload detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation management, and in particular to an intelligent transportation management method and system for multi-perspective perception fusion. Background Art

[0002] With the rapid development of the transportation industry, trucks, as the main means of transportation, play an important role in road transportation. However, the problem of overloaded trucks remains serious in some areas, which not only exacerbates road damage but may also lead to traffic accidents and even pose a threat to vehicle safety. In order to ensure road traffic safety, many countries and regions have begun to implement detection and management measures for overloaded trucks.

[0003] Currently, traditional weighing methods usually rely on ground weighing sensors or weighing stations to detect the weight of vehicles. Although these methods can provide relatively accurate weight data, since weighing stations are usually set at fixed locations, real-time and dynamic detection cannot be achieved, and there is a situation where vehicles queue up and wait, reducing the efficiency of road traffic. Summary of the Invention

[0004] The purpose of the present invention is to provide an intelligent transportation management method and system for multi-perspective perception fusion to solve the technical problems of insufficient detection accuracy and real-time performance in traditional overloading detection methods, including:

[0005] In the first aspect, the present invention provides an intelligent transportation management method for multi-perspective perception fusion, including: collecting multi-angle data of the carriage and tires of a target truck through visual sensors and lidar to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; respectively extracting contour features from the carriage image set and the tire image set to obtain a carriage contour set and a tire contour set; using the carriage contour set and the tire contour set to respectively perform point cloud fitting on the carriage point cloud set and the tire point cloud set, constructing a carriage three-dimensional model and multiple tire three-dimensional models according to the point cloud fitting results, and analyzing the tire deformation degree based on the multiple tire three-dimensional models; if the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index, an overloading warning is issued for the target truck.

[0006] Preferably, the intelligent transportation management method for multi-perspective perception fusion further includes: randomly selecting a first carriage image from the carriage image set; pre-training a carriage contour extractor, inputting the first carriage image into the carriage contour extractor for contour feature extraction, outputting a first carriage contour, and adding it to the carriage contour set.

[0007] Preferably, the intelligent transportation management method based on multi - perspective perception fusion further includes: collecting multiple sample carriage images to construct a sample carriage image set, performing carriage contour annotation on different sample carriage images to obtain sample carriage contours, and constructing a sample carriage contour set; constructing a carriage contour extractor based on a convolutional neural network, and using the sample carriage image set and the sample carriage contour set to perform supervised training and verification on the carriage contour extractor until the expected convergence constraint is reached, thereby obtaining a trained carriage contour extractor.

[0008] Preferably, the intelligent transportation management method based on multi - perspective perception fusion further includes: randomly selecting a first tire image from the tire image set; pre - training a tire contour extractor, inputting the first tire image into the tire contour extractor for contour feature extraction, outputting a first tire contour, and adding it to the tire contour set.

[0009] Preferably, the intelligent transportation management method based on multi - perspective perception fusion further includes: using the carriage contour set to perform mapped point cloud fitting on the carriage point cloud set to obtain multiple optimal carriage point cloud fitting results, and combining according to the multiple optimal carriage point cloud fitting results to construct a carriage three - dimensional model; using the tire contour set to perform mapped point cloud fitting on the tire point cloud set to obtain multiple optimal tire point cloud fitting results, and constructing multiple tire three - dimensional models according to the multiple optimal tire point cloud fitting results.

[0010] Preferably, the intelligent transportation management method based on multi - perspective perception fusion further includes: randomly selecting a first carriage contour from the carriage contour set, and obtaining a first carriage point cloud set at the same position as the first carriage contour; performing random fitting on the first carriage point cloud set according to the first carriage contour to obtain a first point cloud fitting result, and calculating a first fitting accuracy, where the first fitting accuracy is the ratio of the number of point clouds in the first point cloud fitting result to the total number of point clouds in the first carriage point cloud set; performing random fitting on the first carriage point cloud set again according to the first carriage contour to obtain a second point cloud fitting result, and calculating a second fitting accuracy; continuing iterative fitting until a predetermined number of fittings is reached, and outputting the point cloud fitting result corresponding to the maximum fitting accuracy as the first optimal point cloud fitting result, and adding it to the multiple optimal carriage point cloud fitting results.

[0011] Preferably, the intelligent transportation management method based on multi - perspective perception fusion further includes: constructing a tire deformation analysis model through a convolutional neural network, and collecting sample data for training until convergence; using the tire deformation analysis model to perform deformation analysis on the multiple tire three - dimensional models, outputting multiple individual tire deformation degrees, and obtaining the tire deformation degree after mean calculation.

[0012] Second aspect, the present invention also provides an intelligent transportation management system for multi - perspective perception fusion, which is used to execute a multi - perspective perception fusion intelligent transportation management method as described in the first aspect, including: a multi - angle data acquisition module, which is used to perform multi - angle data acquisition on the carriage and tires of a target truck through visual sensors and lidar, so as to obtain a carriage image set, a carriage point cloud set, a tire image set and a tire point cloud set; a contour feature extraction module, which is used to perform contour feature extraction on the carriage image set and the tire image set respectively, so as to obtain a carriage contour set and a tire contour set; a three - dimensional model construction module, which is used to use the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, construct a carriage three - dimensional model and multiple tire three - dimensional models according to the point cloud fitting results, and analyze the tire deformation degree based on the multiple tire three - dimensional models; a truck overload warning module, which is used to give an overload warning to the target truck if the carriage three - dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index.

[0013] The embodiments of the present invention have the following advantages:

[0014] Perform multi - angle data acquisition on the carriage and tires of a target truck through visual sensors and lidar, so as to obtain a carriage image set, a carriage point cloud set, a tire image set and a tire point cloud set; perform contour feature extraction on the carriage image set and the tire image set respectively, so as to obtain a carriage contour set and a tire contour set; use the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, construct a carriage three - dimensional model and multiple tire three - dimensional models according to the point cloud fitting results, and analyze the tire deformation degree based on the multiple tire three - dimensional models; if the carriage three - dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index, then give an overload warning to the target truck. That is to say, through the data fusion of multiple sensors, the accuracy of overload detection can be effectively improved, thus significantly improving the real - time performance and accuracy of overload detection. Description of the Drawings

[0015] Figure 1 is the step flow chart of a multi - perspective perception fusion intelligent transportation management method of the present invention;

[0016] Figure 2 is the structural schematic diagram of an intelligent transportation management system for multi - perspective perception fusion of the present invention.

[0017] Description of the Reference Numerals:

[0018] Multi - angle data acquisition module 11, contour feature extraction module 12, three - dimensional model construction module 13, truck overload warning module 14. Detailed Embodiments

[0019] The present invention provides a smart traffic management method and system for multi-perspective perception fusion, which solves the technical problems of insufficient detection accuracy and real-time performance in traditional overloading detection methods. Through data fusion of multiple sensors, the accuracy of overloading detection can be effectively improved, thereby significantly enhancing the real-time performance and accuracy of overloading detection.

[0020] Next, the technical solutions in the present invention will be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments of the present invention. It should be understood that the present invention is not limited by the example embodiments described herein. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention. Additionally, it should be noted that for the sake of description, only the parts related to the present invention are shown in the accompanying drawings rather than all of them.

[0021] Embodiment 1, please refer to the attached Figure 1 , the present invention provides a smart traffic management method for multi-perspective perception fusion, which is applied to a smart traffic management system for multi-perspective perception fusion, and specifically includes the following steps:

[0022] Step 101: Collect multi-angle data of the carriage and tires of the target truck through a vision sensor and a lidar to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; Step 102: Extract contour features from the carriage image set and the tire image set respectively to obtain a carriage contour set and a tire contour set; Step 103: Use the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, construct a carriage three-dimensional model and multiple tire three-dimensional models according to the point cloud fitting results, and analyze the tire deformation degree based on the multiple tire three-dimensional models; Step 104: If the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index, an overloading warning is issued for the target truck.

[0023] Specifically, a visual sensor (such as a camera) and a lidar are used to comprehensively collect multi-angle data of the target truck. That is, images of the carriage are obtained from multiple angles through the visual sensor (such as a high-definition camera) to provide two-dimensional information on the appearance of the carriage; the carriage is scanned by the lidar to obtain three-dimensional point cloud data of the carriage, capturing the shape, size, and structural information of the carriage; image data of the tires is obtained through the visual sensor to provide the appearance features of the tires; the tires are scanned by the lidar to obtain three-dimensional point cloud data of the tires, helping to construct the detailed three-dimensional shape of the tires; an image set of the carriage, a point cloud set of the carriage, an image set of the tires, and a point cloud set of the tires are obtained. Then, contour feature extraction is respectively performed on the carriage image set and the tire image set, that is, contour features are extracted from the carriage image set and the tire image set through image processing algorithms to obtain a carriage contour set and a tire contour set.

[0024] Then, using the carriage contour set and the tire contour set, point cloud fitting is respectively performed on the carriage point cloud set and the tire point cloud set, and a three-dimensional model of the carriage and multiple three-dimensional models of the tires are constructed according to the point cloud fitting results. That is, using the carriage contour set and the carriage point cloud set, technologies such as point cloud registration and fitting are applied to convert the three-dimensional point cloud data of the carriage into a three-dimensional model of the carriage, and the geometric structure of the carriage is completely reconstructed; similarly, using the tire contour set and the tire point cloud set, through the point cloud fitting algorithm, a three-dimensional model of each tire is reconstructed. Further, based on the three-dimensional model of the tire obtained by fitting, the deformation degree of each tire is analyzed. The deformation degree may be related to the force and load conditions of the tire. An excessive deformation degree usually indicates an overloaded load. Then, it is checked whether the size of the three-dimensional model of the carriage exceeds the range of the standard carriage size. If the carriage size exceeds the standard, it may mean that the goods are overloaded; it is analyzed whether the deformation degree of the tire exceeds the predetermined standard deformation degree index. An excessive deformation degree of the tire may be caused by overloading, indicating that the load of the vehicle exceeds the designed load range. If the carriage size exceeds the standard or the tire deformation degree exceeds the standard, the system will trigger an overloading warning to remind the traffic management department or the driver to check the vehicle or take measures to avoid potential safety hazards caused by overloading.

[0025] Through the collaborative action of multiple sensors, accurate identification of the overloading situation of the truck is achieved. Combining the three-dimensional modeling of the carriage and the tires and the analysis of the tire deformation degree, it is possible to judge in real time and accurately whether the vehicle is overloaded; this technical solution solves the deficiencies of traditional overloading detection methods in terms of real-time performance and accuracy, and provides an efficient solution for the intelligent transportation management system.

[0026] Furthermore, the present application further includes the following steps:

[0027] Randomly select a first carriage image from the carriage image set; pre-train a carriage contour extractor, input the first carriage image into the carriage contour extractor for contour feature extraction, and output a first carriage contour, which is added to the carriage contour set.

[0028] Specifically, first, randomly select any car body image from the set of car body images as the first car body image. Then, use a pre-trained car body contour extractor (such as a convolutional neural network or other image processing algorithms), which has been trained with a large number of car body images and can accurately identify the edges and contours of the car body. Next, input the first car body image into the car body contour extractor for processing to extract the contour features in the image. Through the extractor, the system can identify and output the geometric contour in the car body image, such as parameters like the length, width, and height of the car body. The extractor will generate and output the contour data of the first car body, that is, the edges and geometric features of the car body, called the first car body contour. Then, add the extracted first car body contour to the car body contour set (i.e., the contour set of all car body images) to provide contour features for subsequent processing steps (such as point cloud fitting).

[0029] Through this step, the contour features of each freight car can be extracted from the set of car body images and gradually added to the car body contour set to form a complete geometric feature library of the car body, which provides basic data for subsequent point cloud fitting and 3D modeling, ensures that the 3D structure of the car body can be accurately reconstructed, and further helps to determine whether the freight car is overloaded.

[0030] Furthermore, the present application further includes the following steps:

[0031] Collect multiple sample car body images to construct a sample car body image set, and perform car body contour annotation on different sample car body images to obtain sample car body contours and construct a sample car body contour set. Build a car body contour extractor based on a convolutional neural network, and use the sample car body image set and the sample car body contour set to perform supervised training and verification on the car body contour extractor until the expected convergence constraint is reached, and obtain a trained car body contour extractor.

[0032] Specifically, collect multiple different sample car body images to ensure that the sample car body image set contains car body images of various different types, angles, and lighting conditions to enhance the generalization ability and adaptability of the model. Then, perform contour annotation on each collected car body image. The annotation process includes manually or semi-automatically marking the external contour of the car body (such as the edges, corners, and main features of the car body). After each sample image is annotated, it corresponds to a car body contour, that is, a sample car body contour. And the annotated car body contours are grouped together to form a complete sample car body contour set. This contour set is used for subsequent training of the car body contour extractor as the basis for supervised learning.

[0033] A carriage contour extractor is constructed using a Convolutional Neural Network (CNN). CNN has powerful feature extraction capabilities in image processing tasks and can automatically learn important edges and geometric features in carriage images. The carriage contour extractor gradually extracts the contour features of the carriage through structures such as convolutional layers, pooling layers, and fully connected layers. Using the sample carriage image set and the sample carriage contour set as input data, a supervised learning method is used to train the carriage contour extractor. During the training process, the carriage image is used as the input, and the carriage contour is used as the label. The error between the network output and the labeled contour is calculated for backpropagation to adjust the model parameters. During the training process, part of the data is used as a validation set to evaluate the accuracy and generalization ability of the training process in real-time; the role of the validation set is to prevent the model from overfitting and ensure that the carriage contour extractor can also perform well on unseen data; after multiple rounds of training and validation, until the error of the model tends to be the smallest and reaches the predetermined convergence standard, the training process of the carriage contour extractor is completed. At this time, the extractor can efficiently and accurately extract the contour features of the carriage from the carriage image.

[0034] Through this step, the construction and training process of the carriage contour extractor is completed. By collecting diverse sample carriage images and performing contour annotation, combined with supervised learning using a convolutional neural network, an accurate carriage contour extractor is trained. This extractor can efficiently extract contour information from newly input carriage images, laying a foundation for subsequent carriage point cloud fitting and 3D modeling, which provides important image processing support for truck overloading detection.

[0035] Furthermore, this application also includes the following steps:

[0036] Randomly select a first tire image from the tire image set; pre-train a tire contour extractor, input the first tire image into the tire contour extractor for contour feature extraction, and output the first tire contour, which is added to the tire contour set.

[0037] Specifically, randomly select any tire image from the tire image set as the first tire image; then use a pre-trained tire contour extractor (such as image processing methods like convolutional neural networks), which has been trained with a large number of tire images and can automatically identify and extract the contour features of the tire. Next, input the selected first tire image into the tire contour extractor. Through this extractor, the system can identify and output the contour features of the tire, such as the outer edge, tread pattern, and geometric shape of the tire; the extractor takes the contour features extracted from the first tire image as the first tire contour, that is, the edge and geometric feature data of the contour in this tire image. Add the extracted first tire contour to the tire contour set (i.e., the contour set of all tire images) to provide contour features for subsequent point cloud fitting and 3D modeling. In this step, by randomly selecting images from the tire image set and using a pre-trained tire contour extractor, the contour features of each tire image are automatically extracted. These contour features are added to the tire contour set to form a complete tire geometric feature library, providing basic data for subsequent tire point cloud fitting and 3D modeling, and further helping to determine whether the truck is overloaded.

[0038] Furthermore, the present application further includes the following steps:

[0039] Use the carriage contour set to perform mapped point cloud fitting on the carriage point cloud set to obtain multiple optimal carriage point cloud fitting results, and construct a carriage 3D model according to the combination of the multiple optimal carriage point cloud fitting results; use the tire contour set to perform mapped point cloud fitting on the tire point cloud set to obtain multiple optimal tire point cloud fitting results, and construct multiple tire 3D models according to the multiple optimal tire point cloud fitting results.

[0040] Specifically, use the already constructed carriage contour set to map the input carriage point cloud set. By matching each point in the point cloud with the contour features, the point cloud data of the carriage is fitted into the optimal point cloud result that matches the contour; this fitting process is achieved through a point cloud fitting algorithm (such as the ICP algorithm) to ensure that the point cloud data conforms as accurately as possible to the actual shape of the carriage. According to the point cloud fitting process, multiple optimal carriage point cloud fitting results are obtained, and each result represents the best match of the carriage point cloud data under different fitting strategies; combine the obtained multiple optimal carriage point cloud fitting results, and comprehensively consider the characteristics of different fitting results to construct a 3D model of the carriage, which can accurately represent the size, shape, and geometric features of the carriage.

[0041] On the other hand, using the tire contour set, the input tire point cloud set is mapped. Through a point cloud fitting algorithm (such as the ICP algorithm or other optimization methods), the tire point cloud data is best matched with the tire contour features. Similar to the fitting of the carriage, multiple optimal tire point cloud fitting results are obtained, and each fitting result represents the best match between the tire point cloud data and the contour. Then, multiple optimal tire point cloud fitting results are combined to generate a three-dimensional model of the tire. The accuracy of the tire three-dimensional model is enhanced through the combined results, ensuring that the model truly reflects the size, shape, and deformation degree of the tire. Through this step, the contour feature sets of the carriage and the tire are used to map and fit the point cloud data of the carriage and the tire respectively. Through the combination of multiple optimal fitting results, accurate three-dimensional models of the carriage and the tire are finally constructed; these three-dimensional models provide detailed geometric data support for subsequent overload detection, helping to evaluate the overload situation of the carriage and the deformation degree of the tire, thereby achieving accurate overload warning.

[0042] Furthermore, the present application further includes the following steps:

[0043] Randomly select a first carriage contour from the carriage contour set, and obtain a first carriage point cloud set at the same position as the first carriage contour; perform random fitting on the first carriage point cloud set according to the first carriage contour to obtain a first point cloud fitting result, and calculate a first fitting accuracy, where the first fitting accuracy is the ratio of the number of point clouds in the first point cloud fitting result to the total number of point clouds in the first carriage point cloud set; perform random fitting on the first carriage point cloud set again according to the first carriage contour to obtain a second point cloud fitting result, and calculate a second fitting accuracy; continue iterative fitting until a predetermined number of fitting times is reached, and output the point cloud fitting result corresponding to the maximum fitting accuracy as the first optimal point cloud fitting result, and add it to the multiple optimal carriage point cloud fitting results.

[0044] Specifically, first, randomly select a first carriage contour from the carriage contour set, and obtain a first carriage point cloud set at the same position as the first carriage contour; then, use the first carriage contour and the corresponding first carriage point cloud set to perform the first random fitting. The fitting process matches the geometric relationship between the contour and the point cloud through an algorithm to generate a fitting result, and calculate the first fitting accuracy. The fitting accuracy is measured by calculating the ratio of the number of point clouds in the fitting result to the number of point clouds in the actual point cloud set. The higher the fitting accuracy, the more accurate the fitting result; perform the second random fitting again according to the same carriage contour and point cloud set. At this time, the fitting algorithm will try to find a new fitting result and calculate the fitting accuracy of the second fitting, just like the calculation method of the first fitting accuracy.

[0045] Continue to perform random fitting multiple times iteratively. Calculate the fitting accuracy each time. Use the new fitting result as the input for each fitting to make more precise adjustments. Keep iterating until the predetermined number of fitting times is reached, that is, perform the fitting operation up to the set number of times. If the fitting accuracy no longer improves significantly during multiple fittings, the iteration can also be terminated prematurely. During all fitting processes, select the fitting result with the maximum fitting accuracy and set it as the first optimal point cloud fitting result. Add the obtained first optimal point cloud fitting result to the carriage point cloud fitting result set to provide point cloud data with higher accuracy for subsequent 3D model construction. The calculation of the fitting accuracy each time can select the point cloud data that best matches the carriage contour. Ultimately, through the point cloud fitting result with the maximum fitting accuracy, a more accurate 3D model of the carriage can be constructed to provide high-precision data support for subsequent overloading detection.

[0046] Furthermore, the present application further includes the following steps:

[0047] Construct a tire deformation analysis model through a convolutional neural network and collect sample data to train it until convergence. Use the tire deformation analysis model to perform deformation analysis on the multiple tire 3D models and output multiple individual tire deformation degrees. After calculating the mean value, obtain the tire deformation degree.

[0048] Specifically, use a convolutional neural network (CNN) to construct a tire deformation analysis model. CNN is a deep learning algorithm that performs excellently in image processing and pattern recognition and can extract multi-level features from images. The task of the tire deformation analysis model is to judge the degree of deformation based on the 3D geometric features of the tire (such as shape, texture, contour, etc.). This model learns a large number of sample data to identify the normal form and deformation state of the tire. Collect multiple real tire sample data sets, including the tire forms and their deformation states under different load conditions. Use these sample data to train the model. During the training process, input the 3D model data of the tire and output the tire deformation degree (for example, the flatness, expansion degree, etc. of the tire). Train until the model converges to ensure the accuracy and generalization ability of the model.

[0049] Then, input multiple tire three-dimensional models into the trained tire deformation analysis model. The model will perform deformation analysis based on the geometric features and loading conditions of each tire. The output of the deformation analysis is the deformation degree of each tire, which is usually reflected as the deviation of the tire's morphological changes (such as deformation amount, flatness, local expansion, etc.) from the standard morphology. After obtaining the deformation degrees of all tires, calculate the mean value of these deformation degrees. The mean value of the tire deformation degree can help us understand the overall deformation situation of the wheels and whether there is a risk of excessive deformation of any tire; obtain the calculated mean value of the tire deformation degree as the final result, which is used to judge whether the truck is at risk of overloading. If the deformation degree of a tire exceeds a preset threshold, it may indicate that the tire is at risk of overworking or being damaged. By constructing a tire deformation analysis model through a convolutional neural network and making the model converge through training sample data, it is possible to perform deformation analysis on multiple tire three-dimensional models and obtain the deformation degree of each tire; by calculating the mean value of the tire deformation degrees, the health status of the vehicle's tires as a whole can be obtained, providing a basis for overloading warning.

[0050] In summary, a multi-perspective perception fusion intelligent transportation management method provided by the present invention has the following technical effects:

[0051] Collect multi-angle data of the carriage and tires of the target truck through visual sensors and lidar to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; extract contour features from the carriage image set and the tire image set respectively to obtain a carriage contour set and a tire contour set; use the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, construct a carriage three-dimensional model and multiple tire three-dimensional models based on the point cloud fitting results, and analyze the tire deformation degree based on the multiple tire three-dimensional models; if the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index, issue an overloading warning for the target truck. That is to say, through the data fusion of multiple sensors, the accuracy of overloading detection can be effectively improved, thus significantly improving the real-time performance and accuracy of overloading detection.

[0052] Embodiment 2. Based on the same inventive concept as a multi-perspective perception fusion intelligent transportation management method in the foregoing embodiment, the present invention also provides a multi-perspective perception fusion intelligent transportation management system. Please refer to the appendix Figure 2, including: a multi-angle data acquisition module 11, configured to perform multi-angle data acquisition on the carriage and tires of the target truck through a vision sensor and a lidar, so as to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; a contour feature extraction module 12, configured to perform contour feature extraction on the carriage image set and the tire image set respectively, so as to obtain a carriage contour set and a tire contour set; a three-dimensional model construction module 13, configured to use the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, construct a carriage three-dimensional model and multiple tire three-dimensional models according to the point cloud fitting results, and analyze the tire deformation degree based on the multiple tire three-dimensional models; a truck overloading warning module 14, configured to perform an overloading warning on the target truck if the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index.

[0053] Further, the intelligent transportation management system with multi-view perception fusion is further configured to: randomly select a first carriage image from the carriage image set; pre-train a carriage contour extractor, input the first carriage image into the carriage contour extractor for contour feature extraction, output a first carriage contour, and add it to the carriage contour set.

[0054] Further, the intelligent transportation management system with multi-view perception fusion is further configured to: collect multiple sample carriage images to construct a sample carriage image set, perform carriage contour annotation on different sample carriage images to obtain sample carriage contours, and construct a sample carriage contour set; construct a carriage contour extractor based on a convolutional neural network, and use the sample carriage image set and the sample carriage contour set to perform supervised training and verification on the carriage contour extractor until the expected convergence constraint is reached, and obtain a trained carriage contour extractor.

[0055] Further, the intelligent transportation management system with multi-view perception fusion is further configured to: randomly select a first tire image from the tire image set; pre-train a tire contour extractor, input the first tire image into the tire contour extractor for contour feature extraction, output a first tire contour, and add it to the tire contour set.

[0056] Further, the intelligent transportation management system with multi-view perception fusion is further configured to: use the carriage contour set to perform mapped point cloud fitting on the carriage point cloud set to obtain multiple optimal carriage point cloud fitting results, and construct a carriage three-dimensional model according to the combination of the multiple optimal carriage point cloud fitting results; use the tire contour set to perform mapped point cloud fitting on the tire point cloud set to obtain multiple optimal tire point cloud fitting results, and construct multiple tire three-dimensional models according to the multiple optimal tire point cloud fitting results.

[0057] Furthermore, the intelligent transportation management system for multi-perspective perception fusion is also used for: randomly selecting a first carriage contour from the set of carriage contours, and obtaining a first carriage point cloud set at the same position as the first carriage contour; randomly fitting the first carriage point cloud set according to the first carriage contour to obtain a first point cloud fitting result, and calculating a first fitting accuracy, where the first fitting accuracy is the ratio of the number of point clouds in the first point cloud fitting result to the total number of point clouds in the first carriage point cloud set; randomly fitting the first carriage point cloud set according to the first carriage contour again to obtain a second point cloud fitting result, and calculating a second fitting accuracy; continuing iterative fitting until a predetermined number of fittings is reached, and outputting the point cloud fitting result corresponding to the maximum fitting accuracy as the first optimal point cloud fitting result, which is added to the multiple optimal carriage point cloud fitting results.

[0058] Furthermore, the intelligent transportation management system for multi-perspective perception fusion is also used for: constructing a tire deformation analysis model through a convolutional neural network, and collecting sample data for training until convergence; using the tire deformation analysis model to perform deformation analysis on the multiple tire three-dimensional models, and outputting multiple individual tire deformation degrees, and obtaining the tire deformation degree after calculating the average value.

[0059] The various embodiments in this specification are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The multi-perspective perception fusion intelligent transportation management method and specific examples in the foregoing Embodiment 1 are equally applicable to the multi-perspective perception fusion intelligent transportation management system in this embodiment. Through the foregoing detailed description of a multi-perspective perception fusion intelligent transportation management method, those skilled in the art can clearly know the multi-perspective perception fusion intelligent transportation management system in this embodiment. Therefore, for the sake of simplicity of the specification, it will not be elaborated here. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description in the method part.

[0060] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0061] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A smart traffic management method based on multi - perspective perception fusion, characterized in that, The method includes: Performing multi-angle data collection on the carriage and tires of the target truck through a vision sensor and a lidar to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; Performing contour feature extraction on the carriage image set and the tire image set respectively to obtain a carriage contour set and a tire contour set; Using the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, constructing a carriage three-dimensional model and multiple tire three-dimensional models according to the point cloud fitting results, and analyzing the tire deformation degree based on the multiple tire three-dimensional models; If the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index, an overloading warning is given to the target truck.

2. The intelligent transportation management method for multi-perspective perception fusion according to claim 1, characterized in that Performing contour feature extraction on the carriage image set to obtain a carriage contour set, including: Randomly selecting a first carriage image from the carriage image set; Pre-training a carriage contour extractor, inputting the first carriage image into the carriage contour extractor for contour feature extraction, outputting a first carriage contour, and adding it to the carriage contour set.

3. The intelligent transportation management method for multi-view perception fusion according to claim 2, characterized in that Pre-training a carriage contour extractor, including: Collecting multiple sample carriage images to construct a sample carriage image set, performing carriage contour annotation on different sample carriage images to obtain sample carriage contours, and constructing a sample carriage contour set; Constructing a carriage contour extractor based on a convolutional neural network, and using the sample carriage image set and the sample carriage contour set to perform supervised training and verification on the carriage contour extractor until the expected convergence constraint is reached, obtaining a trained carriage contour extractor.

4. The intelligent transportation management method for multi-perspective perception fusion according to claim 2, characterized in that, Performing contour feature extraction on the tire image set to obtain a tire contour set, including: Randomly selecting a first tire image from the tire image set; Pre-training a tire contour extractor, inputting the first tire image into the tire contour extractor for contour feature extraction, outputting a first tire contour, and adding it to the tire contour set.

5. The intelligent transportation management method for multi-perspective perception fusion according to claim 1, wherein, Using the carriage contour set and the tire contour set to perform point cloud fitting on the carriage point cloud set and the tire point cloud set respectively, constructing a carriage three-dimensional model and a tire three-dimensional model according to the point cloud fitting results, including: Using the carriage contour set to perform mapped point cloud fitting on the carriage point cloud set to obtain multiple optimal carriage point cloud fitting results, and constructing a carriage three-dimensional model according to the combination of the multiple optimal carriage point cloud fitting results; Using the tire contour set to perform mapped point cloud fitting on the tire point cloud set to obtain multiple optimal tire point cloud fitting results, and constructing multiple tire three-dimensional models according to the multiple optimal tire point cloud fitting results.

6. The intelligent transportation management method for multi-perspective perception fusion according to claim 5, characterized in that Using the carriage contour set to perform mapped point cloud fitting on the carriage point cloud set, including: Randomly selecting a first carriage contour from the carriage contour set, and obtaining a first carriage point cloud set at the same position as the first carriage contour; Performing random fitting on the first carriage point cloud set according to the first carriage contour to obtain a first point cloud fitting result, and calculating a first fitting accuracy, where the first fitting accuracy is the ratio of the number of point clouds in the first point cloud fitting result to the total number of point clouds in the first carriage point cloud set; Randomly fit the first carriage point cloud set according to the first carriage contour again to obtain a second point cloud fitting result, and calculate the second fitting accuracy; Continue iterative fitting until a predetermined number of fittings is reached, and output the point cloud fitting result corresponding to the maximum fitting accuracy as the first optimal point cloud fitting result, and add it to the multiple optimal carriage point cloud fitting results.

7. The intelligent transportation management method for multi-perspective perception fusion according to claim 1, characterized in that, Analyze the tire deformation degree based on the multiple tire three-dimensional models, including: Construct a tire deformation analysis model through a convolutional neural network, and collect sample data for training until convergence; Use the tire deformation analysis model to perform deformation analysis on the multiple tire three-dimensional models, output multiple individual tire deformation degrees, and calculate the mean value to obtain the tire deformation degree.

8. The intelligent transportation management system for multi-perspective perception fusion is characterized in that, Steps for implementing the multi-view perception fusion intelligent transportation management method according to any one of claims 1 to 7, including: A multi-angle data acquisition module for acquiring multi-angle data of the carriage and tires of the target truck through visual sensors and lidar to obtain a carriage image set, a carriage point cloud set, a tire image set, and a tire point cloud set; A contour feature extraction module for respectively extracting contour features from the carriage image set and the tire image set to obtain a carriage contour set and a tire contour set; A three-dimensional model construction module for respectively performing point cloud fitting on the carriage point cloud set and the tire point cloud set by using the carriage contour set and the tire contour set, constructing a carriage three-dimensional model and multiple tire three-dimensional models according to the point cloud fitting results, and analyzing the tire deformation degree based on the multiple tire three-dimensional models; A truck overloading warning module for warning of overloading of the target truck if the carriage three-dimensional model exceeds the standard carriage size or the tire deformation degree is greater than a predetermined deformation index.

Images