Intelligent campus material distribution method and system based on AI

Abstract

The invention discloses an AI-based smart campus material distribution method and system, and relates to the technical field of material distribution, and the method comprises the steps: collecting the multi-modal data of trolley positioning, fusing the modal data based on a graph optimization model, outputting the three-dimensional position of a trolley in a campus map, and constructing a time-varying position dynamic graph of the trolley; analyzing a difference value between a fusion position at each moment and an output position of each modal data in the dynamic graph, calculating a contribution weight of each modal data at each moment, and calculating a data contribution instability value; state increment changes of the graph optimization model in multiple rounds of iteration are recorded, and a convergence abnormal value in the iteration convergence process is calculated; and finally, comprehensively judging whether a fusion result is reliable or not by combining a convergence abnormal value and a data contribution unstable value. And if a judgment result shows that data fusion has an error, the system immediately stops the material distribution task of the current trolley, so that distribution faults or potential safety hazards caused by positioning errors are prevented, and the safety and accuracy of overall operation of the distribution system are ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of material distribution, and in particular to an AI-based smart campus material distribution method and system. Background Art

[0002] As smart campus construction continues to advance, material distribution needs are becoming increasingly frequent and diverse. Traditional manual delivery methods are not only inefficient but also face high management costs. With the development of AI technology, existing AI-based campus material distribution systems utilize delivery robots as the core, integrating AI technologies such as computer vision, LiDAR, deep learning, path planning, and multimodal perception to achieve fully automated management of material dispatch, autonomous navigation, safe obstacle avoidance, and efficient delivery within closed or semi-closed campus environments.

[0003] Especially in the process of material distribution, the car uses multiple modal data to perform real-time analysis to determine the car's position on campus, and automatically controls the car's speed according to the car's position, ensuring that the car can deliver the materials to the target in a timely manner; for example, by obtaining the car's GPS positioning, visual data, LiDAR point cloud data and other modal data, and fusing them, the car's position can be accurately determined in real time, improving the delivery robot's autonomous navigation capabilities. At the same time, it can also promptly detect whether the car has stagnated and other problems, and promptly carry out the car repair.

[0004] When positioning the vehicle, the position of the vehicle is determined by fusing multiple modal data. However, the existing vehicle positioning system has not yet judged the accuracy of the fusion of multiple modal data. It defaults to the position of the vehicle after the fusion of the vehicle positioning system is accurate, which may cause problems with the accuracy of the vehicle positioning and lead to errors in subsequent material distribution. Summary of the Invention

[0005] The purpose of the present invention is to solve the above-mentioned problems and provide an AI-based smart campus material distribution method and system.

[0006] In a first aspect of the present invention, a method for smart campus material distribution based on AI is first proposed, the method comprising:

[0007] Collect multiple modal data for vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data;

[0008] Based on the graph optimization model, multiple modal data are integrated to output the three-dimensional position of the car and construct a dynamic graph of the car's position evolving over time.

[0009] Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value;

[0010] Extract the 3D coordinates at each moment from the dynamic graph, obtain the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculate the convergence anomaly of the data fusion;

[0011] According to the convergence outlier value and the data contribution instability value, it is judged whether the data fusion result of the three-dimensional position of the vehicle positioning system is wrong. If the data fusion result is wrong, the material distribution is stopped immediately.

[0012] 2. The AI-based smart campus material distribution method according to claim 1 is characterized in that the steps of analyzing the contribution weight value changes of each modal data in the dynamic graph and calculating the data contribution instability value are as follows:

[0013] Extract the three-dimensional coordinates of the position at each moment from the dynamic graph and record them as the first coordinates corresponding to each moment;

[0014] The three-dimensional coordinates of the car position corresponding to each modal data at each moment are marked as the second coordinates, and the difference between each second coordinate and the first coordinate is calculated. and calculate Euclidean distance

[0015] The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10 -6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment;

[0016] Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1;

[0017] According to the dynamic graph, the fluctuation value of the contribution weight value of each modal data over time is analyzed. According to the fluctuation value of the contribution weight value of all modal data over time, the data contribution instability value is calculated. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

[0018] 3. The AI-based smart campus material distribution method according to claim 1, wherein the step of calculating the convergence outlier value of the data fusion is:

[0019] When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained, and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization;

[0020] The Euclidean norm of the state increment vector after each round of iterative optimization is subtracted from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and the difference is divided by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio;

[0021] Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization.

[0022] All outliers are added together and the sum is used as the converged outlier value for data fusion.

[0023] 4. The AI-based smart campus material distribution method according to claim 1 is characterized in that the step of determining whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is erroneous based on the convergence outlier and the data contribution instability value, and immediately stopping the material distribution if the data fusion result is erroneous, comprises:

[0024] The convergence outlier value and the data contribution instability value are weighted and summed to obtain the fusion result judgment index;

[0025] The fusion result judgment index is compared with the preset fusion result judgment index threshold, and the data fusion result of the three-dimensional position of the car positioning system is checked based on the comparison result to see if there is any error.

[0026] 5. The AI-based smart campus material distribution method according to claim 4 is characterized in that the step of determining whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is erroneous based on the comparison result is:

[0027] If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the car positioning system about the current three-dimensional position of the car is wrong, and the output three-dimensional coordinates cannot accurately locate the current position of the car, and the material distribution is stopped immediately;

[0028] If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car positioning system about the current three-dimensional position of the car is small, and the output three-dimensional coordinates can accurately locate the current position of the car and continue material distribution.

[0029] In a second aspect of the present invention, an AI-based smart campus material distribution system is proposed, comprising:

[0030] Data acquisition module: collects multiple modal data integrated during vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data;

[0031] Animation graph construction module: This module fuses multiple modal data based on a graph optimization model, outputs the 3D position of the car, and constructs a dynamic graph of the car's position evolving over time.

[0032] Contribution instability module: Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value;

[0033] Convergence anomaly module: Extracts the 3D coordinates at each moment from the dynamic graph, obtains the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculates the convergence anomaly value of the data fusion;

[0034] Distribution module: Determine whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is wrong based on the convergence outlier value and the data contribution instability value. If the data fusion result is wrong, the material distribution will be stopped immediately.

[0035] 7. The AI-based smart campus material distribution system according to claim 6, wherein the unstable contribution module comprises:

[0036] First coordinate module: extracts the three-dimensional coordinates of the position at each moment from the dynamic graph and records them as the first coordinates corresponding to each moment;

[0037] Distance module: Mark the three-dimensional coordinates of the car position corresponding to each modal data at each moment as the second coordinate, and calculate the difference between each second coordinate and the first coordinate and calculate Euclidean distance

[0038] Normalization module: The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10-6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment;

[0039] Contribution weight value module: Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1;

[0040] Data contribution instability value module: Analyze the fluctuation value of the contribution weight value of each modal data over time based on the dynamic graph, and calculate the data contribution instability value based on the fluctuation value of the contribution weight value of all modal data over time. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

[0041] 8. The AI-based smart campus material distribution system according to claim 6 is characterized in that the convergence exception module includes

[0042] Computational module: When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization;

[0043] Reduction ratio module: subtract the Euclidean norm of the state increment vector after each round of iterative optimization from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and divide the difference by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio;

[0044] Comparison module: Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization.

[0045] Convergence outlier module: Add all outliers and use the sum as the convergence outlier of data fusion.

[0046] 9. The AI-based smart campus material distribution system according to claim 6, wherein the distribution module comprises:

[0047] Judgment index module: performs weighted summation of convergence outliers and data contribution instability values ​​to obtain the fusion result judgment index;

[0048] Delivery status module: Compare the fusion result judgment index with the preset fusion result judgment index threshold, and determine whether there is any error in the data fusion result of the three-dimensional position of the vehicle positioning system based on the comparison result.

[0049] 10. The AI-based smart campus material distribution system according to claim 9, wherein the distribution status module comprises:

[0050] First state module: If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the vehicle positioning system regarding the vehicle's current three-dimensional position is wrong, and the output three-dimensional coordinates cannot accurately locate the vehicle's current position, and material distribution is stopped immediately;

[0051] Second state module: If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car positioning system about the current three-dimensional position of the car is small, and the output three-dimensional coordinates can accurately locate the current position of the car and continue material distribution.

[0052] Beneficial effects of the present invention:

[0053] The present invention proposes an AI-based smart campus material distribution method and system. The method collects multiple modal data integrated during vehicle positioning and fuses the multiple modal data based on a graph optimization model to output the three-dimensional position of the vehicle and construct a dynamic graph of the vehicle's position evolving over time. The method analyzes the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determines the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyzes the change in the contribution weight value of each modal data in the dynamic graph, and calculates the data contribution instability value. The method extracts the three-dimensional coordinates at each moment from the dynamic graph, obtains the incremental change of the three-dimensional coordinates at each moment after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculates the convergence outlier of the data fusion. The method determines whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is erroneous based on the convergence outlier and the data contribution instability value. If the data fusion result is erroneous, the material distribution is stopped immediately. In this way, the accuracy of the data fusion of the vehicle positioning system can be judged, ensuring that the vehicle position after fusion of the vehicle positioning system is accurate, and reducing the probability of errors in subsequent vehicle material distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The present invention will be further described below with reference to the accompanying drawings.

[0055] Figure 1 This is a flowchart of an AI-based smart campus material distribution method;

[0056] Figure 2 This is a framework diagram of an AI-based smart campus material distribution system. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0058] The embodiment of the present invention provides an AI-based smart campus material distribution method. Figure 1 , Figure 1 A flowchart of an AI-based smart campus material distribution method provided in an embodiment of the present invention. The method includes the following steps:

[0059] Collect multiple modal data for vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data;

[0060] Based on the graph optimization model, multiple modal data are integrated to output the three-dimensional position of the car and construct a dynamic graph of the car's position evolving over time.

[0061] Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value;

[0062] Extract the 3D coordinates at each moment from the dynamic graph, obtain the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculate the convergence anomaly of the data fusion;

[0063] According to the convergence outlier value and the data contribution instability value, it is judged whether the data fusion result of the three-dimensional position of the vehicle positioning system is wrong. If the data fusion result is wrong, the material distribution is stopped immediately.

[0064] Based on an AI-based smart campus material distribution method provided by an embodiment of the present invention, through the above method, when the position of the vehicle is determined by fusing multiple modal data, the accuracy of the data fusion of the vehicle positioning system can be judged, ensuring that the position of the vehicle after the fusion of the vehicle positioning system is accurate, and reducing the probability of errors in subsequent vehicle material distribution.

[0065] In one embodiment, multiple modal data are collected and fused when the vehicle is positioned, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data;

[0066] In one implementation, the car collects multimodal sensor data in real time at each time t:

[0067] By collecting GPS three-dimensional coordinates, usually longitude loN, latitude lat and altitude alt, expressed as: P GPS =[lon,lat,alt], where P GPS The original three-dimensional geographic coordinates of the car from the GPS sensor are mapped to the local three-dimensional coordinate system of the campus map using the coordinate conversion function: P local =T geo2local (P GPS ), P local It is the three-dimensional position of the car converted to the local coordinate system, and the unit is usually meters. geo2local (·) The projection transformation from the geographic coordinate system to the local plane coordinate system is usually the UTM projection or geodetic coordinate system transformation, which converts it into the three-dimensional coordinates in the metric plane coordinate system;

[0068] The point cloud data of the current frame at time t is collected by LiDAR, and the cloud data is recorded as p i =(x i ,y i ,z i ), p i is the 3D coordinate of the i-th point relative to the LiDAR coordinate system, N is the number of points in the point cloud, and through a point cloud registration algorithm such as the ICP algorithm or the NDT algorithm, the current point cloud is matched with the pre-built campus map point cloud, and the minimum error transformation matrix between the point clouds is calculated. The 3D position and posture of the car relative to the map are extracted from it, and the 3D coordinates of the car in the map coordinate system are output; the rigid body transformation from the current point cloud to the existing campus map point cloud is calculated: W t is the rotation matrix of the car's posture at time t, v t is the position vector of the car at time t, that is, the three-dimensional coordinate, D t is a 4x4 transformation matrix, including the rotation matrix W t ∈SO(3) and the translation vector v t ∈R 3 ; R 3 represents a three-dimensional real space, which means that the position is composed of three values, representing the X, Y, and Z positions of the car in the map coordinate system; SO(3) represents a 3*3 rotation matrix; solve T t Minimize the registration error function: q iis the point cloud with p i Corresponding matching points, ‖·‖ is the Euclidean distance, minimizing E(T t ) Get the current three-dimensional position of the car in the map coordinate system;

[0069] The camera captures continuous frame images, extracts image feature points and performs feature matching, and uses visual odometry or visual SLAM algorithms (such as ORB-SLAM) to estimate the relative motion of the car between adjacent frames, and then calculates the three-dimensional position and posture changes of the car in the local coordinate system of the map, realizing visual positioning and outputting three-dimensional coordinates; the visual sensor extracts feature points through continuous image frames Perform inter-frame feature matching and calculate camera motion. Let the frame pose corresponding to the previous moment t-1 be T t-1 , the current frame pose is T t , solve T by PnP or optical flow estimation t , T t The current camera pose includes the 3D position and translation of the car. After global map optimization, such as visual SLAM, the 3D position of the car in the map coordinate system is obtained.

[0070] Ultrasonic sensors measure the distance between the car and surrounding obstacles. Combined with the known locations of obstacles in the campus environment, the car's position is locally corrected through distance constraints. Multi-point distance measurement combined with triangulation or filtering methods is used to calculate the car's relative three-dimensional position in the local map.

[0071] In one embodiment, a graph optimization model is used to fuse multiple modal data, output the three-dimensional position of the car, and construct a dynamic graph of the car's position evolving over time;

[0072] In one implementation method, after completing the individual positioning estimation of each modal sensor, a preliminary estimate of the three-dimensional position of the car under each mode is obtained. Multi-source information is fused through a graph optimization model to obtain a globally consistent three-dimensional position of the car. The position estimates and their errors output by each modal are converted into constraints in the optimization graph, and a nonlinear least squares optimization model is constructed. The optimal position is solved through multiple rounds of iterations.

[0073] Establish optimization variables and set the state variables of the car at the current moment to be: where Q t ∈R 3 , represents the three-dimensional position of the car in the local coordinate system of the map, that is, the three-dimensional coordinates to be solved in the end, in meters; A t ∈SO(3), represents the posture rotation matrix of the car in the map coordinate system;

[0074] In the graph optimization model, each sensor data contributes an independent error constraint term:

[0075] GPS error term: GPS provides geographic coordinates, which are converted into p by coordinate conversion function. GPS , its error term e GPS For: e GPS =Q t -p GPS ;

[0076] LiDAR point cloud data error term e LiDAR For: e LiDAR =Q t -p LiDAR ;p LiDAR is the three-dimensional coordinate output by LiDAR point cloud data;

[0077] Visual data error term e Vis For: e Vis =Q t -p Vis ;p Vis is the three-dimensional coordinate output through visual data;

[0078] Assume that the measured distance to obstacle k is d k , the known position of the obstacle in the map is o k , then the error term e US,k For: e US,k =(‖Q t -o k ‖-d k );

[0079] All error terms are combined to form the overall optimization objective function:

[0080]

[0081] Where Ω is the inverse covariance matrix of each error term, i.e., the information matrix, which measures the confidence weight of each data source on the final fusion result; 2 Ω represents the weighted square error;

[0082] Use iterative optimization algorithms, such as Gauss-Newton and Levenberg-Marquardt, to perform multiple rounds of optimization. The update formula for the uth round of iteration is: Where Δx t is the state increment obtained in the current u-th iteration; is the state estimate of the uth round; the iteration termination condition is the increment ‖Δx t ‖converges below the threshold and finally outputs the fused three-dimensional coordinates of the car in the campus map.

[0083] It should be noted that in practical applications, a car independently estimates its 3D position using multiple sensors, including vision, LiDAR, GPS, and ultrasound. These initial estimates often suffer from measurement errors and environmental adaptability differences. Based on a graph optimization model, the position information of each sensor and its corresponding error constraints are transformed into edges in an optimization graph. By constructing a nonlinear least-squares objective function, this model leverages the advantages of multi-source data for joint optimization. Specifically, the model iteratively updates state variables—the car's 3D position and posture—to continuously reduce the overall error and achieve globally consistent position estimates. For example, in iteration u, the position is updated using the current estimate and state increments until the increments converge to a preset threshold, ensuring stable and accurate results. This fusion approach significantly improves positioning accuracy and robustness, overcoming the limitations of single sensors susceptible to environmental interference or data loss. It leverages the complementary advantages of different sensors in terms of spatial coverage, ranging accuracy, and real-time response, providing reliable 3D position information for the car's autonomous navigation and path planning in complex campus environments. By fusion-enhancing multi-source data using a graph optimization model, the advantages and complementarities of each sensor can be fully utilized: GPS provides an absolute positioning reference, vision and LiDAR compensate for GPS's shortcomings in obstructed or weak signal areas, and ultrasound provides local constraints for close-range positioning of obstacles. The graph optimization fusion model systematically balances the errors and uncertainties of each sensor's data, effectively reducing the error propagation of a single sensor and significantly improving the vehicle's positioning accuracy. This makes it particularly suitable for high-reliability, real-time positioning tasks in complex, semi-enclosed environments like campuses.

[0084] In one embodiment, the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data is analyzed to determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, and the contribution weight value change of each modal data in the dynamic graph is analyzed to calculate the data contribution instability value;

[0085] In one embodiment, the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data is analyzed to determine the contribution weight of each modal data in the three-dimensional coordinates of the position at each moment:

[0086] It should be noted that the above-mentioned graph-based optimization model is used to fuse multiple modal data and output the three-dimensional position of the car on the campus in real time, forming a dynamic position graph of the car's position evolving over time;

[0087] In one implementation, the steps of analyzing the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determining the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, and analyzing the change in the contribution weight value of each modal data in the dynamic graph, and calculating the data contribution instability value are as follows:

[0088] Extract the three-dimensional coordinates of the position at each moment from the dynamic graph and record them as the first coordinates corresponding to each moment;

[0089] The three-dimensional coordinates of the car position corresponding to each modal data at each moment are marked as the second coordinates, and the difference between each second coordinate and the first coordinate is calculated. Where, represents the first coordinate at time t, Indicates the second coordinate corresponding to the e-th modal data at the t-th time, represents the difference between the second coordinate and the first coordinate corresponding to the e-th modal data at time t; and calculates Euclidean distance The calculation formula is:

[0090] The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10 -6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment;

[0091] Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1;

[0092] According to the dynamic graph, the fluctuation value of the contribution weight value of each modal data over time is analyzed. According to the fluctuation value of the contribution weight value of all modal data over time, the data contribution instability value is calculated. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, and its value is 4. It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

[0093] It should be noted that the data contribution instability value is an indicator used to measure the fluctuation range of the contribution weight of multimodal sensor data in the fusion process. It reflects the change in the contribution of different modal data in the time series, that is, the stability of the weight of a certain modal data in the fusion positioning result. If the data contribution instability value is too large, it means that the contribution weight of the modal data fluctuates more violently over time, which means that the quality of the data source or the consistency in the fusion process has large fluctuations, which may be affected by external environmental interference, sensor anomalies or insufficient adaptability of the algorithm model. Specifically, when the car positioning system fuses multiple modal data, the contribution weight of each modal data should ideally be relatively stable, because stable contribution weights indicate that each sensor data maintains relatively reliable and consistent positioning information support throughout the fusion process. If the contribution weight fluctuates significantly, it means that at certain moments, the positioning results of a certain modal data deviate significantly from the fusion results, forcing the system to dynamically adjust the weight of that data. This instability will reduce the overall accuracy and reliability of the fused positioning results. Moreover, the contribution weight fluctuation itself will also affect the system's integration and accumulation of historical information of each modal data. When the contribution instability value is high, it means that the system cannot form a stable and reliable fusion trajectory in terms of temporal continuity, resulting in large jumps or drifts in the overall positioning output, which in turn reduces the overall accuracy and consistency of the final three-dimensional positioning results. In other words, the greater the data contribution instability value, the worse the fusion system's ability to adapt to data noise, sensor anomalies, and scene changes, the weaker the robustness and fault tolerance of the fusion algorithm, and the lower the accuracy and availability of the final output of the car's three-dimensional positioning coordinates. This will reduce the accuracy of the car's positioning system and cause chaos in the subsequent distribution of campus materials.

[0094] It should be noted that the benefit of analyzing the data contribution instability value for judging the accuracy of the car positioning system's output of the car's three-dimensional coordinates by fusing multiple modal data is as follows: the data contribution instability value can reveal whether the positioning system relies on certain unstable data sources in the long-term positioning process, and whether it frequently adjusts data weights to compensate for the lack of information or quality degradation of certain sensors by analyzing the time series fluctuations of the contribution weights of each modal data, thereby reflecting in more depth whether the internal coordination mechanism of the fusion algorithm is reliable; this evaluation method can not only serve as a priori indicator or health monitoring basis for positioning accuracy, but also provide data support for subsequent anomaly detection, adaptive weight update, redundant data redundancy analysis, etc., further improving the reliability, continuity and intelligent control capabilities of the positioning system in complex environments.

[0095] In one embodiment, the three-dimensional coordinates at each moment are extracted from the dynamic graph, and the incremental changes of the three-dimensional coordinates at each moment after each round of iterative optimization are obtained when multimodal data fusion is performed based on a graph optimization model, and the convergence outlier of the data fusion is calculated;

[0096] Specifically, when multimodal data fusion is performed through the graph optimization model, the difference between the state increment after each round of iterative optimization and the state increment of the previous round should be gradually smaller, and the closer to the last few rounds of iterations, the state increment is almost close to 0, so that the three-dimensional coordinates of the fused car finally solved are closer to the optimal solution; however, the difference between the state increment after each round of iterative optimization and the state increment of the previous round is not as mentioned above, even if the increment ‖Δx t ‖ converges below the threshold and meets the iteration termination condition, then the final solved fused three-dimensional coordinates of the car are still not the optimal solution. The lower the accuracy of the car positioning system for the fusion of multiple modal data and the output of the car's three-dimensional coordinates, this is because it shows that the optimization process may be disturbed by noise data, abnormal observations or inconsistent sensor information, causing the algorithm to stagnate near the local optimal solution and fail to find the global optimal solution; in addition, the conflict or inaccuracy between sensor data will make the shape of the optimization objective function complex, and oscillation or non-monotonic decline will occur during the iteration process, affecting the stability and accuracy of the fusion result; therefore, even if the formal convergence conditions are met, the final output fused three-dimensional coordinates may still have deviations, reflecting that the car positioning system is insufficient in data quality and robustness of the optimization algorithm when fusing multimodal data, thereby reducing the overall accuracy and reliability of positioning.

[0097] Therefore, by calculating the convergence outlier of data fusion, we can further judge whether there is a problem with the accuracy of the three-dimensional coordinates output by the multimodal data fusion of the car positioning system;

[0098] In one implementation, the steps for calculating the convergence outlier value of data fusion are:

[0099] When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained, and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization;

[0100] The Euclidean norm of the state increment vector after each round of iterative optimization is subtracted from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and the difference is divided by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio. The calculation formula is: Where, YH m YM is the Euclidean norm of the state increment vector after the optimization of the mth iteration, m-1 is the Euclidean norm of the state increment vector after the m-1th round of iterative optimization, HY m is the reduction ratio after the optimization of the mth round of iteration;

[0101] Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization.

[0102] All outliers are added together and the sum is used as the converged outlier value for data fusion.

[0103] It should be noted that the convergence outlier is a metric used to measure the overall convergence stability and efficiency of a graph optimization algorithm during multimodal data fusion. It reflects whether the changes in the state update amplitude during successive iterations of a graph optimization algorithm conform to the theoretically expected smooth and gradual convergence trend. Normally, in each iteration of a graph optimization algorithm, as the state gradually approaches the global optimal solution, the Euclidean norm of the state increment vector should decrease. In other words, the optimization adjustment amplitude should decrease with each iteration until it approaches zero. The convergence outlier compares the changing trends of the state increments between two consecutive iterations to detect abnormal fluctuations in the convergence process. If the Euclidean norm of the state increment in one iteration actually increases compared to the previous iteration (i.e., the decrease ratio is greater than zero), it indicates that the optimization round failed to effectively approach the optimal solution. This may be due to noise interference, model mismatch, data inconsistency, or observation outliers, causing the optimization process to oscillate, jump, or freeze in certain areas. If such abnormal fluctuations occur frequently throughout the optimization process, the accumulated convergence outliers will continue to increase, which means that the algorithm has made more unstable adjustments on the overall convergence path, causing the final fusion output of the car's three-dimensional position to deviate from the true position, resulting in a decrease in the overall positioning accuracy and reliability of the system. In short, the larger the convergence outlier, the more likely the system fusion algorithm is to be affected by abnormal data during the fusion process, and it is unable to smoothly and stably approach the global optimum, and the final output position estimation deviation and uncertainty also increase. Therefore, the convergence outlier can not only indirectly reflect the adaptability and robustness of the fusion algorithm to multimodal data conflicts, but also serve as a sensitivity indicator for diagnosing hidden anomalies in the multimodal fusion process, helping to discover in advance the potential risks and accuracy risks that may exist in the positioning system in actual complex scenarios.

[0104] In one embodiment, the steps of determining whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is erroneous based on the convergence outlier and the data contribution instability value, and immediately stopping the material distribution if the data fusion result is erroneous, are as follows:

[0105] The convergence outlier value and the data contribution instability value are weighted and summed to obtain the fusion result judgment index. The calculation formula of the fusion result judgment index is: Where vgh is the fusion result judgment index, GTY and FY are the convergence outlier and data contribution instability values ​​respectively;

[0106] The fusion result judgment index is compared with the preset fusion result judgment index threshold, and the data fusion result of the three-dimensional position of the car positioning system is checked based on the comparison result to see if there is any error.

[0107] It should be noted that, in general, the weight values ​​of convergence outliers and data contribution unstable values ​​are the same, and the sum of the weights is 1.

[0108] In one embodiment, the steps of comparing the fusion result judgment index with a preset fusion result judgment index threshold and determining whether an error occurs in the data fusion result of the vehicle positioning system regarding the three-dimensional position based on the comparison result are as follows:

[0109] If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the car positioning system about the current three-dimensional position of the car is wrong, and the output three-dimensional coordinates cannot accurately locate the current position of the car, and the material distribution is stopped immediately;

[0110] If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car positioning system about the current three-dimensional position of the car is small, and the output three-dimensional coordinates can accurately locate the current position of the car and continue material distribution.

[0111] It should be noted that when the fusion result judgment index is not less than the preset threshold, it indicates that the current multimodal fusion process may have experienced significant error fluctuations, increased instability, or accumulated convergence anomalies. This means that the current output 3D coordinates may deviate from the vehicle's true position, making it unable to provide reliable navigation and positioning support for the material distribution system. Continuing the delivery mission may cause the vehicle to deviate, collide, or fail. The system must immediately issue a stop command and wait for further positioning confirmation or troubleshooting. When the fusion result judgment index is less than the preset threshold, it indicates that the data fusion process is generally stable, with high consistency between the multimodal data and smooth convergence. The output 3D coordinates basically accurately reflect the vehicle's current true position in the campus environment, and the system can allow the vehicle to continue safely executing the material distribution mission along the established route. Through this dynamic control mechanism based on the fusion result judgment index, the system can automatically monitor positioning reliability in real time and make intelligent decisions in complex environments, data noise interference, or temporary abnormal conditions, effectively improving the safety, stability, and accuracy of the entire AI campus material distribution system in actual operation, and reducing the delivery risks and system failure probability caused by positioning anomalies.

[0112] Based on the same inventive concept, the present invention also provides an AI-based smart campus material distribution system. Figure 2 , Figure 2A framework diagram of an AI-based smart campus material distribution system provided in an embodiment of the present invention, the system includes:

[0113] Data acquisition module: collects multiple modal data integrated during vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data;

[0114] Animation graph construction module: This module fuses multiple modal data based on a graph optimization model, outputs the 3D position of the car, and constructs a dynamic graph of the car's position evolving over time.

[0115] Contribution instability module: Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value;

[0116] Convergence anomaly module: Extracts the 3D coordinates at each moment from the dynamic graph, obtains the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculates the convergence anomaly value of the data fusion;

[0117] Distribution module: Determine whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is wrong based on the convergence outlier value and the data contribution instability value. If the data fusion result is wrong, the material distribution will be stopped immediately.

[0118] An AI-based smart campus material distribution system provided by an embodiment of the present invention can, through the above method, determine the position of the vehicle by fusing multiple modal data, and judge the accuracy of the data fusion of the vehicle positioning system, thereby ensuring that the position of the vehicle after the fusion of the vehicle positioning system is accurate, thereby reducing the probability of errors in subsequent vehicle material distribution.

[0119] In one embodiment, contributing an unstable module includes:

[0120] First coordinate module: extracts the three-dimensional coordinates of the position at each moment from the dynamic graph and records them as the first coordinates corresponding to each moment;

[0121] Distance module: Mark the three-dimensional coordinates of the car position corresponding to each modal data at each moment as the second coordinate, and calculate the difference between each second coordinate and the first coordinate and calculate Euclidean distance

[0122] Normalization module: The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10-6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment;

[0123] Contribution weight value module: Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1;

[0124] Data contribution instability value module: Analyze the fluctuation value of the contribution weight value of each modal data over time based on the dynamic graph, and calculate the data contribution instability value based on the fluctuation value of the contribution weight value of all modal data over time. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

[0125] In one embodiment, the convergence exception module includes

[0126] Computational module: When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization;

[0127] Reduction ratio module: subtract the Euclidean norm of the state increment vector after each round of iterative optimization from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and divide the difference by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio;

[0128] Comparison module: Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization.

[0129] Convergence outlier module: Add all outliers and use the sum as the convergence outlier of data fusion.

[0130] In one embodiment, the delivery module includes:

[0131] Judgment index module: performs weighted summation of convergence outliers and data contribution instability values ​​to obtain the fusion result judgment index;

[0132] Delivery status module: Compare the fusion result judgment index with the preset fusion result judgment index threshold, and determine whether there is any error in the data fusion result of the three-dimensional position of the vehicle positioning system based on the comparison result.

[0133] In one embodiment, the delivery status module includes:

[0134] First state module: If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the vehicle positioning system regarding the vehicle's current three-dimensional position is wrong, and the output three-dimensional coordinates cannot accurately locate the vehicle's current position, and material distribution is stopped immediately;

[0135] Second state module: If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car's current three-dimensional position by the car positioning system is small, and the output three-dimensional coordinates can accurately locate the current position of the car, and continue material distribution.

[0136] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A smart campus material distribution method based on AI, characterized in that: The following steps are involved: Collect multiple modal data for vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data; Based on the graph optimization model, multiple modal data are integrated to output the three-dimensional position of the car and construct a dynamic graph of the car's position evolving over time. Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value; Extract the 3D coordinates at each moment from the dynamic graph, obtain the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculate the convergence anomaly of the data fusion; According to the convergence outlier value and the data contribution instability value, it is judged whether the data fusion result of the three-dimensional position of the vehicle positioning system is wrong. If the data fusion result is wrong, the material distribution is stopped immediately.

2. The AI-based smart campus material distribution method according to claim 1 is characterized in that: Analyze the changes in the contribution weight value of each modal data in the dynamic graph and calculate the steps of the data contribution instability value as follows: Extract the three-dimensional coordinates of the position at each moment from the dynamic graph and record them as the first coordinates corresponding to each moment; The three-dimensional coordinates of the car position corresponding to each modal data at each moment are marked as the second coordinates, and the difference between each second coordinate and the first coordinate is calculated. and calculate Euclidean distance The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10 -6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment; Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1; According to the dynamic graph, the fluctuation value of the contribution weight value of each modal data over time is analyzed. According to the fluctuation value of the contribution weight value of all modal data over time, the data contribution instability value is calculated. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

3. The AI-based smart campus material distribution method according to claim 1 is characterized in that: The steps of calculating the convergence outlier value of the data fusion are: When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained, and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization; The Euclidean norm of the state increment vector after each round of iterative optimization is subtracted from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and the difference is divided by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio; Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization. All outliers are added together and the sum is used as the converged outlier value for data fusion.

4. The AI-based smart campus material distribution method according to claim 1 is characterized in that: Based on the convergence outlier and the data contribution instability value, it is judged whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is wrong. If the data fusion result is wrong, the steps to immediately stop the material distribution are as follows: The convergence outlier value and the data contribution instability value are weighted and summed to obtain the fusion result judgment index; The fusion result judgment index is compared with the preset fusion result judgment index threshold, and the data fusion result of the three-dimensional position of the car positioning system is checked based on the comparison result to see if there is any error.

5. The AI-based smart campus material distribution method according to claim 4 is characterized in that: According to the comparison results, the steps to check whether the data fusion results of the three-dimensional position of the car positioning system are wrong are as follows: If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the car positioning system about the current three-dimensional position of the car is wrong, and the output three-dimensional coordinates cannot accurately locate the current position of the car, and the material distribution is stopped immediately; If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car positioning system about the current three-dimensional position of the car is small, and the output three-dimensional coordinates can accurately locate the current position of the car and continue material distribution.

6. An AI-based smart campus material distribution system, characterized by: The system comprises: Data acquisition module: collects multiple modal data integrated during vehicle positioning, including visual data, LiDAR point cloud data, ultrasonic data, and GPS data; Animation graph construction module: This module fuses multiple modal data based on a graph optimization model, outputs the 3D position of the car, and constructs a dynamic graph of the car's position evolving over time. Contribution instability module: Analyze the difference between the three-dimensional coordinates of the position at each moment in the dynamic graph and the three-dimensional coordinates corresponding to each modal data, determine the contribution weight value of each modal data in the three-dimensional coordinates of the position at each moment, analyze the change in the contribution weight value of each modal data in the dynamic graph, and calculate the data contribution instability value; Convergence anomaly module: Extracts the 3D coordinates at each moment from the dynamic graph, obtains the incremental changes after each round of iterative optimization when performing multimodal data fusion based on the graph optimization model, and calculates the convergence anomaly value of the data fusion; Distribution module: Determine whether the data fusion result of the vehicle positioning system regarding the three-dimensional position is wrong based on the convergence outlier value and the data contribution instability value. If the data fusion result is wrong, the material distribution will be stopped immediately.

7. The AI-based smart campus material distribution system according to claim 6 is characterized in that: The contribution unstable module includes: First coordinate module: extracts the three-dimensional coordinates of the position at each moment from the dynamic graph and records them as the first coordinates corresponding to each moment; Distance module: Mark the three-dimensional coordinates of the car position corresponding to each modal data at each moment as the second coordinate, and calculate the difference between each second coordinate and the first coordinate and calculate Euclidean distance Normalization module: The error reciprocal normalization method is used to calculate the reciprocal sum of the differences of all types of data: In the formula, ∈ is a minimum constant to prevent division by zero, and its value is 10 -6 ;ED t is the inverse sum of the differences of all e types of modal data at time t, is the Euclidean distance corresponding to the difference between the second coordinate and the first coordinate of the e-th type of data at the t-th moment; Contribution weight value module: Calculate the contribution weight value of each modal data at time t. The calculation formula is: Where, is the contribution weight value of the e-th type of data at the t-th moment; the sum of the contribution weight values ​​of all modal data is 1; Data contribution instability value module: Analyze the fluctuation value of the contribution weight value of each modal data over time based on the dynamic graph, and calculate the data contribution instability value based on the fluctuation value of the contribution weight value of all modal data over time. The calculation formula is: In the formula, B represents the total number of moments, A represents the total number of modal data, It represents the fluctuation value of the weight value of the contribution of the e-th modal data over time, and GTY represents the unstable value of the data contribution.

8. The AI-based smart campus material distribution system according to claim 6 is characterized in that: The convergence exception module includes Computational module: When the three-dimensional coordinates at each moment are fused through multimodal data based on the graph optimization model, the state increment vector after each round of iterative optimization is obtained and the Euclidean norm of the state increment vector is calculated to measure the magnitude of the iterative adjustment after this round of optimization; Reduction ratio module: subtract the Euclidean norm of the state increment vector after each round of iterative optimization from the Euclidean norm of the state increment vector after the previous round of iterative optimization, and divide the difference by the Euclidean norm of the state increment vector after the previous round of iterative optimization to obtain the reduction ratio; Comparison module: Compare the reduction ratio after each round of iterative optimization with 0. If the reduction ratio is greater than 0, the reduction ratio is regarded as the abnormal value after this round of iterative optimization. Convergence outlier module: Add all outliers and use the sum as the convergence outlier of data fusion.

9. The AI-based smart campus material distribution system according to claim 6 is characterized in that: The distribution module includes: Judgment index module: performs weighted summation of convergence outliers and data contribution instability values ​​to obtain the fusion result judgment index; Delivery status module: Compare the fusion result judgment index with the preset fusion result judgment index threshold, and determine whether there is any error in the data fusion result of the three-dimensional position of the vehicle positioning system based on the comparison result.

10. The AI-based smart campus material distribution system according to claim 9 is characterized in that: The delivery status module includes: First state module: If the fusion result judgment index is not less than the preset fusion result judgment index threshold, it means that the data fusion result of the vehicle positioning system regarding the vehicle's current three-dimensional position is wrong, and the output three-dimensional coordinates cannot accurately locate the vehicle's current position, and material distribution is stopped immediately; Second state module: If the fusion result judgment index is less than the preset fusion result judgment index threshold, it means that the probability of error in the data fusion result of the car positioning system about the current three-dimensional position of the car is small, and the output three-dimensional coordinates can accurately locate the current position of the car and continue material distribution.

Images