A logistics missing loading detection method based on loading trajectory recognition
By collecting the movement trajectories of loading equipment and materials in real time, a spatiotemporal trajectory point cloud sequence is generated. This sequence is compared with a standard trajectory template, and the loading path is adaptively adjusted. This solves the problems of real-time performance and accuracy in detecting missing items during loading, and enables real-time response and precise control during the loading process.
Patent Information
- Application Number
- CN202611180831.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-05
- Publication Date
- 2026-11-17
AI Technical Summary
There are instances of missing items in the current loading process, which leads to reduced transportation efficiency, increased reloading, and customer complaints. Existing detection methods are either unable to intervene in real time or lack sufficient accuracy.
By collecting the movement trajectory of loading equipment and materials in real time, a spatiotemporal trajectory point cloud sequence is generated. This sequence is compared with a standard trajectory template, and the threshold for missing detection is adaptively calculated. The loading path is adjusted in real time and closed-loop iterative control is performed to achieve real-time response and accurate identification of missing loading.
It enables real-time detection of missing items during the loading process, avoiding the lag in post-loading detection, improving the accuracy and efficiency of the loading process, and can identify differences in loading distribution in different areas of the car body, achieving synchronous iteration of detection and control.
Smart Images

Figure CN123117947A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of logistics management technology, and more specifically, to a logistics missing loading detection method based on loading trajectory recognition. Background Technology
[0002] In the logistics and transportation sector, loading is a crucial step in the transfer of goods from warehousing to transportation. During loading, due to factors such as deviations in loading equipment operation, uneven material flow, or incomplete execution of control commands, unloaded sections of the truck compartment can easily occur. If these unloaded sections are not detected in time, the actual load on the transport vehicle will be insufficient even when fully loaded, leading to a series of problems such as reduced transportation efficiency, increased operating costs from repeated reloading, and customer complaints.
[0003] Currently, the detection of missing loads during loading operations mainly relies on two methods: one is to weigh the load after loading is completed using a weighbridge and compare the actual loaded weight with the expected weight to determine if there are any missing loads; the other is for on-site operators to visually observe the loading situation during the loading process. Weighing only provides results after loading is complete, making real-time intervention for missing loads impossible during the loading process. Furthermore, when the densities of various materials differ significantly, a weight exceeding the standard does not reliably reflect that all areas of the truck bed have been evenly filled. Visual observation is limited by the operator's field of vision and attention span, and the rate of missed detection increases significantly in nighttime operations, dusty environments, or high-speed loading scenarios. Therefore, this invention proposes a logistics missing load detection method based on loading trajectory recognition to address the aforementioned problems. Summary of the Invention
[0004] To achieve the above objectives, the present invention provides the following technical solution: A logistics missing shipment detection method based on loading trajectory recognition includes the following steps: Step 1: According to the preset sampling interval and spatial location point coordinate acquisition benchmark of the loading operation, the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material are collected in real time. The continuous motion trajectory and the scattered motion trajectory are discretized into spatial location point coordinate values at several sampling times according to the time series. The spatial location point coordinate values at each sampling time and their corresponding timestamps are encoded into a spatiotemporal trajectory point cloud sequence. Step 2: Obtain the spatiotemporal trajectory point cloud sequence and simultaneously retrieve the pre-generated standard loading trajectory template. Compare the spatiotemporal trajectory point cloud sequence and the standard loading trajectory template point by point under the same time reference. Based on the spatial distance deviation value and spatial offset direction angle at each comparison point, generate an action deviation vector field containing the deviation magnitude and deviation vector. Step 3: Obtain the motion deviation vector field, statistically analyze the magnitude distribution characteristics and vector dispersion of all vectors in the motion deviation vector field, adaptively calculate and output the dynamic missed detection judgment threshold based on the variance value in the magnitude distribution characteristics and the maximum directional curvature value in the vector dispersion. The dynamic missed detection judgment threshold is used to distinguish between normal loading actions and missed loading events in the current loading environment. Step 4: Obtain the dynamic missed detection threshold, and determine the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold. Calculate the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area, and generate the loading integrity probability coefficient based on the overlap value. Step 5: Based on the difference between the probability coefficient of loading integrity and the expected integrity target value, calculate the feedforward compensation and feedback correction respectively. Output the feedforward compensation to the motion control terminal of the loading equipment to adjust the subsequent loading path in real time. At the same time, use the feedback correction as an adjustment command for the preset sampling interval and spatial position point coordinate acquisition benchmark, and send it back to Step 1 to update the generation benchmark of the spatiotemporal trajectory point cloud sequence for the next cycle, so as to realize the closed-loop iteration of detection and control.
[0005] In a preferred embodiment, after step one, according to the preset sampling interval and spatial location point coordinate acquisition benchmark, real-time acquisition of the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material, a data preprocessing step is also included: The collected raw trajectory data is subjected to noise filtering to remove abnormal outliers that deviate from the normal movement path by more than a preset deviation threshold, and the effective trajectory points after filtering are used as spatial location coordinates.
[0006] In a preferred embodiment, in step two, the standard loading trajectory template is generated as follows: Several complete and qualified loading operations without omissions were collected in advance. The spatiotemporal trajectory point cloud sequences of each qualified loading operation were superimposed and averaged according to the same time base. The average trajectory point coordinate sequence after superposition and averaging was used as the standard loading trajectory template.
[0007] In a preferred embodiment, in step three, the specific method for adaptively calculating and outputting the dynamic missed detection threshold based on the variance value in the modulus distribution characteristics and the maximum directional curvature value in the vector dispersion is as follows: After normalizing the variance and maximum directional curvature values respectively, the weighted sum is used as the dynamic omission detection threshold under the current vehicle loading environment. When the weighted sum increases, the dynamic omission detection threshold increases accordingly to reduce the false alarm rate. When the weighted sum decreases, the dynamic omission detection threshold decreases accordingly to improve the sensitivity of omission detection.
[0008] In a preferred embodiment, the specific method for determining the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold in step four is as follows: The coordinates of all spatial locations in the spatiotemporal trajectory point cloud sequence are mapped onto a two-dimensional horizontal projection plane to form a loading landing point distribution map. Several equal-area grids are divided on the loading landing point distribution map. Grids with landing point density lower than the dynamic missed detection threshold are marked as suspected missed loading areas. The area ratio of all suspected missed loading areas to all grids is calculated and used as a reverse judgment indicator of the integrity of the loading coverage area.
[0009] In a preferred embodiment, the specific calculation method for calculating the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area in step four is as follows: Perform a Boolean intersection operation between the loading point distribution map and the standard full-load coverage area in the same two-dimensional coordinate system, calculate the ratio of the overlapping area between the loading point distribution map and the standard full-load coverage area to the total area of the standard full-load coverage area, and use the ratio value as the overlap value.
[0010] In a preferred embodiment, in step five, the specific method for calculating the feedforward compensation and feedback correction amounts based on the difference between the loading integrity probability coefficient and the expected integrity target value is as follows: When the difference exceeds the preset allowable deviation range, closed-loop correction calculation is initiated, and the difference is decomposed into feedforward and feedback components proportionally. The feedforward component is directly superimposed on the next cycle motion command of the loading equipment to achieve advance adjustment, while the feedback component is used to adjust the preset sampling interval and the value of the spatial position point coordinate acquisition reference.
[0011] In a preferred embodiment, the maximum directional curvature value in step three is calculated as follows: The spatial offset direction angle of each vector in the motion deviation vector field is divided into several direction angle intervals according to a preset angle interval. The proportion of the number of vectors falling into each direction angle interval to the total number of vectors in the motion deviation vector field is counted. The gradient change rate of the proportion between adjacent direction angle intervals is calculated. The maximum absolute value of the gradient change rate is taken as the maximum direction curvature value. The maximum direction curvature value is used to characterize the spatial position where the deviation vector direction in the motion deviation vector field undergoes a sharp turn.
[0012] In a preferred embodiment, the calculation method for the feedforward compensation amount and the feedback correction amount in step five is as follows: The probability coefficient of vehicle loading integrity is formed into a probability coefficient fluctuation curve according to the time series. The probability coefficient fluctuation curve is decomposed into several intrinsic mode components and a residual trend term. The intrinsic mode component with the highest frequency is used as the instantaneous random disturbance characterization quantity to generate the feedforward compensation quantity, and the residual trend term is used as the system cumulative drift characterization quantity to generate the feedback correction quantity.
[0013] In a preferred embodiment, step five involves outputting the feedforward compensation amount to the motion control terminal of the loading equipment to adjust the subsequent loading path in real time, specifically as follows: After receiving the feedforward compensation amount, the motion control terminal superimposes the feedforward compensation amount onto the basic motion command of the loading equipment at the next sampling moment to form a corrected motion command after disturbance pre-compensation. The loading equipment executes the loading action of the next period according to the corrected motion command to offset the real-time disturbance caused by external load changes and signal fading on the loading trajectory. The specific method for transmitting the feedback correction amount as an adjustment command for the preset sampling interval and spatial location point coordinate acquisition reference back to step one is as follows: After receiving the feedback correction amount, the real-time acquisition step in step one adjusts the time length of the preset sampling interval and the origin offset and axial rotation of the spatial position point coordinate acquisition reference according to the feedback correction amount, so as to update the generation reference of the spatiotemporal trajectory point cloud sequence of the next cycle, thereby ensuring that the reference coordinate frame on which the subsequently acquired spatial position point coordinate values are based remains synchronized with the actual spatial pose of the loading equipment. The feedforward compensation and feedback correction are output simultaneously. The feedforward compensation is output to the motion control end along the forward path, and the feedback correction is transmitted back to the real-time acquisition stage in step one along the feedback path. The forward path and the feedback path run in parallel and independently, and are executed synchronously in time, so as to simultaneously realize the real-time advance disturbance suppression of the loading path and the dynamic tracking calibration of the acquisition reference.
[0014] The technical effects and advantages of this invention are as follows: This invention discretizes the motion trajectory of the end effector of the loading equipment and the loaded material into a spatiotemporal trajectory point cloud sequence in real time, and simultaneously performs point-by-point comparison with the standard loading trajectory template during the loading process. This eliminates the need for the identification of missing loading areas to rely on the weighing results after loading is completed. Instead, the spatial position deviation information and the integrity judgment results of the loading coverage area can be obtained at each sampling time during the loading process. This fundamentally eliminates the problem of post-event detection lag caused by the weighing method where the results can only be obtained after loading is completed, and enables the detection of missing loading to have a real-time response capability that is synchronized with the loading operation.
[0015] This invention adaptively calculates and outputs a dynamic missing detection threshold by statistically analyzing the variance and maximum directional curvature of all vectors in the motion deviation vector field. This allows the detection threshold used for missing loading to change in the same direction as the severity of actual deviations in the current loading environment, avoiding the failure of fixed thresholds due to dynamic conditions such as loading speed fluctuations, material flow changes, or equipment aging. Simultaneously, this invention combines the determination of the integrity of the loading coverage area with the calculation of the overlap of the standard full-load coverage area. It establishes a direct correlation between the grid density on the two-dimensional projection plane and the spatial coverage of the actual loading area of the truck bed. The probability coefficient of loading integrity quantitatively characterizes the degree of missing loading. Compared to weighing methods that rely solely on the total weight value, this invention can accurately identify differences in loading distribution in different areas of the truck bed, giving the missing loading detection results spatial distribution directionality and effectively solving the problem of missing loading in some areas even when the weight meets the standard.
[0016] This invention generates feedforward compensation and feedback correction quantities based on the difference between the probability coefficient of loading integrity and the expected integrity target value. The feedforward compensation quantity is output to the motion control terminal to adjust the subsequent loading path, while the feedback correction quantity is fed back to step one as an adjustment command for the preset sampling interval and spatial position point coordinate acquisition reference. This allows the detection signal to directly participate in the control correction of the loading action. Unlike the passive detection mode of weighing methods, which only provides result notification but cannot intervene in the loading process, this invention converts deviation information into a path adjustment quantity on the forward path and an acquisition reference calibration quantity on the feedback path. This allows the missing loading detection result to directly affect the correction execution of the loading action and the synchronous update of the data acquisition reference in the next cycle within the same sampling period. This achieves synchronous closed-loop iteration of detection and control, overcoming the shortcomings of existing visual observation methods that can only passively record missing loading results and cannot synchronously correct missing loading actions during the loading process. Attached Figure Description
[0017] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings; Figure 1 This is a schematic diagram of a logistics missing shipment detection method based on loading trajectory recognition, as described in this invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0019] Reference Figure 1 The following examples were obtained: Example 1: A logistics missing loading detection method based on loading trajectory recognition, comprising the following steps: Step 1: According to the preset sampling interval and spatial location point coordinate acquisition benchmark of the loading operation, the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material are collected in real time. The continuous motion trajectory and the scattered motion trajectory are discretized into spatial location point coordinate values at several sampling times according to the time series. The spatial location point coordinate values at each sampling time and their corresponding timestamps are encoded into a spatiotemporal trajectory point cloud sequence. Step 2: Obtain the spatiotemporal trajectory point cloud sequence and simultaneously retrieve the pre-generated standard loading trajectory template. Compare the spatiotemporal trajectory point cloud sequence and the standard loading trajectory template point by point under the same time reference. Based on the spatial distance deviation value and spatial offset direction angle at each comparison point, generate an action deviation vector field containing the deviation magnitude and deviation vector. Step 3: Obtain the motion deviation vector field, statistically analyze the magnitude distribution characteristics and vector dispersion of all vectors in the motion deviation vector field, adaptively calculate and output the dynamic missed detection judgment threshold based on the variance value in the magnitude distribution characteristics and the maximum directional curvature value in the vector dispersion. The dynamic missed detection judgment threshold is used to distinguish between normal loading actions and missed loading events in the current loading environment. Step 4: Obtain the dynamic missed detection threshold, and determine the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold. Calculate the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area, and generate the loading integrity probability coefficient based on the overlap value. Step 5: Based on the difference between the probability coefficient of loading integrity and the expected integrity target value, calculate the feedforward compensation and feedback correction respectively. Output the feedforward compensation to the motion control terminal of the loading equipment to adjust the subsequent loading path in real time. At the same time, use the feedback correction as an adjustment command for the preset sampling interval and spatial position point coordinate acquisition benchmark, and send it back to Step 1 to update the generation benchmark of the spatiotemporal trajectory point cloud sequence for the next cycle, so as to realize the closed-loop iteration of detection and control.
[0020] In step one, after collecting the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material in real time according to the preset sampling interval and spatial location coordinate acquisition benchmark of the loading operation, a data preprocessing step is also included: The collected raw trajectory data is first subjected to noise filtering. The noise filtering process adopts the sliding window mean filtering method. The length of the sliding window is set according to the movement speed of the end effector of the loading equipment. The typical value range is 5 to 15 sampling points. If the window length is too short, the filtering effect will be insufficient. If the window length is too long, the real trajectory inflection point information will be smoothed out.
[0021] A preset deviation judgment threshold is set to determine whether a certain spatial location point is an abnormal outlier. The value of the preset deviation judgment threshold is based on the historical statistical results of the normal movement path of the end effector of the loading equipment under both no-load and full-load conditions. The typical value is 0.1 meters. In actual applications, it can be adjusted within the range of 0.05 meters to 0.15 meters according to the model of the loading equipment and the operating speed.
[0022] Abnormal outliers that deviate from the normal motion path by more than a preset deviation threshold are removed. The specific method for identifying abnormal outliers is as follows: calculate the straight-line distance between the current spatial position and the average position of all valid spatial positions within the sliding window. If the straight-line distance exceeds the preset deviation threshold, the current spatial position is marked as an abnormal outlier. If the straight-line distance does not exceed the preset deviation threshold, the current spatial position is retained as a valid trajectory point. All valid trajectory points after filtering out the outliers constitute all the spatial position coordinate values required for subsequent encoding.
[0023] For example, in a loading operation, the coordinates of three consecutive spatial position points on the normal movement path of the end effector of the loading equipment are (1.0, 2.0, 0.5), (1.1, 2.1, 0.6), and (1.2, 2.2, 0.7), respectively. The coordinates of the fourth spatial position point are (5.0, 5.0, 2.0). The straight-line distance between this point and the average position of the first three points within the sliding window is 6.2 meters, which exceeds the preset deviation judgment threshold of 0.1 meters. Therefore, the fourth point is removed and not included in the subsequent spatiotemporal trajectory point cloud sequence encoding range.
[0024] In step two, the standard loading trajectory template is generated as follows: Several complete and unmissable loading operations are collected in advance. The completeness and unmissable status of the loading operations are confirmed by manual visual inspection or by weighbridge verification results. The number of qualified loading operations collected in advance is set according to the model of loading equipment and the degree of difference in working environment. For the same model of loading equipment and the same type of loaded materials, the number of collections should not be less than 20, with a typical value of 30. Too few collections will result in insufficient statistical representativeness, while too many collections will increase the amount of data processing while the improvement in accuracy is limited.
[0025] When the spatiotemporal trajectory point cloud sequences of each qualified loading operation are superimposed and averaged according to the same time reference, the same time reference is the start time of each loading operation as the time zero point. The spatial position coordinate values at the same relative time in each operation are superimposed point by point and then divided by the number of acquisitions. The meaning of point by point superposition is that at each same relative time, the X-direction values of the spatial position coordinate values of all qualified operations at that time are added together and then divided by the number of acquisitions to obtain the average X-direction value. The Y-direction values and Z-direction values are added together and then divided by the number of acquisitions.
[0026] The average trajectory point coordinate sequence after superposition and averaging is used as the standard loading trajectory template. The standard loading trajectory template is a complete sequence containing the average spatial location point coordinate values at each sampling time from the loading start time to the loading completion time. For example, after collecting 30 complete and qualified loading operations without omissions, for the 10th sampling time after loading starts, the spatial location point coordinate values at each of the 30 operations are (0.5, 1.2, 0.3), (0.6, 1.1, 0.4), (0.5, 1.3, 0.2), etc. The average X-direction value is obtained by summing all 30 X-direction values and dividing by 30. The Y and Z directions are processed in the same way, and the average trajectory point coordinate value at each sampling time is (0.55, 1.25, 0.33). The average trajectory point coordinate values at all sampling times are calculated in the same way, thus obtaining the standard loading trajectory template for various loading equipment and various types of loaded materials.
[0027] When comparing the spatial position difference of a spatiotemporal trajectory point cloud sequence with a standard loading trajectory template under the same time reference, "under the same time reference" means comparing the spatial position coordinates of points at the same relative time. Specifically, starting from the loading start time, the spatial position coordinates of the nth sampling time in the spatiotemporal trajectory point cloud sequence are paired with the average spatial position coordinates of the nth sampling time in the standard loading trajectory template to form a comparison point pair. The spatial distance deviation value of the comparison point pair is obtained based on the straight-line distance between the two spatial position coordinates in each comparison point pair. The spatial offset direction angle of the comparison point pair is obtained based on the direction vector from the average spatial position coordinates in the standard loading trajectory template to the actual spatial position coordinates in the spatiotemporal trajectory point cloud sequence. The spatial distance deviation values and spatial offset direction angles of all comparison point pairs together constitute all vectors in the motion deviation vector field.
[0028] In step three, the variance value in the modulus distribution characteristics is the numerical value obtained by statistically calculating the variance of the deviation modulus of all vectors in the motion deviation vector field. The value reflects the overall dispersion of the deviation modulus of all vectors relative to their average value. The larger the value, the greater the difference in the degree of deviation of the spatial distance deviation of each comparison point pair from the average. The maximum directional curvature value in the vector dispersion is a numerical value that characterizes the degree of change of the spatial offset direction angle of all vectors in the spatial distribution. The value is obtained by calculating the local directional change rate of all vectors according to the distribution of their spatial position coordinates in three-dimensional space, and taking the maximum value of all local directional change rates as the maximum directional curvature value.
[0029] When normalizing the variance and maximum directional curvature values separately and then weighting the sum, the normalization process involves dividing the actual variance value by the maximum variance value found in historical loading data to obtain the first normalized value, and dividing the actual maximum directional curvature value by the maximum maximum directional curvature value found in historical loading data to obtain the second normalized value. The weighted summation involves adding the first product (multiplied by the first weighting coefficient) to the second product (multiplied by the second weighting coefficient) to obtain the weighted sum. The values of the first and second weighting coefficients are based on the correlation statistics between the variance and maximum directional curvature values in historical loading data and the probability of missed loading events. The typical range for the first weighting coefficient is 0.4 to 0.6, and the typical range for the second weighting coefficient is also 0.4 to 0.6. The sum of the first and second weighting coefficients equals 1.
[0030] The specific method for using the weighted summation result as the dynamic missed detection threshold under the current vehicle installation environment is as follows: The weighted summation result ranges from 0 to 1. The value is directly mapped to the base of the dynamic missed detection threshold under the current vehicle loading environment. Then, it is multiplied by a preset threshold dimension conversion coefficient to obtain the actual threshold value used to determine the landing point density. The threshold dimension conversion coefficient is the conversion ratio between the number of landing points per square meter obtained from experimental data and the normalized value. A typical value is 100 landing points per square meter.
[0031] The specific method for reducing the false alarm rate by raising the dynamic missed detection threshold accordingly when the weighted summation result increases is as follows: The dynamic missed detection threshold increases synchronously in a linear proportional relationship based on the weighted summation result. When the weighted summation result increases from 0.2 to 0.8, the actual value of the dynamic missed detection threshold increases linearly from 20 landing points per square meter to 80 landing points per square meter.
[0032] When the weighted summation result decreases, the dynamic missed detection threshold is lowered accordingly to improve the sensitivity of missing parts detection. The specific method is as follows: The dynamic missed detection threshold decreases synchronously in a linear proportional relationship based on the weighted summation result. When the weighted summation result decreases from 0.8 to 0.2, the actual value of the dynamic missed detection threshold decreases linearly from 80 landing points per square meter to 20 landing points per square meter. Therefore, the adaptive adjustment in this invention uses a linear mapping method, ensuring that the dynamic missed detection threshold always changes in the same direction as the severity of the actual deviation in the current loading environment.
[0033] In practical applications, in addition to linear proportional mapping, nonlinear mapping or other adaptive methods can also be used to calculate the dynamic missed detection threshold. These methods are all disclosed in the existing technology. For example, a nonlinear mapping method based on radial basis functions can be used. By utilizing the local nonlinear function approximation capability unique to radial basis functions, the weighted summation result is mapped to the dynamic missed detection threshold through a radial basis neural network. This makes the threshold more sensitive to changes when the weighted summation result is in the middle range, and the change tends to be gentle when the weighted summation result is in the extreme value range at both ends.
[0034] Alternatively, a nonlinear transformation method based on fuzzy rules can be adopted, using the weighted summation result as a fuzzy input variable, performing fuzzy inference through a preset fuzzy membership function and fuzzy rule base to obtain a fuzzy output value, and then defuzzifying the fuzzy output value as a dynamic missed detection threshold.
[0035] Alternatively, a nonlinear mapping method based on an exponential function can be adopted, using the weighted summation result as the independent variable of the exponential function. A nonlinear mapping relationship is constructed through a preset base and exponential coefficient. When the weighted summation result increases, the dynamic missed detection threshold increases according to an exponential law. When the weighted summation result decreases, the dynamic missed detection threshold decreases according to an exponential law. The threshold change rate under the exponential law has different change gradients in different numerical ranges of the weighted summation result.
[0036] In addition to the methods mentioned above, other adaptive threshold determination methods in existing technologies can also be adopted. For example, an adaptive threshold construction method based on dynamic parameter relationship reasoning can be used to construct an adaptive threshold detection model by reasoning the dynamic relationship between multiple parameters; or an adaptive threshold update mechanism based on a threshold cyclic update strategy can be designed to combine internal and external loops to achieve dynamic adjustment of the anomaly detection threshold; or the threshold can be associated with system error, so that the dynamic missed detection judgment threshold changes accordingly when the system error changes.
[0037] All of the above nonlinear mapping and adaptive methods can replace or supplement the original linear proportional mapping method without violating the existing description of this invention. The specific mapping relationship can be selected and set by the user according to the characteristics of the actual vehicle environment, the distribution pattern of historical experimental data, and the different emphasis requirements of the system on false alarm rate and false negative rate.
[0038] In step four, the specific method for determining the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold is as follows: The coordinate values of all spatial location points in the spatiotemporal trajectory point cloud sequence are projected vertically onto a two-dimensional horizontal projection plane. The coordinate system of the two-dimensional horizontal projection plane takes the horizontal plane where the bottom of the carriage is located as the reference plane, the length of the carriage as the horizontal axis, and the width of the carriage as the vertical axis. The vertical coordinate values of the spatial location points are discarded during the projection process, and only the horizontal axis coordinate values and the vertical axis coordinate values are retained to form two-dimensional projection points. The set of all two-dimensional projection points is the loading landing point distribution map.
[0039] When dividing the loading point distribution map into several equal-area grids, each grid is a square. The side length of the square is set according to the positioning accuracy of the end effector of the loading equipment. The positioning accuracy of the end effector is the repeatability accuracy value given in the equipment manual, typically 0.05 meters. The side length of the square is set to be between two and five times the positioning accuracy value, typically 0.1 meters. The total length and total width of the car body floor are divided by the side length of the square and rounded up to obtain the number of grids in the horizontal and vertical directions, respectively. The product of the number of grids in the horizontal and vertical directions is the total number of grids.
[0040] For each grid, the number of 2D projection points falling within the grid is counted, and this number is divided by the grid area to obtain the grid's point density value. The point density value of each grid is compared with a dynamic omission detection threshold. If the point density value of a grid is lower than the dynamic omission detection threshold, the grid is marked as a suspected omission area. If the point density value of a grid is not lower than the dynamic omission detection threshold, the grid is not marked as a suspected omission area. For example, in a loading operation, the total length of the truck bed is 4 meters, the total width is 2 meters, and the grid side length is 0.1 meters. Therefore, there are 40 grids along the horizontal axis and 20 grids along the vertical axis, for a total of 800 grids. If a grid contains only one 2D projection point and has an area of 0.01 square meters, the point density is 100 points per square meter. If the current dynamic omission detection threshold is 150 points per square meter, the point density is lower than the current dynamic omission detection threshold, and the grid is marked as a suspected omission area.
[0041] The specific method for calculating the proportion of all suspected missing areas to the total area of the entire grid is as follows: The total area of suspected missing areas is calculated by multiplying the number of grids marked as potentially missing areas by the area of each grid. The total area of suspected missing areas is then divided by the total area of all grids to obtain the area ratio. This area ratio serves as a reverse indicator of the integrity of the loading coverage area. The meaning of this reverse indicator is that a larger area ratio indicates poorer integrity of the loading coverage area, while a smaller area ratio indicates better integrity of the loading coverage area.
[0042] In step four, the specific calculation method for calculating the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area is as follows: The loading point distribution map and the standard full-load coverage area are subjected to Boolean intersection operation in the same two-dimensional coordinate system. The Boolean intersection operation means that the loading point coverage area formed by all the two-dimensional projection points covered by the loading point distribution map is geometrically superimposed with the standard full-load coverage area, and the common area covered by the two areas in space is taken as the intersection area.
[0043] The specific method for calculating the proportion of the overlap area between the loading point distribution map and the standard full-load coverage area to the total area of the standard full-load coverage area is as follows: First, calculate the area of the intersection region. Divide this area by the total area of the standard fully loaded coverage area; the resulting ratio is the overlap value. For example, suppose the standard fully loaded coverage area is the entire area of the car's floor, totaling 8 square meters. The loaded area represented by the loading point distribution map is a portion of the car's floor, with an area of 6 square meters. The intersection area between the loaded area and the entire car's floor is 6 square meters. Dividing 6 square meters by 8 square meters gives an overlap value of 0.75.
[0044] The specific method for generating the probability coefficient of loading integrity based on the overlap value is as follows: The overlap value is directly used as the base probability coefficient, and then multiplied by a preset correction coefficient. The range of the correction coefficient is set based on experimental data of the natural angle of repose of the loaded materials. When the loaded material is a granular material with good flowability, the correction coefficient is between 1.0 and 1.05, with a typical value of 1.02. When the loaded material is a blocky material with poor flowability, the correction coefficient is between 0.95 and 1.0, with a typical value of 0.98. This yields the loading integrity probability coefficient. The correction coefficient is set based on the fact that the difference in the stacking morphology of different materials will lead to different actual loading volumes under the same two-dimensional projection coverage area. Therefore, it is necessary to make corrections according to the type of material to ensure that the loading integrity probability coefficient can accurately reflect the integrity of the actual loading operation.
[0045] In step three, the maximum directional curvature value in the vector dispersion is calculated as follows: The spatial offset direction angle of each vector in the motion deviation vector field is divided into several direction angle intervals according to a preset angle range. The preset angle range is a complete circumferential angle range from 0 degrees to 360 degrees. The number of direction angle intervals is set according to the motion direction resolution of the end effector of the loading equipment. The motion direction resolution is determined by the angle control accuracy of the loading equipment control system. A typical value is to divide 360 degrees into 8 direction angle intervals. The angle span of each direction angle interval is 45 degrees. The 8 direction angle intervals are 0 degrees to 45 degrees, 45 degrees to 90 degrees, 90 degrees to 135 degrees, 135 degrees to 180 degrees, 180 degrees to 225 degrees, 225 degrees to 270 degrees, 270 degrees to 315 degrees, and 315 degrees to 360 degrees.
[0046] To calculate the proportion of vectors falling within each directional angle interval to the total number of vectors in the motion deviation vector field, first count the number of vectors contained in each directional angle interval. Then divide the number of vectors by the total number of vectors in the motion deviation vector field to obtain the proportion of vectors in each directional angle interval. For example, if the total number of vectors in the motion deviation vector field is 1000, and the number of vectors falling within the 0-45 degree directional angle interval is 120 divided by 1000, which equals 0.12, or 12%. The other seven directional angle intervals are calculated in the same way to determine their respective proportions.
[0047] When calculating the gradient rate of change of the proportion between adjacent directional angle intervals, adjacent directional angle intervals refer to two directional angle intervals that are adjacent in circumferential order. For example, the 0° to 45° interval and the 45° to 90° interval are a pair of adjacent directional angle intervals, and so on. The 315° to 360° interval and the 0° to 45° interval form the last pair of adjacent directional angle intervals on the circumference. The gradient rate of change between each pair of adjacent directional angle intervals is calculated by taking the absolute value of the difference between the vector proportion values of the two directional angle intervals, and then dividing it by the angular span value between the two directional angle intervals. Since each directional angle interval has an equal angular span, which is a constant of 45 degrees, the magnitude of the gradient rate of change depends only on the magnitude of the absolute value of the difference between the adjacent proportion values.
[0048] When the maximum absolute value of the gradient change rate is used as the maximum directional curvature value, the maximum absolute value refers to the maximum value among all adjacent directional angle intervals. The absolute value of the maximum value is taken as the maximum directional curvature value. The maximum directional curvature value is used to characterize the spatial location where the deviation vector direction in the action deviation vector field undergoes a sharp turn. The physical basis for this characterization function is that when the end effector of the loading equipment experiences abnormal events such as material impacting the truck bed baffle or sudden equipment stoppage and turn in a certain spatial area, the spatial offset directional angles of each comparison point pair near the area will suddenly change. This causes the proportion of vectors falling into the directional angle interval to suddenly increase or decrease, resulting in a peak value in the gradient change rate of the proportion between adjacent directional angle intervals. The directional angle interval corresponding to the peak value can indicate the dominant direction of the sharp turn, and the magnitude of the peak value itself indicates the severity of the turn.
[0049] In step five, the calculation methods for the feedforward compensation and feedback correction are as follows: The probability coefficient of loading integrity is formed into a probability coefficient fluctuation curve according to the time series. The horizontal axis of the probability coefficient fluctuation curve is the sampling time number, and the vertical axis is the specific value of the probability coefficient of loading integrity corresponding to each sampling time. The length of the time series is the total number of samplings from the loading start time to the current time, typically ranging from one thousand to five thousand times. The sampling frequency is set according to the data refresh rate of the loading equipment control system, typically one hundred hertz.
[0050] When performing empirical mode decomposition (EMD) on the probability coefficient fluctuation curve, no basis functions need to be preset. Instead, the curve is decomposed layer by layer based solely on its local extremum characteristics, resulting in several intrinsic mode components and a residual trend term. Each intrinsic mode component satisfies two fundamental conditions: first, the absolute value of the difference between the number of local maxima and local minima on the intrinsic mode component does not exceed one; second, the mean of the upper envelope formed by the local maxima and the lower envelope formed by the local minima of the intrinsic mode component is zero at any given time.
[0051] The empirical mode decomposition (EMD) process employs two levels of screening stopping criteria. The first level is the screening stopping criterion for a single intrinsic mode component (EMC), using the standard deviation criterion as the stopping condition. The standard deviation criterion is calculated by taking the sum of the squares of the differences between the current screening result and the previous screening result, and then dividing this sum by the sum of the squares of the previous screening result. If the standard deviation is less than 0.2, the screening process for the current EMC stops, and the current screening result is output as the EMC. If the standard deviation is greater than or equal to 0.2, the next screening iteration continues. The second level is the stopping criterion for the overall decomposition process. After extracting local maxima and local minima from the current remaining signal sequence, if the total number of local maxima and local minima in the remaining signal sequence does not exceed two, the overall decomposition process stops, and the remaining signal sequence is directly output as the residual trend term. If the total number of local extrema in the remaining signal sequence exceeds two, the extraction of the next order EMC continues.
[0052] Using the two stopping criteria described above, the system can automatically determine the specific number of several intrinsic mode components until the remaining signal no longer exhibits wave-like mode characteristics, thus obtaining a complete set of intrinsic mode components and a residual trend term. The highest-frequency intrinsic mode component is used as a characterization of instantaneous random disturbance to generate the feedforward compensation quantity. The highest-frequency intrinsic mode component refers to the first intrinsic mode component after sorting the components by frequency from high to low. Its frequency value is determined by the reciprocal of the average time interval between adjacent extreme points in the intrinsic mode component, typically ranging from 20 Hz to 100 Hz. The residual trend term is used as a characterization of the system's cumulative drift to generate the feedback correction quantity. The residual trend term consists of the low-frequency monotonic trend component or extremely low-frequency wave component remaining after several empirical mode decompositions, with its frequency value typically ranging from zero to one Hz.
[0053] The specific methods for calculating the feedforward compensation and feedback correction amounts for the difference between the probability coefficient of vehicle loading integrity and the expected integrity target value are as follows: First, calculate the difference between the expected integrity target value and the current loading integrity probability coefficient. The expected integrity target value is based on the full-load coverage area standard in the loading operation process requirements, typically set at 95%. When the difference exceeds the preset deviation tolerance range, a closed-loop correction calculation is initiated. The preset deviation tolerance range is set according to the allowable full-load deviation tolerance for loading operations, typically set at ±5%. If the absolute value of the difference exceeds 5%, the closed-loop correction calculation is initiated; if the absolute value of the difference does not exceed 5%, the closed-loop correction calculation is not initiated, and the current parameters are maintained.
[0054] When decomposing the difference into feedforward and feedback components proportionally, the decomposition ratio is set based on the different emphasis requirements of the system on disturbance response speed and steady-state accuracy. The feedforward component accounts for 60% to 80% of the total difference, and the feedback component accounts for 20% to 40%. A typical value is 70% for the feedforward component and 30% for the feedback component. For example, if the current vehicle assembly integrity probability coefficient is 85%, and the expected integrity target value is 95%, then the difference is 10%. Since the absolute value exceeds the preset deviation tolerance range of 5%, closed-loop correction calculation is initiated. The 10% difference is then decomposed into a feedforward component of 70% and a feedback component of 30%, resulting in a feedforward component of 7% and a feedback component of 3%.
[0055] The specific method for outputting the feedforward compensation to the motion control terminal of the loading equipment to adjust the subsequent loading path in real time is as follows: After receiving the feedforward compensation, the motion control terminal superimposes the feedforward compensation onto the basic motion command of the loading equipment at the next sampling moment. The basic motion command is the position control command or speed control command generated by the loading equipment control system according to the preset standard trajectory. The feedforward compensation is directly algebraically added to the basic motion command in the form of a correction term to obtain the corrected motion command after disturbance pre-compensation. The loading equipment executes the loading action of the next time period according to the corrected motion command to offset the real-time disturbance caused by external load mutations and signal fading on the loading trajectory. The external load mutation is specifically manifested as the instantaneous flow fluctuation of the loaded material causing a sudden change in the reaction force on the end effector of the loading equipment. The signal fading is specifically manifested as the decrease in the signal-to-noise ratio of the sensor signal in a strong dust environment, causing random drift of the spatial position point coordinate acquisition value.
[0056] The specific method for transmitting the feedback correction amount as an adjustment command for the preset sampling interval and spatial location point coordinate acquisition reference back to step one is as follows: In the real-time acquisition phase of step one, after receiving the feedback correction, the preset sampling interval time length, origin offset, and axial rotation of the spatial location point coordinate acquisition reference are adjusted according to the feedback correction. The preset sampling interval time length is adjusted by algebraically adding the product of the current sampling interval's base time length and the feedback correction multiplied by the sampling interval adjustment coefficient. A typical value for the sampling interval adjustment coefficient is 0.1 seconds. The origin offset of the spatial location point coordinate acquisition reference is adjusted by adding the three-dimensional coordinates of the current coordinate reference's origin position to the product of the feedback correction multiplied by the origin offset adjustment coefficient in each of the three axes. A typical value for the origin offset adjustment coefficient is 0.01 meters.
[0057] The method for adjusting the axial rotation of the spatial location point coordinate acquisition reference is as follows: The rotation angles of the current coordinate reference along each of the three axes are added to the product of the feedback correction amount and the axial rotation adjustment coefficient. A typical value for the axial rotation adjustment coefficient is 0.01 degrees. This adjustment method updates the generation reference for the spatiotemporal trajectory point cloud sequence of the next cycle, ensuring that the reference coordinate framework upon which the subsequently acquired spatial location point coordinate values are based remains synchronized with the actual spatial pose of the loading equipment.
[0058] The feedforward compensation and feedback correction are output simultaneously. The feedforward compensation is output to the motion control end along the forward path, while the feedback correction is fed back to the real-time acquisition stage in step one along the feedback path. The forward and feedback paths operate in parallel and independently, executing synchronously in time to simultaneously achieve real-time advance disturbance suppression of the loading path and dynamic tracking calibration of the acquisition reference. The physical meaning of the parallel synchronous execution mode is that at each sampling moment, the feedforward compensation first corrects the motion command for the next moment to cope with the upcoming disturbance, while the feedback correction synchronously corrects the acquisition reference to adapt to the system drift that has already occurred. Both are completed simultaneously within the same sampling period, without any sequential waiting relationship, thus forming a dual-channel closed-loop control structure that combines fast response and steady-state accuracy.
[0059] The above algorithms or formulas are all dimensionless and numerical calculations, and the results are obtained by software simulation based on a large amount of collected data to obtain the most recent real-world results. The preset parameters are set by those skilled in the art according to the actual situation.
[0060] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0061] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0062] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0063] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A method for detecting missing loading based on loading track recognition, characterized in that, Includes the following steps: Step 1: According to the preset sampling interval and spatial location point coordinate acquisition benchmark of the loading operation, the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material are collected in real time. The continuous motion trajectory and the scattered motion trajectory are discretized into spatial location point coordinate values at several sampling times according to the time series. The spatial location point coordinate values at each sampling time and their corresponding timestamps are encoded into a spatiotemporal trajectory point cloud sequence. Step 2: Retrieve the pre-generated standard loading trajectory template, compare the spatial position difference of the spatiotemporal trajectory point cloud sequence with the standard loading trajectory template under the same time reference, and generate a motion deviation vector field containing the deviation magnitude and deviation vector based on the spatial distance deviation value and spatial offset direction angle at each comparison point. Step 3: Analyze the magnitude distribution characteristics and vector dispersion of all vectors in the action deviation vector field. Based on the variance value in the magnitude distribution characteristics and the maximum directional curvature value in the vector dispersion, adaptively calculate and output the dynamic missed detection threshold. The dynamic missed detection threshold is used to distinguish between normal loading actions and missed loading events in the current loading environment. Step 4: Determine the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold. Calculate the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area, and generate a loading integrity probability coefficient based on the overlap value. Step 5: Based on the difference between the probability coefficient of loading integrity and the expected integrity target value, calculate the feedforward compensation amount and the feedback correction amount respectively. Output the feedforward compensation amount to the motion control terminal of the loading equipment to adjust the subsequent loading path in real time. At the same time, use the feedback correction amount as the adjustment command for the preset sampling interval and spatial position point coordinate acquisition benchmark, and send it back to Step 1 to update the generation benchmark of the spatiotemporal trajectory point cloud sequence for the next cycle. 2.The method of claim 1, wherein, In step one, after collecting the continuous motion trajectory of the end effector of the loading equipment in three-dimensional space and the scattered motion trajectory of the loaded material in real time according to the preset sampling interval and spatial location coordinate acquisition benchmark of the loading operation, a data preprocessing step is also included: The collected raw trajectory data is subjected to noise filtering to remove abnormal outliers that deviate from the normal movement path by more than a preset deviation threshold, and the effective trajectory points after filtering are used as spatial location coordinates. 3.The method of claim 2, wherein, In step two, the standard loading trajectory template is generated as follows: Several complete and qualified loading operations without omissions were collected in advance. The spatiotemporal trajectory point cloud sequences of each qualified loading operation were superimposed and averaged according to the same time base. The average trajectory point coordinate sequence after superposition and averaging was used as the standard loading trajectory template.
4. The method according to claim 3, wherein, In step three, the specific method for adaptively calculating and outputting the dynamic missed detection threshold based on the variance value in the modulus distribution characteristics and the maximum directional curvature value in the vector dispersion is as follows: After normalizing the variance and maximum directional curvature values respectively, the weighted sum is used as the dynamic omission detection threshold under the current vehicle loading environment. When the weighted sum increases, the dynamic omission detection threshold increases accordingly to reduce the false alarm rate. When the weighted sum decreases, the dynamic omission detection threshold decreases accordingly to improve the sensitivity of omission detection.
5. The method of claim 4, wherein the method further comprises: In step three, the maximum directional curvature value in the vector dispersion is calculated as follows: The spatial offset direction angle of each vector in the motion deviation vector field is divided into several direction angle intervals according to a preset angle interval. The proportion of the number of vectors falling into each direction angle interval to the total number of vectors in the motion deviation vector field is counted. The gradient change rate of the proportion between adjacent direction angle intervals is calculated. The maximum absolute value of the gradient change rate is taken as the maximum direction curvature value. The maximum direction curvature value is used to characterize the spatial position where the deviation vector direction in the motion deviation vector field undergoes a sharp turn.
6. The method of claim 5, wherein the method further comprises: In step four, the specific method for determining the integrity of the loading coverage area of the spatiotemporal trajectory point cloud sequence based on the dynamic missed detection threshold is as follows: The coordinates of all spatial locations in the spatiotemporal trajectory point cloud sequence are mapped onto a two-dimensional horizontal projection plane to form a loading landing point distribution map. Several equal-area grids are divided on the loading landing point distribution map. Grids with landing point density lower than the dynamic missed detection threshold are marked as suspected missed loading areas. The area ratio of all suspected missed loading areas to all grids is calculated and used as a reverse judgment indicator of the integrity of the loading coverage area.
7. The method of claim 6, wherein the method further comprises: In step four, the specific calculation method for calculating the spatial overlap between the loaded area represented by the spatiotemporal trajectory point cloud sequence and the preset standard full-load coverage area is as follows: Perform a Boolean intersection operation between the loading point distribution map and the standard full-load coverage area in the same two-dimensional coordinate system. Calculate the ratio of the overlapping area between the loading point distribution map and the standard full-load coverage area to the total area of the standard full-load coverage area. Use the ratio as the overlap value, and then multiply it by a preset correction coefficient to obtain the loading integrity probability coefficient. 8.The method of claim 7, wherein, In step five, the specific methods for calculating the feedforward compensation and feedback correction amounts based on the difference between the probability coefficient of vehicle loading integrity and the expected integrity target value are as follows: When the difference exceeds the preset allowable deviation range, closed-loop correction calculation is initiated, and the difference is decomposed into feedforward and feedback components proportionally. The feedforward component is directly superimposed on the next cycle motion command of the loading equipment to achieve advance adjustment, while the feedback component is used to adjust the preset sampling interval and the value of the spatial position point coordinate acquisition reference. 9.The method of claim 8, wherein, In step five, the calculation methods for the feedforward compensation and feedback correction are as follows: The probability coefficient of vehicle loading integrity is formed into a probability coefficient fluctuation curve according to the time series. The probability coefficient fluctuation curve is decomposed into several intrinsic mode components and a residual trend term. The intrinsic mode component with the highest frequency is used as the instantaneous random disturbance characterization quantity to generate the feedforward compensation quantity, and the residual trend term is used as the system cumulative drift characterization quantity to generate the feedback correction quantity.
