A full load determination method for compressed garbage truck filling image recognition

Abstract

The application discloses a full-load determination method for compressed garbage truck filling image recognition, and relates to the technical field of intelligent control of compressed garbage trucks. In the stable window period after the completion of the pressing and filling action of the scraper and the retraction in the pressing cycle, a plurality of material surface images are continuously collected through an industrial camera; dust adaptive enhancement processing is performed on each collected material surface image, specifically including: calculating a dust concentration estimation value of the current frame image; adaptively adjusting the transmittance parameter of the dark channel prior dehazing algorithm according to the dust concentration estimation value to obtain an enhanced image; the application directly collects the material surface image by installing the industrial camera inside the filler, overcomes the inherent defects of the traditional hydraulic pressure method, proximity switch method and other indirect inference methods, can truly reflect the material level state inside the filler, and the determination result is more intuitive and reliable.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for compressed garbage trucks, specifically to a method for determining full load by recognizing the filling material image of a compressed garbage truck. Background Technology

[0002] Compactor garbage trucks are core equipment in urban waste collection and transportation systems. They primarily use hydraulically driven scrapers and sliding mechanisms to repeatedly compress the garbage inside the truck compartment, increasing the loading density. When the internal space of the compactor is nearly full, the compaction operation must be stopped promptly; otherwise, it will not only waste the energy of the hydraulic system but may also cause mechanical overload or even equipment damage. Therefore, accurately determining when the compactor has reached full load is a crucial aspect of the intelligent control of compactor garbage trucks.

[0003] Currently, the industry mainly relies on the following methods to determine the full load status. The first is the hydraulic pressure threshold method, which considers the cylinder full when the pressure exceeds a certain set value. The problem with this method is that the pressure rise is not only related to the material level but also affected by various factors such as the type of waste, moisture content, and compression ratio. The same material level may correspond to completely different pressure values ​​under different working conditions, resulting in a high false alarm rate. The second method is a combination of proximity switches and pressure sensors, which indirectly determines the full load by detecting whether the scraper or slide plate has reached its limit position. This method can only reflect whether the mechanism has reached its position and cannot directly reflect the actual material level inside the stuffer. The third method is ultrasonic level gauges, which can theoretically measure the distance between the material surface and the surface. However, the stuffer is filled with extremely high dust concentration and water vapor, and ultrasonic waves are severely scattered in this harsh environment, making it difficult to guarantee measurement accuracy. Furthermore, single detection methods are often insufficient to cope with complex and changing working conditions. Hydraulic parameters are easy to obtain but have limited accuracy, and image information is rich but greatly affected by environmental interference.

[0004] This application proposes a full-load determination method for image recognition of compressed garbage trucks, which can effectively fuse information from two modalities. When the dust concentration is low, image determination is emphasized, and when the dust concentration is high, hydraulic determination is emphasized, thus balancing accuracy and reliability. Summary of the Invention

[0005] To address the aforementioned technical problems, a method for determining full load using image recognition of filler material in compressed garbage trucks is provided. This technical solution solves the problems that hydraulic parameters are easy to obtain but have limited accuracy, and that image information is rich but is greatly affected by environmental interference.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A method for determining full load using image recognition of filler material in a compressed garbage truck includes the following steps:

[0008] S1: An industrial camera is installed on the top of the inner cavity of the compactor of the garbage truck. During the stable window period after the scraper completes the compaction action and retracts in the compaction cycle, multiple frames of material surface images are continuously acquired through the industrial camera.

[0009] The stabilization window period refers to the time after the scraper completes the compaction action and retracts, during which both the material surface and dust tend to stabilize. Immediately after the scraper retracts, the first frame image is acquired, and the dust concentration estimate 'd' for this frame image is calculated according to the method in step S2. This estimate is used to determine the stabilization window period (at this stage, only the 'd' value needs to be calculated; full image enhancement processing is not performed). When the dust concentration estimate 'd' drops below 0.3 and the change in the material surface contour of three consecutive frames is less than 5 pixels, the stabilization window period is considered to have begun, and the acquisition of multiple frames for subsequent processing commences.

[0010] The reason for choosing to collect data during the stable window period after the scraper retracts is that during compaction, the scraper moves violently, debris splashes, and dust concentration rises sharply, resulting in extremely poor image quality. After the scraper retracts, wait a few seconds for the dust to settle and the material surface to stabilize, at which point the acquired image will better reflect the true material level.

[0011] S2: Perform dust adaptive enhancement processing on the material surface image acquired in each frame, specifically including: calculating the dust concentration estimate of the current frame image; adaptively adjusting the transmittance parameter of the dark channel prior dehazing algorithm according to the dust concentration estimate to obtain the enhanced image.

[0012] The dust inside the filler is essentially the same as fog, consisting of suspended particles scattering light. Dark channel prior dehazing algorithms, originally designed for outdoor haze images, can lead to over-enhancement and color distortion when applied directly. This invention dynamically adjusts the dehazing intensity based on the dust concentration of the current frame, strengthening dehazing when dust concentration is high and weakening it when dust concentration is low, avoiding the side effects of a one-size-fits-all approach.

[0013] S3: Extract the material surface contour from the enhanced image, divide the inner cavity of the filler into N regions in the image along the height direction, and calculate the waste filling ratio of each region, where N≥2.

[0014] The surface of the waste is highly irregular, and the accumulation patterns of different types of waste vary greatly, so a single height value cannot be used to represent the overall material level. Dividing the filler into sections along the height direction and evaluating the filling status of each section separately is more reliable than using a global threshold. A corner filling first does not necessarily mean the whole area is full; section-based evaluation can reveal this unevenness.

[0015] S4: Synchronously collect real-time pressure data of the hydraulic cylinder in the hydraulic system, perform low-pass filtering preprocessing on the collected pressure signal, and calculate the pressure change rate in the current compaction cycle.

[0016] As the material level rises, the remaining space inside the stuffer becomes smaller and smaller. The resistance that the hydraulic system needs to overcome during each compaction gradually increases, resulting in a faster rate of pressure increase. The rate of pressure change is a better indicator of material level changes than the absolute pressure value because it eliminates the influence of factors such as different types of waste and moisture content on the pressure baseline.

[0017] S5: Perform dual-modal cross-validation between the waste filling ratio of each region obtained in step S3 and the pressure change rate obtained in step S4, and output the full-load status signal according to the preset fusion judgment rule.

[0018] Single-modal judgments all have blind spots. Image-based judgments are highly accurate at low dust concentrations, but image quality deteriorates and judgment results become unreliable at high dust concentrations. Hydraulic judgments are unaffected by dust, but their response to changes in material level is delayed. Dual-modal fusion can complement the advantages and disadvantages of both, improving the accuracy and robustness of the judgment.

[0019] Further, the specific method for calculating the dust concentration estimate of the current frame image in step S2 is as follows: convert the current frame image into a grayscale image; divide the grayscale image into M×M uniform sub-blocks, where M is a positive integer; calculate the grayscale mean and grayscale standard deviation for each sub-block; record the number of sub-blocks in all sub-blocks whose grayscale mean is lower than a preset dark threshold and whose grayscale standard deviation is lower than a preset fluctuation threshold as the number of dust region sub-blocks; calculate the ratio of the number of dust region sub-blocks to the total number of sub-blocks M×M, as the dust concentration estimate.

[0020] Areas with high dust concentration appear as large, dark, and blurry regions in an image, with low mean grayscale and small standard deviation. Sub-block statistics can quickly estimate the overall dust concentration of the current frame, requiring minimal computation and making it suitable for real-time processing.

[0021] Furthermore, the specific method for adaptively adjusting the transmittance parameter of the dark channel prior defogging algorithm based on the dust concentration estimate in step S2 is as follows: when the dust concentration estimate is less than the first concentration threshold, the lower bound of transmittance is taken as the first value; when the dust concentration estimate is greater than or equal to the first concentration threshold and less than the second concentration threshold, the lower bound of transmittance is taken as the second value, which is less than the first value; when the dust concentration estimate is greater than or equal to the second concentration threshold, the lower bound of transmittance is taken as the third value, which is less than the second value; wherein the first concentration threshold is less than the second concentration threshold.

[0022] Three-level adjustment is a practical compromise in engineering. When the dust concentration is low, strong dehazing is not necessary, and preserving the original image information is more important. When the dust concentration is high, dehazing must be strengthened, otherwise the image will be almost unusable. The middle level serves as a transition. Three levels are sufficient to handle most working conditions. Too many levels will only increase the complexity of parameter adjustment.

[0023] Further, in step S3, the method for dividing the stuffer cavity into N regions in the image along the height direction is as follows: semantic segmentation is performed on the enhanced image to classify the pixels into waste surface region, stuffer wall region, and background region; the upper boundary line of the waste surface region is extracted as the surface contour line; based on the total vertical projection height H of the stuffer cavity in the image, the projection range is divided into N regions from bottom to top, where N is 3, namely the bottom region, middle region, and top region; the ratio of the number of pixels covered by the waste surface region in each region to the total number of pixels in that region is calculated as the waste filling ratio of the corresponding region.

[0024] Furthermore, the method for temporal fusion of multiple material surface images in step S3 is as follows: K frames of material surface images are continuously acquired within a stable window period, where K ≥ 3; steps S2 and S3 are executed for each frame of material surface images to obtain the waste filling ratio of each region in K groups; the median of the K filling ratio values ​​corresponding to the same region in K frames is taken as the fusion filling ratio of the region; or, the K filling ratio values ​​corresponding to the same region in K frames are assigned increasing weights according to the acquisition time, and the weighted average value is calculated as the fusion filling ratio of the region.

[0025] Furthermore, it also includes cross-cycle trend analysis and full-load prediction steps based on the exponential decay model of filling increment: recording the fusion filling ratio of each cycle in P consecutive filling cycles. ,in , ; Calculate the fill increment sequence between two adjacent cycles ,in .

[0026] Establish an exponential decay model for filling increments. ,in For the initial fill increment, The attenuation coefficient is obtained by curve fitting the filled increment sequence using the least squares method. and initial fill increment The optimal estimate.

[0027] Based on the fitted exponential decay model, predict the future... The fill increment of each cycle Predicting the future Cumulative fill percentage after one cycle .

[0028] When the predicted cumulative fill percentage Greater than or equal to the preset full load threshold When full load is reached, a full load warning signal is output, and the remaining number of cycles required to reach full load is calculated based on the exponential decay model. , making .

[0029] The full load prediction result is compared and verified with the current judgment result of the dual-modal cross-validation in step S5: when the prediction result is that it is about to be fully loaded and the dual-modal judgment result is close to full load, a full load confirmation signal is output; when the prediction result is that it is about to be fully loaded but the dual-modal judgment result is normal loading, a trend warning signal is output and monitoring continues; when the prediction result is that it is not yet fully loaded but the dual-modal judgment result is close to full load, an abnormality prompt signal is output, indicating that there may be local overloading or sensor deviation; when the prediction result is that it is not yet fully loaded and the dual-modal judgment result is normal loading, a normal loading signal is output.

[0030] Furthermore, before step S1, a camera calibration step is included: at least two calibration reference marks with known physical heights are set on the inner wall of the stuffer; images of the calibration reference marks are acquired using an industrial camera, and the pixel ordinates of each calibration reference mark in the image are extracted; based on the physical height values ​​of the calibration reference marks and the corresponding pixel ordinates, a univariate linear regression mapping model from the pixel ordinates to the actual physical height is established to obtain the mapping slope and intercept; in the subsequent step S3, when calculating the proportion of waste filling in each area, the mapping model is used to convert the pixel ordinates of the material surface contour line into actual physical height values.

[0031] The image captures pixel coordinates, but engineering focuses on the actual physical height. By establishing a mapping from pixel coordinates to physical height through calibration, the material surface profile can be converted into an actual height curve, facilitating comparison and verification with hydraulic parameters.

[0032] Furthermore, before performing temporal fusion on the multi-frame material surface images in step S3, an abnormal frame filtering step is also included: for each frame of material surface image acquired within the stable window period, the variance of its Laplacian operator response is calculated as a sharpness evaluation index; when the sharpness evaluation index is lower than a preset sharpness threshold, the current frame is determined to be a motion-blurred frame and is removed; the global mean of the dark channel map of the enhanced image is calculated, and when the global mean of the dark channel map is higher than a preset occlusion threshold, the current frame is determined to be a severely occluded frame and is removed; and the temporal fusion step is performed on the remaining valid frames.

[0033] In actual data acquisition, some image frames may exhibit motion blur due to camera shake, residual vibrations from scraper movement, or be obscured by flying garbage bags or large pieces of trash. These abnormal frames, if included in the fusion process, will contaminate the results. By using sharpness assessment and occlusion detection, substandard frames can be removed before fusion, improving the reliability of the fusion results.

[0034] Furthermore, the fusion decision rule for bimodal cross-validation in step S5 is as follows: define the image modality confidence level. and hydraulic modal confidence When the percentage of the top region's blended fill in each region is greater than or equal to a preset material level threshold, the image modality is determined to be close to full load. Take the higher value; otherwise Take the lower value; when the absolute value of the pressure change rate is greater than or equal to the preset pressure threshold, the hydraulic mode is judged to be close to full load. Take the higher value; otherwise Take the lower value; the fusion judgment logic is as follows: when High value and When the value is high, a full load confirmation signal is output; when High value and When the value is low, a level one warning signal is output, the image acquisition interval is reduced, and compaction continues; when Low value and When the value is high, a level 2 warning signal is output, indicating that there may be uneven loading; when Low value and When the value is low, a normal loading signal is output, and compaction continues.

[0035] The four-way integrated judgment logic covers all possible combinations. Double high is the ideal situation, which can confirm full load. Double low indicates that there is still enough space. High image and low liquid indicates that the image judgment is close to full load but the hydraulic pressure has not yet risen. It may be that the garbage is loose and the compression ratio is low. It needs to continue compaction but should be closely monitored. Low image and high liquid indicates that the hydraulic pressure has risen but the image has not yet shown the high material level. It may be that the garbage distribution in the stuffer is uneven and a certain area is full first. It is necessary to remind the operator to check.

[0036] Furthermore, image modal confidence and hydraulic modal confidence The dynamic allocation method is as follows: based on the dust concentration estimate obtained in step S2... Image mode weights are calculated using continuous functions. Hydraulic modal weights The `clamp` function restricts the input value to the range [0,1]. When... hour, , ;when hour, , ;when At that time, the weights transition linearly and smoothly with dust concentration. The overall confidence level of the fusion decision... When the overall confidence level C is greater than or equal to the preset confidence threshold, a full load confirmation signal is output.

[0037] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] This invention overcomes the inherent shortcomings of traditional indirect inference methods such as hydraulic pressure and proximity switch methods by directly acquiring material surface images through an industrial camera installed inside the stuffing box. This provides a more accurate reflection of the material level inside the stuffing box, resulting in more intuitive and reliable judgments. For the special environment of high dust concentration inside the stuffing box, an adaptive image enhancement algorithm based on dust concentration estimation is designed. This algorithm dynamically adjusts the defogging intensity according to the dust concentration, ensuring image usability under high dust conditions while avoiding over-enhancement under low dust conditions.

[0039] By evaluating the filling ratio in sections along the height of the stuffer cavity, compared to global threshold judgment, uneven material distribution can be detected, providing early warning of local overfilling and avoiding unbalanced loading problems caused by one side filling first. Dual-modal cross-validation of image and hydraulic modes is performed, and the weight allocation of the two modes is dynamically adjusted according to dust concentration, achieving complementary advantages. Image judgment is emphasized when dust concentration is low, while hydraulic judgment is emphasized when dust concentration is high, maintaining high judgment accuracy even under complex and variable operating conditions.

[0040] By introducing multi-frame temporal fusion and cross-cycle trend analysis, not only are the random errors of a single image eliminated, but the incremental decreasing trend is also used to provide early warning of the full load state, which has better foresight and stability than simple threshold determination. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall process of the method of the present invention;

[0042] Figure 2 This is a schematic diagram of the dust adaptive enhancement treatment process;

[0043] Figure 3 This is a schematic diagram of material surface contour extraction and partitioning;

[0044] Figure 4 This is the output three-category label image;

[0045] Figure 5 This is a schematic diagram of the dual-modal fusion decision logic. Detailed Implementation

[0046] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0047] The first embodiment, referred to Figure 1As shown, a method for determining full load based on image recognition of compressive garbage truck filler includes: Step S1: Image acquisition, where an industrial camera is installed at the top center of the inner cavity of the compressive garbage truck filler, with the lens facing the material surface inside the filler. The camera is fixed by a shock-absorbing bracket to reduce the impact of vibration during compaction operations on image quality.

[0048] The dust concentration inside the filler is extremely high. Camera lenses exposed to this environment for extended periods are prone to dust and oil buildup, leading to a continuous deterioration in image quality. This embodiment employs a standard solution to address this issue: a transparent protective window is installed in front of the camera lens. This window is made of removable tempered glass and can be replaced or cleaned periodically. Alternatively, an air nozzle can be added to the outside of the protective window. Before each image acquisition, a high-pressure airflow is blown through the nozzle to remove surface dust from the protective window. The airflow lasts 0.5-1 second, with the air source being the vehicle's compressed air system, and the pressure not less than 0.4 MPa. The system periodically performs a lens cleanliness self-check, acquiring one image before and after air blowing, and comparing the sharpness of the two images. If the improvement in sharpness before and after cleaning is less than a preset cleaning threshold, it indicates that the protective window is severely contaminated and requires manual replacement.

[0049] The definition of the stabilization window period: The stabilization window period refers to the time after the scraper has completed the compaction action and retracted, during which both the material surface and dust levels tend to stabilize. The specific criterion is: after the scraper retracts, wait for the estimated dust concentration... When the value drops below 0.3 and the change in the material surface contour (the root mean square value of the difference in the ordinate of the pixel coordinates of the material surface contour lines in adjacent frames) is less than 5 pixels across 3 consecutive frames, it is considered to have entered a stable window period. In engineering practice, this stable window period generally occurs 3-8 seconds after the scraper retracts, lasting approximately 2-4 seconds. Within the stable window period, K frames of material surface images are continuously acquired using an industrial camera, where K is set to 5.

[0050] Regarding the timeout tolerance mechanism: Under harsh operating conditions (such as high-moisture waste generating large amounts of water vapor, poor scraper sealing leading to continuous dust generation, and continuous material surface collapse causing contour fluctuations), the entry conditions for the stable window period may not be met for an extended period. To avoid system congestion, a timeout handling logic is implemented: if the entry conditions for the stable window period are not met within 10 seconds after the scraper retracts, the three clearest frames from the image frames acquired within one second before and after the current time are taken as valid frames, and subsequent steps are forcibly executed. At this point, the image modality weights are used in the overall confidence calculation. Automatically reduced to 0.4, hydraulic modal weight. The value was increased to 0.6 to prioritize ensuring that the system can output judgment results.

[0051] The reason for choosing to collect data during the stable window period after the scraper retracts is that during compaction, the scraper moves violently, debris splashes, and dust concentration rises sharply, resulting in extremely poor image quality. After the scraper retracts, the dust begins to settle and the material surface stabilizes, allowing images collected at this time to better reflect the true material level.

[0052] Step S2: Dust Adaptive Enhancement

[0053] Each frame of the acquired material surface image undergoes adaptive dust enhancement processing, the process is as follows: Figure 2 As shown.

[0054] First, the dust concentration estimate for the current frame image is calculated as follows: The current frame image is converted to a grayscale image; the grayscale image is divided into 8×8 uniform sub-blocks; the grayscale mean and standard deviation of each sub-block are calculated; a preset dark threshold of 60 and a preset fluctuation threshold of 15 are used; the number of sub-blocks with a grayscale mean below 60 and a grayscale standard deviation below 15 are recorded as the number of dust region sub-blocks; the ratio of the number of dust region sub-blocks to the total number of sub-blocks (64) is calculated as the dust concentration estimate. .

[0055] Areas with high dust concentration appear as large, dark, and blurry regions in an image, with low mean grayscale and small standard deviation. Sub-block statistics can quickly estimate the overall dust concentration of the current frame, requiring minimal computation and making it suitable for real-time processing.

[0056] Then, based on the dust concentration estimate, the transmittance parameters of the dark channel prior defogging algorithm are adaptively adjusted. Specifically, the first concentration threshold is preset to 0.2, and the second concentration threshold is preset to 0.5. At that time, the lower limit of transmittance Take 0.8; when hour, Take 0.6; when hour, We set the value to 0.4. Using the adjusted lower bound of transmittance, we performed a dark channel prior dehazing algorithm to obtain the enhanced image.

[0057] Three-level adjustment is a practical compromise in engineering. When the dust concentration is low, strong defogging is not necessary, and preserving the original image information is more important. When the dust concentration is high, defogging must be strengthened, otherwise the image will be almost unusable. The middle level serves as a transition. Currently, three levels are sufficient to handle most working conditions. Too many levels would only increase the complexity of parameter adjustment.

[0058] Step S3: Extract the material surface contour and fill the partitions;

[0059] The enhanced image is processed by extracting the surface contour and calculating the area filling ratio. The process is as follows: Figure 3 As shown.

[0060] First, semantic segmentation is performed on the enhanced image, classifying each pixel into a waste surface region, a filler wall region, or a background region. In this embodiment, a semantic segmentation network based on DeepLabv3+ is used. The encoder employs a ResNet-50 pre-trained model, and the decoder uses a dilated spatial pyramid pooling (ASPP) module to extract multi-scale features. ASPP uses three parallel dilated convolutional branches with dilation rates of 6, 12, and 18, plus a global average pooling branch. The input image size is scaled to 512×512 pixels, and the output is a three-class label map of the same size as the input, as shown below. Figure 4 As shown. The training dataset was obtained by manually labeling each pixel of images collected inside the compactor of a garbage truck using a labeling tool. The collected and labeled images were divided into training, validation, and test sets in an 8:1:1 ratio. The labeling specifications are defined as follows: The garbage surface region refers to all pixel regions in the image occupied by garbage material, including the outer surface of the garbage pile and its visible internal parts; for gaps between garbage, if the gap size is less than 1 / 3 of the size of a single garbage unit, it is considered occluded by garbage and labeled as a garbage region; if the gap clearly shows the wall of the compactor, it is labeled as a wall region; labeling is performed with pixel-level precision, and the pixel at the boundary is assigned to the region where the center point of that pixel is located. These labeling specifications ensure consistency among different labelers; the labeling consistency (Cohen's Kappa coefficient) between two labelers on the same set of 100 test images reached 0.91. Data augmentation during training included random horizontal flipping, random rotation (±15°), and random brightness adjustment (±20%). The cross-entropy loss function was used, with Adam as the optimizer. The initial learning rate was 0.001, a multinomial decay strategy was employed with a power of 0.9, and the batch size was 8. After 100 epochs of training, the model with the highest mIoU on the validation set was selected as the final model. The three-class classification mIoU on the test set reached 89.3%.

[0061] The upper boundary line of the waste material surface area is extracted as the surface outline. The specific method is as follows: for each column of pixels, search from top to bottom for the first pixel belonging to the waste material surface area. The ordinate of this pixel is the height of the surface in that column. After traversing all columns, the complete surface outline is obtained.

[0062] Using the total vertical projection height H of the stuffer cavity in the image as a reference, the projection range is divided into three equal regions from bottom to top: the bottom region, the middle region, and the top region. The ratio of the number of pixels covered by the waste surface area to the total number of pixels in each region is calculated as the waste filling ratio for that region. Let the filling ratio of the bottom region be... The central area is filled with a proportion of The top area fill ratio is .

[0063] A value of N=3 is a reasonable division. The bottom region reflects the loading status of the stuffer foundation, the middle region reflects the filling progress of the main body, and the top region is directly related to whether it is close to full load. If too many regions are divided, the filling ratio of a single region will not change significantly, but it will increase the calculation burden.

[0064] Step S3': Multi-frame temporal fusion

[0065] Five frames of material surface images were continuously acquired during the stable window period. Steps S2 and S3 were performed on each frame to obtain five groups of waste filling percentages for each region.

[0066] The median of the five fill percentage values ​​corresponding to the same region across five frames is taken as the blended fill percentage for that region. For example, if the fill percentages of the top region in the five frames are 0.35, 0.38, 0.42, 0.36, and 0.40, respectively, after sorting, they become 0.35, 0.36, 0.38, 0.40, and 0.42. The median is 0.38, so the blended fill percentage for the top region is... .

[0067] Single-frame image processing is susceptible to interference from transient factors, such as a garbage bag accidentally obscuring the material surface or dust being stirred up by the movement of a scraper. Multi-frame fusion can eliminate the interference of these accidental factors. Median filtering is insensitive to outliers and is suitable for this scenario.

[0068] Step S3'': Abnormal Frame Filtering

[0069] Before performing time-series fusion on multiple material surface images, abnormal frames are filtered out.

[0070] For each frame of material surface image acquired within the stable window period, the variance of its Laplacian operator response is calculated as a sharpness evaluation index. The Laplacian operator is sensitive to high-frequency components (edges, textures) in the image; sharp images have a large variance in their Laplacian response, while blurry images have a small variance. A preset sharpness threshold of 50 is used; when the sharpness evaluation index is below 50, the current frame is determined to be a motion-blurred frame and discarded.

[0071] The global mean of the dark channel map is calculated for the enhanced image. The global mean of the dark channel map reflects the overall brightness level of the image. Severely occluded images are usually darker overall, and have a higher global mean of the dark channel map. The preset occlusion threshold is 80. When the global mean of the dark channel map is higher than 80, the current frame is determined to be a severely occluded frame and is discarded. The above-mentioned sharpness threshold of 50 and occlusion threshold of 80 are calibrated based on the input image size of 512×512 pixels. If different input resolutions are used in actual deployment, the thresholds need to be adjusted accordingly according to the resolution ratio, or the thresholds need to be recalibrated.

[0072] Perform the timing fusion step S3' on the remaining valid frames.

[0073] In actual data acquisition, some image frames may exhibit motion blur due to camera shake, residual vibrations from scraper movement, or be obscured by flying garbage bags or large pieces of trash. These abnormal frames, if included in the fusion process, will contaminate the results. By using sharpness assessment and occlusion detection, substandard frames can be removed before fusion, improving the reliability of the fusion results.

[0074] Step S4: Calculation of hydraulic pressure change rate

[0075] Real-time pressure data of the hydraulic filling cylinder in the hydraulic system is collected synchronously. Pressure values ​​are acquired at a sampling frequency of 100Hz through a pressure sensor installed on the hydraulic circuit of the filling cylinder. The raw signal acquired by the pressure sensor is preprocessed by low-pass filtering. In this embodiment, a Butterworth second-order low-pass filter with a cutoff frequency of 10Hz is used to eliminate high-frequency noise caused by hydraulic pump pulsation and directional valve impact. The filtered pressure signal is used for subsequent rate of change calculation.

[0076] The method for calculating the rate of pressure change within the current compaction cycle is as follows: During the compaction phase of the current compaction cycle, record the process of pressure rising from the initial value to the peak value, and calculate the rate of pressure increase.

[0077]

[0078] in, This represents the peak pressure of the current cycle. This is the initial pressure for the current cycle. This refers to the time it takes for the pressure to rise from its initial value to its peak value.

[0079] As the material level rises, the remaining space inside the stuffer becomes smaller and smaller. The resistance that the hydraulic system needs to overcome during each compaction gradually increases, resulting in a faster rate of pressure increase. The rate of pressure change is a better indicator of material level changes than the absolute pressure value because it eliminates the influence of factors such as different types of waste and moisture content on the pressure baseline.

[0080] Step S5: Dual-modal cross-validation

[0081] The waste filling ratio of each region obtained in step S3 and the pressure change rate obtained in step S4 are subjected to dual-modal cross-validation. The judgment logic is as follows: Figure 5 As shown.

[0082] Regarding time synchronization: Define the compaction cycle number. , No. Hydraulic pressure change rate during secondary compaction cycle Record the data for this cycle's compaction phase (during the scraper advance process). The material surface image corresponding to this cycle is acquired during the stabilization window after compaction ends and the scraper retracts. Both are assigned the same cycle number. Perform correlation pairing. Although the pressure data and image data belong to two consecutive stages of the same cycle in time, they have a physical correspondence; that is, the rate of change of hydraulic pressure reflects the first... The image reflects the compression resistance state of the waste inside the stuffer during the second compaction. The state of the material surface inside the stuffer after the second compaction is completed. Both describe the complete working condition of the same compaction cycle. Therefore, it is reasonable and feasible to cross-validate the pressure change rate and the image filling ratio under the same cycle number.

[0083] Define image modal confidence and hydraulic modal confidence When the top area is filled with a certain percentage of the blending ratio. When the level is greater than or equal to the preset material level threshold of 0.7, the image modality is determined to be close to full load. Take the highest value 1; otherwise Take the lower value of 0. The preset material level threshold of 0.7 means that 70% of the top area is already covered by waste, indicating that the material level has risen to near the top of the packer; when the pressure change rate... When the absolute value of the pressure is greater than or equal to the preset pressure threshold of 8 MPa / s, the hydraulic mode is judged to be close to full load. Take the highest value 1; otherwise Take the lowest value of 0. The method for determining the preset pressure threshold of 8MPa / s is as follows: conduct multiple packing tests under two extreme conditions, namely no-load and full-load, and count the average pressure change rate under full load. Take 70% of this average value as the threshold, taking into account both sensitivity and anti-interference ability.

[0084] The fusion determination logic is as follows:

[0085] when and When the load is full, a full load confirmation signal is output, and compaction operations are stopped.

[0086] when and If this occurs, a Level 1 warning signal is output, the image acquisition interval is reduced, and compaction continues. This situation indicates that the image indicates the load is close to full but the hydraulic pressure has not yet reached its target, possibly due to loose waste and a low compression ratio. Compaction needs to continue, but close monitoring is required.

[0087] when and At this time, a level 2 warning signal is output, indicating that there may be uneven loading. This situation indicates that the hydraulic pressure has risen but the image has not yet shown a high material level, which may be due to uneven distribution of debris in the stuffer or that a certain area has filled first. The operator needs to be alerted to check.

[0088] when and At that time, a normal loading signal is output, and compaction continues.

[0089] The four-way integrated judgment logic covers all possible combinations. A double high (high pressure, high volume) situation is the ideal one, confirming full load; a double low (low pressure, low volume) situation indicates there is still sufficient space; a high image and low liquid level situation indicates the image suggests near full load but the hydraulic pressure hasn't yet reached it, possibly due to loose waste and a low compression ratio, requiring continued compaction but close monitoring; a low image and high liquid level situation indicates the hydraulic pressure has risen but the image hasn't shown a high material level yet, possibly due to uneven waste distribution within the packer or a certain area filling first, requiring the operator to be alerted for inspection.

[0090] In the second embodiment, based on the above embodiments, cross-cycle trend analysis and full-load prediction are performed, and the fusion filling ratio of each cycle in P consecutive compaction cycles is recorded. P is 8. Calculate the fill increment sequence between two adjacent cycles. , A total of 7 incremental values ​​were obtained.

[0091] Establish an exponential decay model for the fill increment:

[0092]

[0093] in, This is the initial filling increment, which physically means the percentage of additional filling that can be added in each cycle during the initial compaction phase. The attenuation coefficient reflects the rate of decrease in filling efficiency during waste compaction. The larger the value, the faster the filling increment decays, and the closer the material surface is to saturation.

[0094] Log-linearize the increment sequence: .make , This is then transformed into a univariate linear regression. ,in, The least squares method is used to solve this problem. , ,in, This represents the incremental sample size. and They are respectively and The sample mean.

[0095] Taking a specific compaction operation as an example: the proportions of fusion filling in eight consecutive cycles were 0.12, 0.22, 0.30, 0.37, 0.42, 0.46, 0.49, and 0.51, respectively, with corresponding increment sequences of 0.10, 0.08, 0.07, 0.05, 0.04, 0.03, and 0.02. The fitted result is... , .

[0096] Based on the fitted model, predict the cumulative fill percentage after N cycles: Preset full load threshold .Will Substituting the values ​​into the solution, we get an estimated 5 more cycles needed to reach full load, i.e. .

[0097] Explanation of the two thresholds: The preset material level threshold of 0.7 in step S5 is the current judgment threshold for the image modality, used to determine whether the current frame is close to full load, which is an instantaneous judgment; the full load threshold in this step... The full-load control threshold is used to determine whether compaction operations should be stopped; this is a final value determination. The difference between the two (0.15) serves as a safety buffer zone. Within this range, the system continues compaction but does not confirm full load. The purpose is to further compact the waste through multiple compactions, thereby improving loading efficiency. When the predicted cumulative filling percentage reaches 0.85, it is considered that the compactor has no effective compression space, and only then can full load be confirmed.

[0098] Compare the prediction results with the bimodal determination results in step S5: Currently... The image modality is judged to be normal; the pressure change rate The preset pressure threshold has not been exceeded, and the hydraulic modal assessment is normal. The prediction result is consistent with the dual-modal assessment, both indicating normal loading. A normal loading signal is output, and compaction continues. When the predicted remaining number of cycles... Even if the current dual-modal judgment result has not yet reached full load, a trend warning signal is output in advance to give operators preparation time.

[0099] Regarding the values ​​of parameters in the exponential decay model: the minimum number of cycles P is 5, and 6-10 is recommended. Too few samples result in unstable fitting, while too many samples cause the model to lag in response to changes in operating conditions; the full load threshold... The attenuation coefficient is determined based on the packer's design volume and the target waste compression ratio, and is generally taken as 0.80-0.90. The typical range is 0.05-0.30. This indicates that the garbage is loose and difficult to compact. This indicates that the waste is easily compacted or the packing device is close to full capacity.

[0100] In the third embodiment, the confidence level is dynamically allocated based on the dust concentration estimate obtained in step S2. Image modal weights are calculated using continuous functions: Hydraulic modal weights .

[0101] For example, when a specific numerical value: (When there is almost no dust) , ;when (At medium dust levels) , ;when (High dust) , Weighting The linear and smooth transition within the interval avoids the jump problem caused by piecewise step jumps.

[0102] The overall confidence level of the fusion determination The overall confidence level C is used to provide a continuous reliability assessment based on the fact that both modal confidence levels are discrete values. When and and When all values ​​are high (1), a full load confirmation signal is output; when but and In case of inconsistency (i.e., one is 1 and the other is 0), the warning result of the above four-way decision logic is taken as the final output. The overall confidence level C is only used as an auxiliary reference for the warning level and does not cover the decision conclusion of the four-way decision logic.

[0103] Using a continuous function instead of a piecewise step function can avoid the weight from jumping at the threshold boundary. If a piecewise function is used, when the dust concentration fluctuates near the threshold (such as instantaneous dust caused by air blowing), the weight will switch repeatedly between two levels, causing a step change in the overall confidence level, which may cause frequent oscillations in the full load judgment signal. The continuous function in this embodiment ensures a smooth transition of the weight and improves the stability of the control system.

[0104] Dust concentration directly affects image quality, and consequently, the reliability of image judgment. Low dust concentration results in clear images and highly reliable judgments, leading to higher weighting for the image mode. Conversely, high dust concentration results in blurry images and potentially inaccurate judgments, necessitating a reduction in image weighting and an increase in hydraulic weighting. This dynamic allocation mechanism allows the system to automatically switch to a more reliable mode under harsh conditions, improving the overall robustness of the judgment.

[0105] In addition, the preprocessing steps preceding step S1 include camera calibration. Two calibration reference marks with known physical heights are set on both sides of the inner wall of the stuffer. These marks are made of a high-reflectivity material to facilitate image recognition. Let the physical heights of the two calibration reference marks be... and Images of the calibration reference markers are acquired using an industrial camera, and the pixel coordinates of the two calibration reference markers in the images are extracted. and .

[0106] Based on the physical height values ​​of the two calibration reference markers and their corresponding pixel ordinates, a univariate linear regression mapping model is established from the pixel ordinates to the actual physical height:

[0107]

[0108] Among them, the mapping slope ,intercept .

[0109] In the subsequent step S3, when calculating the proportion of waste filling in each area, the mapping model is used to convert the pixel ordinates of the material surface outline into actual physical height values, which facilitates comparison and verification with hydraulic parameters.

[0110] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A method for determining full load using image recognition of filler material in a compressed garbage truck, characterized in that, Includes the following steps: S1: During the stable window period after the scraper completes the compaction action and retracts in the compaction cycle, multiple frames of material surface images are continuously acquired by an industrial camera. S2: Perform dust adaptive enhancement processing on the material surface image acquired in each frame, specifically including: calculating the dust concentration estimate of the current frame image; adaptively adjusting the transmittance parameter of the dark channel prior dehazing algorithm according to the dust concentration estimate to obtain the enhanced image; S3: Extract the material surface contour from the enhanced image, divide the inner cavity of the filler into N regions in the image according to the height direction, and calculate the waste filling ratio of each region, where N≥2; S4: Synchronously collect real-time pressure data of the hydraulic cylinder in the hydraulic system and calculate the pressure change rate in the current compaction cycle; S5: Perform dual-modal cross-validation between the waste filling ratio of each region obtained in step S3 and the pressure change rate obtained in step S4, and output the full-load status signal according to the preset fusion judgment rule.

2. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 1, characterized in that: The specific method for calculating the dust concentration estimate of the current frame image in S2 is as follows: Convert the current frame image to a grayscale image; divide the grayscale image into M×M uniform sub-blocks, where M is a positive integer; Calculate the mean gray value and standard deviation of gray value for each sub-block; record the number of sub-blocks in the dust area whose mean gray value is lower than the preset dark threshold and whose standard deviation of gray value is lower than the preset fluctuation threshold as the number of sub-blocks in the dust area. The ratio of the number of dusty sub-blocks to the total number of sub-blocks M×M is calculated as the estimated dust concentration.

3. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 2, characterized in that: The specific method for adaptively adjusting the transmittance parameter of the dark channel prior defogging algorithm based on the dust concentration estimate in step S2 is as follows: When the estimated dust concentration is less than the first concentration threshold, the lower bound of transmittance is taken as the first value. When the estimated dust concentration is greater than or equal to the first concentration threshold and less than the second concentration threshold, the lower bound of transmittance is taken as the second value, which is less than the first value. When the estimated dust concentration is greater than or equal to the second concentration threshold, the lower bound of transmittance is taken as a third value, which is less than the second value. The first concentration threshold is less than the second concentration threshold.

4. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 1, characterized in that: The method for dividing the stuffer cavity into N regions along the height direction in the image in step S3 is as follows: Semantic segmentation is performed on the enhanced image to classify pixels into waste surface region, filler wall region and background region; the upper boundary line of the waste surface region is extracted as the surface contour line. Based on the total vertical projection height H of the stuffer cavity in the image, the projection range is divided into N regions from bottom to top, with N being 3, namely the bottom region, the middle region, and the top region; the ratio of the number of pixels covered by the waste material surface area in each region to the total number of pixels in that region is calculated as the waste filling ratio of the corresponding region.

5. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 1, characterized in that: The method for temporal fusion of multiple frames of material surface images in step S3 is as follows: Within the stable window period, K frames of material surface images are continuously acquired, where K ≥ 3; steps S2 and S3 are executed for each frame of material surface image to obtain the waste filling ratio of each region in K groups; the median of the K filling ratio values ​​corresponding to the same region in K frames is taken as the fused filling ratio of the region; or, the K filling ratio values ​​corresponding to the same region in K frames are assigned increasing weights according to the acquisition time, and the weighted average is calculated as the fused filling ratio of the region.

6. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 5, characterized in that: It also includes cross-cycle trend analysis and full load prediction steps based on the fill increment exponential decay model: Record the percentage of fusion filling in each of P consecutive compaction cycles. ,in, , ; Calculate the fill increment sequence between two adjacent cycles ,in, ; Establish an exponential decay model for filling increments. ,in, For the initial fill increment, The attenuation coefficient is obtained by curve fitting the filled increment sequence using the least squares method. and initial fill increment The optimal estimate; Based on the fitted exponential decay model, predict the future... The fill increment of each cycle Predicting the future Cumulative fill percentage after one cycle ; When the predicted cumulative fill percentage When the load is greater than or equal to the preset full load threshold, a full load warning signal is output, and the remaining number of cycles required to reach full load is calculated based on the exponential decay model. , making ,in This is the full load threshold; The full load prediction result is compared and verified with the current judgment result of the dual-modal cross-validation in step S5: when the prediction result is that it is about to be fully loaded and the dual-modal judgment result is close to full load, a full load confirmation signal is output; when the prediction result is that it is about to be fully loaded but the dual-modal judgment result is normal loading, a trend warning signal is output and monitoring continues; when the prediction result is that it is not yet fully loaded but the dual-modal judgment result is close to full load, an abnormality prompt signal is output, indicating that there may be local overloading or sensor deviation; when the prediction result is that it is not yet fully loaded and the dual-modal judgment result is normal loading, a normal loading signal is output.

7. The method for determining full load by image recognition of compressed garbage truck filler as described in claim 1, characterized in that: The fusion determination rule for the dual-modal cross-validation in step S5 is as follows: Define image modal confidence and hydraulic modal confidence ; When the percentage of the top region's blended fill is greater than or equal to a preset material level threshold, the image modality is determined to be close to full load. Take the higher value, otherwise Take the lower value; When the absolute value of the pressure change rate is greater than or equal to the preset pressure threshold, the hydraulic mode is determined to be close to full load. Take the higher value, otherwise Take the lower value; The fusion determination logic is as follows: when High value and When the value is high, a full load confirmation signal is output; when High value and When the value is low, a first-level warning signal is output, the image acquisition interval is reduced, and the compaction continues; when Low value and When the value is high, a level 2 warning signal is output to indicate that there may be uneven loading; when Low value and When the value is low, a normal loading signal is output, and compaction continues.

8. The method for determining full load by image recognition of filler material in a compressed garbage truck according to claim 7, characterized in that: The image modal confidence and hydraulic modal confidence The dynamic allocation method is as follows: Based on the dust concentration estimate obtained in step S2 Image mode weights are calculated using continuous functions. Hydraulic modal weights ; The `clamp` function restricts the input value to the range [0,1]; when hour ,when hour ,when The time weight transitions linearly and smoothly with dust concentration; the overall confidence level of the fusion decision. When the overall confidence level C is greater than or equal to the preset confidence threshold, a full load confirmation signal is output.

9. The method for determining full load by image recognition of filler material in a compressed garbage truck according to claim 1, characterized in that: A camera calibration step is included before step S1: At least two calibration reference marks with known physical heights are set on the inner wall of the stuffer; images of the calibration reference marks are acquired using an industrial camera, and the pixel ordinates of each calibration reference mark in the image are extracted; Based on the physical height value of the calibration reference mark and the corresponding pixel ordinate, a univariate linear regression mapping model from the pixel ordinate to the actual physical height is established to obtain the mapping slope and intercept. In the subsequent step S3, when calculating the proportion of waste filling in each area, the pixel ordinate of the material surface outline is converted into the actual physical height value using the mapping model.

10. The method for determining full load by image recognition of filler material in a compressed garbage truck according to claim 5, characterized in that: Before performing time-series fusion of multiple material surface images in step S3, an abnormal frame filtering step is also included: For each frame of material surface image acquired within the stable window period, the variance of its Laplacian operator response is calculated as a sharpness evaluation index. When the sharpness evaluation index is lower than the preset sharpness threshold, the current frame is determined to be a motion-blurred frame and is removed. The global mean of the dark channel map of the enhanced image is calculated. When the global mean of the dark channel map is higher than the preset occlusion threshold, the current frame is determined to be a severely occluded frame and is removed. The temporal fusion step is performed on the remaining valid frames.

Images