Sand accumulation detection method, device and equipment for sand tank of sand blending vehicle and storage medium

Abstract

The application discloses a sand accumulation detection method, device and equipment for a sand mixing truck, and a storage medium, and belongs to the technical field of image processing. The application obtains a real-time monitoring video at a sand hopper opening of the sand mixing truck, determines a to-be-compared video frame and a reference video frame without sand accumulation from the real-time monitoring video, determines that the sand mixing truck does not accumulate sand if a frame difference between the reference video frame and the to-be-compared video frame is less than a first warning threshold, and determines that the sand mixing truck accumulates sand if the frame difference between the reference video frame and the to-be-compared video frame is greater than the first warning threshold. The application solves the technical problem that sand filling work of the sand hopper of the sand mixing truck needs to be looked after all the time, and the labor cost is high.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus, equipment, and storage medium for detecting sand accumulation in the sand hopper of a sand mixing vehicle. Background Technology

[0002] During the sand filling process of the sand mixing truck, the sand filling bucket may become clogged due to sandbag fragments or other foreign objects in the sand, which will affect the work progress.

[0003] In related technologies, it is usually necessary to have construction personnel supervise the sand adding process throughout the entire process, and to promptly remove any foreign objects that cause blockages. Therefore, the current sand mixing truck sand bucket adding operation has the technical problem of requiring full-time supervision and high labor costs. Summary of the Invention

[0004] The main purpose of this application is to provide a method, device, equipment and storage medium for detecting sand accumulation in the sand hopper of a sand mixing truck, which aims to solve the technical problems of high labor costs and the need for full-time supervision in the current sand mixing truck sand hopper filling operation.

[0005] To achieve the above objectives, this application provides a method for detecting sand accumulation in the sand hopper of a sand mixing truck, the method comprising:

[0006] Obtain real-time monitoring video of the sand hopper opening of the sand mixing truck;

[0007] The video frames to be compared and the reference video frames without sand accumulation are determined from the real-time monitoring video.

[0008] If the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold, it is determined that no sand accumulation has occurred in the sand mixing truck.

[0009] If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold, it is determined that the sand-mixing truck has accumulated sand.

[0010] Optionally, the video frames to be compared are determined from the real-time monitoring video, including:

[0011] Based on a preset extraction period, real-time monitoring video frames are extracted from the real-time monitoring video to obtain a set of video frames.

[0012] From the set of video frames, identify the target real-time monitoring video frame with the smallest inter-frame difference from the reference video frame;

[0013] Use real-time monitoring video frames of the target as the video frames to be compared.

[0014] Optionally, after extracting real-time monitoring video frames from the real-time monitoring video based on a preset extraction period and obtaining a set of video frames, the method further includes:

[0015] Determine the maximum inter-frame difference between all video frames in the video frame set and the reference video frame;

[0016] The first warning threshold is determined based on the maximum inter-frame difference.

[0017] Optionally, if the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold, after determining that no sand accumulation has occurred in the sand mixing truck, the method further includes:

[0018] If no sand accumulation occurs in the sand mixing truck, the reference video frame is updated based on the video frame to be compared.

[0019] Optionally, reference video frames without sand accumulation are determined from the real-time monitoring video, including:

[0020] The first real-time monitoring video frame after the sand bucket enters the working state is used as the reference video frame.

[0021] Optionally, if the inter-frame difference between the reference video frame and the video frame to be compared is greater than a first warning threshold, then it is determined that sand accumulation has occurred, including:

[0022] If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the second warning threshold, then it is determined that the environmental conditions around the sand mixing truck are abnormal; wherein, the second warning threshold is greater than the first warning threshold.

[0023] Optionally, real-time monitoring video of the sand hopper opening of the sand mixing truck can be acquired, including:

[0024] Acquire raw real-time monitoring video;

[0025] The original real-time monitoring video frames are converted into LAB images to obtain the real-time monitoring video.

[0026] Secondly, to achieve the above objectives, this application further provides a sand accumulation detection device for a sand mixing truck hopper, the sand accumulation detection device comprising:

[0027] The acquisition module is used to acquire real-time monitoring video of the sand hopper opening of the sand mixing truck;

[0028] The first confirmation module is used to determine the video frame to be compared and the reference video frame without sand accumulation from the real-time monitoring video.

[0029] The second confirmation module is used to determine that no sand accumulation has occurred in the sand mixing truck if the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold.

[0030] The third confirmation module is used to determine that sand accumulation has occurred in the sand mixing truck if the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold.

[0031] Thirdly, to achieve the above objectives, this application further provides a sand-accumulation detection device for a sand-mixing truck hopper, comprising: a processor, a memory, and a sand-accumulation detection program for a sand-mixing truck hopper stored in the memory, wherein the sand-accumulation detection program for a sand-mixing truck hopper is executed by the processor to implement the steps of the above-mentioned sand-accumulation detection method for a sand-mixing truck hopper.

[0032] Fourthly, to achieve the above objectives, this application further provides a computer-readable storage medium storing a sand-accumulation detection program for a sand-mixing truck hopper, which, when executed by a processor, implements the aforementioned sand-accumulation detection method for a sand-mixing truck hopper.

[0033] It is easy to see that the monitoring screen will change significantly when sand accumulates in the sand hopper. This application monitors the real-time monitoring video of the sand hopper of the sand mixing truck, extracts the video frame to be compared and the reference video frame from it, and performs inter-frame difference analysis on the video frame to be compared and the reference video frame to accurately identify the change. This realizes the automatic detection of the sand accumulation state of the sand hopper of the sand mixing truck and reduces labor costs. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the sand accumulation detection equipment in the sand hopper of the sand mixing truck in this application;

[0035] Figure 2 This is a flowchart illustrating the first embodiment of the sand accumulation detection method for sand mixing trucks in this application;

[0036] Figure 3 This is a detailed flowchart of step S100 of the first embodiment of the sand accumulation detection method for sand mixing trucks in this application;

[0037] Figure 4 This is a first detailed flowchart of step S200 of the first embodiment of the sand accumulation detection method for sand mixing truck hopper of this application;

[0038] Figure 5 This is a second detailed flowchart of step S200 of the first embodiment of the sand accumulation detection method for sand mixing truck hopper of this application;

[0039] Figure 6 This is a flowchart illustrating the second embodiment of the sand accumulation detection method for sand mixing trucks in this application;

[0040] Figure 7 This is a flowchart illustrating the third embodiment of the sand accumulation detection method for sand mixing trucks in this application;

[0041] Figure 8 This is a flowchart illustrating Example 1 of the sand accumulation detection method for sand mixing trucks in this application.

[0042] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0043] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0044] The feed inlet of the sand hopper of a sand mixing truck usually has a filter screen. During the sand adding process, the filter screen may become clogged due to sand bag fragments or other foreign objects in the sand, which will affect the work progress.

[0045] In related technologies, it is usually necessary to have construction personnel supervise the sand adding process throughout the entire process, and to promptly remove any foreign objects that cause blockages. Therefore, the current sand mixing truck sand bucket adding operation has the technical problem of requiring full-time supervision and high labor costs.

[0046] The following embodiments of this application will describe the sand accumulation detection method, apparatus, equipment, and storage medium used in the technical implementation of this application:

[0047] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of the sand accumulation detection equipment in the sand hopper of the sand mixing truck, which is part of the hardware operating environment of the embodiment of this application.

[0048] like Figure 1 As shown, the sand accumulation detection device in the sand hopper of the sand mixing truck may include: a processor 1001, such as a CPU, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to establish communication between these components. The user interface 1003 may include a voice pickup module, such as a microphone array; optionally, the user interface 1003 may also be a display screen or an input unit such as a keyboard. The memory 1005 may be a high-speed RAM or a stable, non-volatile memory, such as a disk storage device. Alternatively, the memory 1005 may be a storage device independent of the aforementioned processor 1001.

[0049] It is understood that the sand accumulation detection device in the sand hopper of the sand mixing truck may also include a network interface 1004, which may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface). Optionally, the sand accumulation detection device in the sand hopper of the sand mixing truck may also include RF (Radio Frequency) circuitry, sensors, audio circuitry, a WiFi module, etc.

[0050] Those skilled in the art will understand that Figure 1The structure of the sand hopper sand accumulation detection device shown in the figure does not constitute a limitation on the sand hopper sand accumulation detection device for sand mixing vehicles. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0051] Based on, but not limited to, the hardware structure of the sand accumulation detection device in the sand hopper of a sand mixing truck, this application provides a first embodiment of a method for detecting sand accumulation in the sand hopper of a sand mixing truck. (Refer to...) Figure 2 , Figure 2 A flowchart illustrating the first embodiment of the sand accumulation detection method for sand mixing truck hopper of this application is shown.

[0052] It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown here.

[0053] In this embodiment, the method for detecting sand accumulation in the sand hopper of a sand mixing truck includes:

[0054] Step S100: Obtain real-time monitoring video of the sand hopper opening of the sand mixing truck.

[0055] In this embodiment, the sand hopper of the sand mixing truck is a screw conveyor sand hopper. When performing sand accumulation detection in the sand hopper, it is necessary to obtain real-time monitoring video of the feed inlet of the screw conveyor sand hopper to obtain information on the sand accumulation status of the feed inlet of the sand hopper, which is convenient for subsequent sand accumulation detection.

[0056] Furthermore, as a specific implementation method, refer to Figure 3 Step S100 includes:

[0057] Step S110: Obtain the original real-time monitoring video.

[0058] Step S120: Convert the video frames of the original real-time monitoring video into LAB images to obtain the real-time monitoring video.

[0059] In this embodiment, the video frames of the original real-time monitoring video are generally RGB color images. The RGB color images can be converted into LAB images to obtain real-time monitoring videos in the LAB color space.

[0060] In a specific example, a LAB image processing function can be used to convert the video frames of the acquired raw real-time monitoring video into LAB images to obtain real-time monitoring video in the LAB color space.

[0061] In another specific example, video frames can also be directly extracted from real-time monitoring videos, processed using LAB image processing functions, and stored in a set to provide a basis for subsequent sand accumulation detection.

[0062] Understandably, converting video frames into LAB color images for analysis can highlight the characteristics of sand in the sand hopper of the sand mixing truck, which can reduce the impact of lighting on sand accumulation detection to some extent when performing subsequent calculations on sand accumulation analysis.

[0063] Step S200: Identify the video frame to be compared and the reference video frame without sand accumulation from the real-time monitoring video.

[0064] In this embodiment, the video frame to be compared is a video frame image from real-time monitoring video that can be used to determine whether there is sand accumulation in the sand hopper of the sand mixing truck. The reference video frame is a video frame image in a state without sand accumulation, which serves as the comparison standard for the video frame to be compared. No sand accumulation has occurred in the sand hopper of the sand mixing truck in the reference video frame. It can be understood that by analyzing the differences between the video frame to be compared and the reference video frame, it is possible to determine whether there is sand accumulation in the sand hopper of the sand mixing truck in the video frame to be compared.

[0065] Furthermore, as a specific implementation method, refer to Figure 4 Step S200 includes:

[0066] Step S210: The first real-time monitoring video frame after the sand bucket is in working state is used as the reference video frame.

[0067] Generally speaking, sand accumulation does not usually occur when the sand mixing truck starts working. Therefore, the first frame of the sand bucket truck in the real-time monitoring video when it is in working state can be used as the reference video frame.

[0068] In a specific example, the video frame corresponding to the moment when the sand mixing truck starts working can be used as a reference video frame by obtaining the time information.

[0069] In another specific example, the image acquisition device only acquires real-time monitoring video synchronously when the sand mixing truck starts working, and the first frame of the acquired real-time monitoring video is used as the reference video frame.

[0070] Understandably, detecting sand accumulation only in real-time monitoring videos of the sand mixing truck during its operation can reduce monitoring errors.

[0071] Furthermore, as a specific implementation method, refer to Figure 4 Step S200 includes:

[0072] Step S220: Extract real-time monitoring video frames from the real-time monitoring video based on a preset extraction period to obtain a video frame set.

[0073] Step S230: From the set of video frames, determine the target real-time monitoring video frame with the smallest inter-frame difference from the reference video frame.

[0074] Step S240: Use the real-time monitoring video frame of the target as the video frame to be compared.

[0075] In this embodiment, the video frame to be compared can be selected from the video frame set of the real-time monitoring video, and the video frame set can be composed of video frames periodically extracted from the real-time monitoring video according to a preset extraction period.

[0076] Inter-frame differences can be determined using the frame difference method. Specifically, the frame difference method can be used to determine the inter-frame differences between each video frame in the video frame set and the reference video frame. The target real-time monitoring video frame with the smallest inter-frame difference from the reference video frame in the video frame set is selected as the video frame to be compared.

[0077] Understandably, by comparing each frame in the video frame set with the reference video frame frame by frame, calculating the inter-frame differences, and selecting the video frame with the smallest inter-frame difference as the comparison video frame for sand accumulation detection, the influence of non-target factors on sand accumulation detection can be reduced, and false alarms caused by false sand accumulation can be effectively avoided.

[0078] In a specific example, n video frames can be selected from the real-time monitoring video at intervals t to form a video frame set. The reference video frame can be the first video frame selected from the real-time monitoring video when the sand mixing truck starts working. Specifically, the inter-frame differences between each video frame in the video frame set and the reference video frame are determined one by one (if the video frame set contains the reference video frame, the inter-frame differences between the other video frames in the video frame set and the reference video frame are determined), and the video frame with the smallest inter-frame difference is taken as the video frame to be compared.

[0079] Inter-frame differences are calculated by comparing the differences between each pixel in two video frames. For each pixel location, the difference between its value in one frame and its value in another frame is calculated. This difference can be measured in various ways, such as using absolute difference, squared difference, etc. In this example, the inter-frame difference is determined as follows: the pixel difference between each video frame in the video frame set and the reference video frame is calculated; based on the pixel difference, the mean square error corresponding to each pixel difference in the video frame set is determined; and the video frame with the smallest mean square error is selected as the comparison video frame. Specifically, see Formula 1:

[0080]

[0081] min_score=min[MSE1,MSE2,...,MSE n ];

[0082] In Formula 1, MSE nLet N be the mean square error of the pixel difference between the nth video frame and the reference video frame in the video frame set. i R is the pixel value of the i-th pixel in this video frame. i Let j be the pixel value of the i-th pixel in the reference video frame, and j be the number of pixels in the video frame. It's understandable that video frames extracted from the same video usually have the same number of pixels. min_score is the minimum mean square error of the pixel differences between each video frame in the set and the reference video frame. The video frame corresponding to min_score is used as the comparison video frame.

[0083] Understandably, the mean square error obtained by calculating the pixel difference can quantify the degree of difference between two video frames, more accurately reflecting the similarity between video frames in the video frame set and reference video frames without sand accumulation. When the mean square error is the smallest, it means that the video frame to be compared is closest to the reference video frame in the initial sand-free state. The selected video frame to be compared can largely eliminate the influence of other interference factors, improving the accuracy of judging whether there is sand accumulation in the sand hopper of the sand mixing truck (for example, it can eliminate false detections of sand accumulation in the sand hopper caused by false sand accumulation to a certain extent).

[0084] In step S300, if the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold, it is determined that no sand accumulation has occurred in the sand mixing truck.

[0085] Step S400: If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold, it is determined that sand accumulation has occurred in the sand mixing truck.

[0086] In this embodiment, the first warning threshold is a reference value for determining whether there is accumulated sand in the sand bucket of the sand mixing truck in the video frame to be compared.

[0087] Specifically, when the difference between the video frame to be compared and the reference video frame is less than or equal to the first warning threshold, it is considered that the sand mixing truck has not accumulated sand. When the difference between the video frame to be compared and the reference video frame is greater than the first warning threshold, it is considered that the sand mixing truck has accumulated sand. At this time, a corresponding alarm message can be issued to remind relevant personnel to check and make adjustments.

[0088] In a specific example, the video frame with the smallest mean square error of the pixel difference between the extracted video frames and the reference video frame from the set of video frames extracted from the real-time monitoring video in the preset extraction period is selected as the video frame to be compared. The mean square error of the pixel difference between the video frame to be compared and the reference video frame is compared with a first warning threshold. If the mean square error is less than or equal to the first warning threshold, it is considered that no sand accumulation has occurred in the sand hopper of the sand mixing truck in the extraction period. If the mean square error is greater than the first warning threshold, it is considered that sand accumulation has occurred in the sand hopper of the sand mixing truck in the extraction period.

[0089] Understandably, subtle changes in the working environment can affect the accuracy of the first warning threshold as a standard for sand accumulation detection. The first warning threshold is determined at the very beginning of the program execution, and then executed again when the working environment changes. One way to confirm the first warning threshold is as follows:

[0090] Specifically, as a particular implementation method, the first warning threshold can be determined in the following way:

[0091] Reference Figure 5 After step S220, the following steps are also included:

[0092] Step S250: Determine the maximum inter-frame difference between all video frames in the video frame set and the reference video frame.

[0093] Step S260: Determine the first warning threshold based on the maximum value of the inter-frame difference.

[0094] Specifically, the inter-frame differences between each video frame in the video frame set and the reference video frame are determined, and the first warning threshold is determined based on the maximum value among the inter-frame differences.

[0095] In a specific example, the pixel difference between each video frame in the video frame set and the reference video frame is calculated. Based on the pixel difference, the mean square error corresponding to each pixel difference in the video frame set is determined. The mean square error is used as the inter-frame difference between each video frame in the video frame set and the reference video frame. The first warning threshold is determined based on the maximum value of the inter-frame difference. For details, see Formula 2:

[0096]

[0097] warn_value=cali_value+bas_value;

[0098] In Formula 2, cali_value is the calibration value, warn_value is the first warning value, and bas_value is the baseline warning value. In this example, the first warning value is determined jointly by the baseline warning value and the calibration value. A preset baseline warning value can be determined based on the inter-frame difference between the reference video frame when sand accumulation occurs and the video frame when sand accumulation occurs under the same working environment. The calibration value is half of the maximum value of the mean square error corresponding to the pixel difference in the video frame set. The first warning threshold is determined based on the baseline warning value and the calibration value. That is, the baseline warning value will be dynamically adjusted according to the maximum value of the mean square error corresponding to the pixel difference in the video frame set in different extraction periods.

[0099] Understandably, during sand accumulation detection, changes in the surrounding environment (such as subtle changes in ambient brightness) or the fluidity of the sand itself during the operation of the sand mixing vehicle can affect inter-frame differences. In other words, subtle changes in the surrounding environment and the fluidity of the sand itself can affect the accuracy of the baseline warning threshold as a standard for sand accumulation detection during the operation of the sand mixing vehicle. This embodiment effectively addresses this sand accumulation detection problem by determining a calibration value. During sand accumulation detection, the first warning threshold is adaptively corrected using the calibration value, improving the robustness of the system and the accuracy of sand accumulation judgment. This avoids false alarms or missed alarms caused by additional factors such as environmental changes and sand fluidity, better meeting the needs of sand accumulation detection in complex working environments.

[0100] Understandably, this embodiment monitors the real-time video of the sand hopper of the sand mixing truck, extracts the video frames to be compared and the reference video frames, and performs inter-frame difference analysis on the two frames. When the inter-frame difference between the video frame to be compared and the reference video frame is less than a preset first warning threshold, it is determined that no sand accumulation has occurred in the sand mixing truck; if it is greater than the threshold, it is determined that sand accumulation has occurred and an alarm is triggered to remind the staff to check and adjust in time. This realizes the automatic detection of the sand accumulation status of the sand hopper of the sand mixing truck, reducing labor costs. At the same time, by using a preset extraction period to select video frames from the real-time monitoring video for analysis, it is not necessary to rely on a large number of training samples, which not only ensures the real-time performance of the detection, but also effectively reduces the amount of computation and resource consumption, improving the operating efficiency of the entire system.

[0101] Furthermore, refer to Figure 6 , Figure 6 This paper presents a flowchart illustrating a second embodiment of the sand accumulation detection method for sand mixing trucks according to this application. In this embodiment, step S400 includes:

[0102] Step S410: If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the second warning threshold, then it is determined that the environmental conditions around the sand mixing truck are abnormal.

[0103] The second warning threshold is greater than the first warning threshold.

[0104] In this embodiment, the second warning threshold is a reference value for judging whether the surrounding environmental conditions are normal, and the second warning threshold is greater than the first warning threshold.

[0105] Specifically, if the inter-frame difference between the video frame to be compared and the reference video frame is less than or equal to the first warning threshold, it is determined that the sand-mixing truck has not accumulated sand. If the inter-frame difference is greater than the first warning threshold and less than or equal to the second warning threshold, it is determined that the sand-mixing truck has accumulated sand. If the inter-frame difference is greater than the second warning threshold, it is determined that the environmental conditions around the sand-mixing truck are abnormal.

[0106] During the operation of the sand mixing truck, other factors can affect the judgment of sand accumulation detection. For example, when a worker is checking the feed inlet of the sand mixing truck's hopper, the real-time monitoring video may record the image information of the worker in work clothes within the video frames of that time period. At this time, the frame difference between the recorded image and the reference video frame will increase significantly, even though no sand accumulation has occurred in the sand mixing truck. In this case, the detection result will not match the actual situation. Furthermore, when the ambient brightness suddenly changes (e.g., a sudden strong light shines into the sand mixing truck's hopper), the frame difference between the recorded image and the reference video frame will also increase significantly, even though no sand accumulation has occurred in the sand mixing truck. In this case, the detection result will also not match the actual situation. A second warning threshold can reduce the possibility of false alarms.

[0107] In a specific example, if the inter-frame difference exceeds the second warning threshold, the environmental conditions around the sand-laden vehicle are considered abnormal. In this case, sand accumulation detection can be stopped for a period of time before restarting. Specifically, if the inter-frame difference between the video frame to be compared and the reference video frame in the current extraction cycle exceeds the second warning threshold, sand accumulation detection will not be performed in the next extraction cycle. Instead, the first frame after the next extraction cycle will be used as the latest reference video frame for subsequent sand accumulation detection. Understandably, the calibration value mentioned above, cali_value, will also be automatically adjusted in this case.

[0108] It is understandable that this embodiment takes into account the impact of non-target factors on the detection results in practical applications, thereby avoiding false alarms caused by temporary interference.

[0109] Furthermore, refer to Figure 7 , Figure 7 This paper presents a flowchart illustrating a third embodiment of the sand accumulation detection method for a sand mixing truck hopper according to the present application. In this embodiment, after step S300, the method further includes:

[0110] In step S500, if no sand accumulation occurs in the sand mixing truck, the reference video frame is updated based on the video frame to be compared.

[0111] In this embodiment, if the sand-mixing truck in the real-time monitoring video within a certain extraction period does not accumulate sand, the comparison video frame used for sand accumulation detection in the real-time monitoring video within that time period is taken as the latest reference video frame, and used as the basis for sand accumulation detection of the sand-mixing truck in the real-time monitoring video of the next extraction period.

[0112] When the sand mixing truck is working, over time, a small amount of sand will accumulate on the filter screen at the feed inlet of the sand hopper. This will cause the difference between the video frame to be compared and the initial reference video frame to increase. Understandably, if the initial reference video frame is always used as the comparison object for the video frame to be compared, it may affect the accuracy of the sand accumulation detection results. Updating the reference video frame using the video frame to be compared can effectively avoid false alarms in the sand accumulation detection results.

[0113] To enable those skilled in the art to better understand the scope of protection of the claims of this application, specific implementation examples in specific application scenarios are used to explain and illustrate the technical solutions described in the claims of this application. It should be understood that the following examples are only used to explain this application and are not intended to limit the scope of protection of the claims of this application.

[0114] like Figure 8 As shown, Example 1 is a sand hopper detection system for a sand mixing truck according to this application:

[0115] In this example, the system acquires real-time monitoring video of the sand hopper opening when the sand mixing truck starts working, and extracts real-time monitoring video frames from the real-time monitoring video at a preset extraction period and places them into an array to obtain a video frame set. The LAB function is used as the image processing method to convert the real-time monitoring video frames in the video frame set from the image RGB to the LAB color space.

[0116] The first video frame obtained from the real-time monitoring video at the start of operation is used as the reference video frame. The video frames in the set of video frames collected in the extraction period are compared with the reference video frame one by one to obtain the inter-frame differences between each set of video frames and the reference video frame. The video frame with the smallest inter-frame difference is used as the video frame to be compared for sand accumulation detection.

[0117] In sand accumulation detection, if the inter-frame difference between the video frame to be compared and the reference video frame is less than or equal to the first warning threshold, it is considered that no sand accumulation has occurred in the sand-mixing vehicle during that extraction cycle. If the inter-frame difference between the video frame to be compared and the reference video frame is greater than the first warning threshold but less than or equal to the second warning threshold, sand accumulation is considered to have occurred, and a corresponding alarm message is generated. If the inter-frame difference between the video frame to be compared and the reference video frame is greater than the second warning threshold, it is considered that the environmental conditions around the sand-mixing vehicle are abnormal.

[0118] The first warning threshold consists of a preset baseline warning value and a calibration value. The baseline warning threshold is the inter-frame difference between video frames that have not accumulated sand and video frames that have accumulated sand under the same working environment. The calibration value can be a value set based on the maximum value of the inter-frame difference between the video frame to be compared and the reference video frame.

[0119] If no sand accumulation occurs in the sand mixing vehicle during the extraction cycle, the video frame to be compared within the extraction cycle is used as the reference video frame for sand accumulation detection in the next extraction cycle.

[0120] If the environmental conditions around the sand truck are abnormal during the extraction cycle, the monitoring video of the last two seconds of the extraction cycle is ignored, and the first frame of the real-time monitoring video two seconds later is used as the new reference video frame to continue the sand accumulation detection, and the calibration value in the first warning value is updated.

[0121] Understandably, this example effectively achieves automated identification and early warning of sand accumulation in the sand mixing truck by acquiring real-time monitoring video of the sand hopper opening during operation and performing periodic extraction and analysis. Utilizing LAB color space conversion highlights the characteristics of sand in the sand hopper, enhancing the sand accumulation detection effect. Comparing each frame in the video frame set with the initial reference frame to calculate inter-frame differences for sand accumulation detection reduces the impact of non-target factors. Dynamically adjusting the first warning threshold using calibration values ​​to adapt to changes in ambient brightness improves the accuracy and robustness of sand accumulation detection. Updating the reference video frame based on the sand accumulation detection results within an extraction cycle ensures continuous and accurate sand accumulation detection. For abnormal environmental conditions, the solution incorporates reasonable ignore and recovery mechanisms, guaranteeing stable system operation and efficient monitoring capabilities.

[0122] Based on the same inventive concept, this application also provides a sand accumulation detection device for sand mixing truck hoppers, the device comprising:

[0123] The acquisition module is used to acquire real-time monitoring video of the sand hopper opening of the sand mixing truck;

[0124] The first confirmation module is used to identify the video frame to be compared and the reference video frame without sand accumulation from the real-time monitoring video.

[0125] The second confirmation module is used to determine that no sand accumulation has occurred in the sand mixing truck if the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold.

[0126] The third confirmation module is used to determine that sand accumulation has occurred in the sand mixing truck if the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold.

[0127] It should be noted that the various embodiments of the sand accumulation detection device in the sand mixing truck hopper in this example, and the technical effects they achieve, can be referred to the various implementation methods of the sand accumulation detection method in the sand mixing truck hopper in the foregoing examples, and will not be repeated here.

[0128] Furthermore, this application also proposes a computer storage medium storing a sand accumulation detection program for a sand mixing truck's sand hopper. When executed by a processor, the sand accumulation detection program implements the steps of the sand accumulation detection method for a sand mixing truck's sand hopper as described above. Therefore, it will not be repeated here. Additionally, the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments of this application, please refer to the description of the method embodiments of this application. As an example, program instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0129] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0130] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided in this application, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0132] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for detecting sand accumulation in the sand hopper of a sand mixing truck, characterized in that, The method includes: Acquire real-time monitoring video of the sand hopper opening of the sand mixing truck, wherein the image frames in the real-time monitoring video are LAB images; The video frames to be compared and the reference video frames without sand accumulation are determined from the real-time monitoring video. If the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold, it is determined that the sand mixing truck has not accumulated sand. If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold, it is determined that the sand-mixing truck has accumulated sand. The step of determining the video frame to be compared from the real-time monitoring video includes: Based on a preset extraction period, real-time monitoring video frames are extracted from the real-time monitoring video to obtain a video frame set; From the set of video frames, identify the target real-time monitoring video frame with the smallest inter-frame difference from the reference video frame; The real-time monitoring video frame of the target is used as the video frame to be compared. Determine the maximum inter-frame difference between all video frames in the video frame set and the reference video frame; The first warning threshold is determined based on the maximum value of the inter-frame difference.

2. The method for detecting sand accumulation in the sand hopper of a sand mixing truck according to claim 1, characterized in that, If the inter-frame difference between the reference video frame and the video frame to be compared is less than a first warning threshold, and it is determined that the sand-mixing vehicle has not accumulated sand, the method further includes: If the sand mixing vehicle does not accumulate sand, the reference video frame is updated based on the video frame to be compared.

3. The method for detecting sand accumulation in the sand hopper of a sand mixing truck according to claim 1, characterized in that, The step of determining the video frame to be compared and the reference video frame without sand accumulation from the real-time monitoring video includes: The first real-time monitoring video frame after the sand bucket is in working state is used as the reference video frame.

4. The method for detecting sand accumulation in the sand hopper of a sand mixing truck according to claim 1, characterized in that, If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold, then it is determined that sand accumulation has occurred, including: If the inter-frame difference between the reference video frame and the video frame to be compared is greater than the second warning threshold, then it is determined that the environmental conditions around the sand mixing truck are abnormal; the second warning threshold is greater than the first warning threshold.

5. The method for detecting sand accumulation in the sand hopper of a sand mixing truck according to claim 1, characterized in that, The acquisition of real-time monitoring video of the sand hopper opening of the sand mixing truck includes: Acquire raw real-time monitoring video; The original real-time monitoring video is converted into LAB images to obtain the real-time monitoring video.

6. A sand accumulation detection device for a sand mixing truck hopper, characterized in that, The sand accumulation detection device in the sand mixing truck hopper includes: The acquisition module is used to acquire real-time monitoring video of the sand hopper opening of the sand mixing truck; The first confirmation module is used to determine the video frame to be compared and the reference video frame without sand accumulation from the real-time monitoring video. The second confirmation module is used to determine that the sand mixing truck has not accumulated sand if the inter-frame difference between the reference video frame and the video frame to be compared is less than the first warning threshold. The third confirmation module is used to determine that the sand-mixing truck has accumulated sand if the inter-frame difference between the reference video frame and the video frame to be compared is greater than the first warning threshold. The step of determining the video frame to be compared from the real-time monitoring video includes: Based on a preset extraction period, real-time monitoring video frames are extracted from the real-time monitoring video to obtain a video frame set; From the set of video frames, identify the target real-time monitoring video frame with the smallest inter-frame difference from the reference video frame; The real-time monitoring video frame of the target is used as the video frame to be compared. Determine the maximum inter-frame difference between all video frames in the video frame set and the reference video frame; The first warning threshold is determined based on the maximum value of the inter-frame difference.

7. A sand accumulation detection device for sand mixing truck hoppers, characterized in that, include: A processor, a memory, and a sand-accumulation detection program for a sand-mixing truck hopper stored in the memory, wherein the sand-accumulation detection program for a sand-mixing truck hopper is executed by the processor to implement the steps of the sand-accumulation detection method for a sand-mixing truck hopper as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a sand accumulation detection program in the sand mixing truck hopper, which, when executed by a processor, implements the sand accumulation detection method in the sand mixing truck hopper as described in any one of claims 1 to 5.

Images