A real-time detection method, medium and equipment for concrete slump in a mixer truck

Abstract

The present invention discloses a real-time detection method, medium and equipment for the slump of concrete in a mixer truck tank, which realizes high-precision dynamic detection by fusing visual and inertial measurement data. The method specifically includes: collecting video streams and IMU posture data of concrete in the mixing tank; detecting at least three positioning markers preset on the inner side of the tank mouth, constructing an affine transformation matrix based on the pixel displacement of the markers between adjacent frames, and performing motion compensation redirection on the video frames; dynamically delineating the region of interest on the concrete surface in the stable image, extracting the surface ripple wavelength, aggregate sedimentation gradient and flow motion vector features; inputting these video features into a trained neural network model to output a preliminary slump value, and combining it with dynamic calibration of the tank body motion state, and finally outputting a smoothed slump value with a confidence mark. The present invention solves the problem that traditional methods are greatly disturbed by the shaking of the tank body and have measurement lags, and is suitable for real-time quality monitoring during the driving process of mixer trucks.

Description

Technical Field

[0001] The present invention relates to the field of concrete quality detection, and in particular to a method, medium and equipment for real-time detection of concrete slump in a mixer truck tank. Background Art

[0002] During concrete construction, slump is a key indicator of concrete performance, directly impacting pumpability and structural quality. Traditional slump testing relies primarily on manual slump cone testing, which suffers from measurement lag and a lack of real-time feedback. This makes it particularly unsuitable for dynamic monitoring during concrete mixer truck transportation.

[0003] Some existing vision-based slump detection methods estimate slump by analyzing the concrete surface morphology. However, the continuous rotation and shaking of the concrete mixer truck during operation and driving can cause severe motion blur and perspective distortion in the captured video images, significantly affecting feature extraction accuracy. Furthermore, existing methods often use static models, failing to consider the impact of dynamic factors such as the concrete's rotational speed and tilt angle on its rheological properties. This can lead to significant deviations between the measured results and the actual slump.

[0004] Therefore, there is an urgent need for a real-time slump detection method that can overcome the interference of tank motion, integrate multimodal sensor data, and realize dynamic correction to meet the needs of modern construction for precise control of concrete quality. Summary of the Invention

[0005] In view of the above problems, the present invention provides a real-time detection method, medium and equipment for the slump of concrete in a mixer truck tank, so as to solve the problem of insufficient accuracy of the existing slump detection method in a mixer truck tank.

[0006] To achieve the above objectives, in a first aspect, the present application provides a method for real-time detection of the slump of concrete in a mixer truck tank, the method comprising the following steps:

[0007] Collect video stream data of concrete in the mixer tank and IMU attitude data of the tank, the video stream data includes multiple video frame images, and the IMU attitude data includes the roll angle and rotation speed of the tank;

[0008] detecting at least three positioning mark points preset on the inner wall of the mixing tank as reference corner points, the positioning mark points being spaced apart on the inner circumference of the tank mouth, constructing an affine transformation matrix based on pixel displacements of the reference corner points between adjacent video frame images, and redirecting the video frame images according to the affine transformation matrix to obtain redirected video frame images;

[0009] Delineating a region of interest on the concrete surface in the redirected video frame image, and extracting video features corresponding to the concrete from the region of interest, the video features including concrete surface ripple wavelength, aggregate settlement gradient, and flow motion vector;

[0010] The video features are input into the trained neural network model, and the preliminary slump value of the concrete in the current mixer tank is output. The preliminary slump value is corrected using formula (1) to obtain the final slump value. The formula (1) is as follows:

[0011] S final =S pred ×(1+a×rpm / rpm base )+b×sin(θ);

[0012] Among them, S final is the final slump value, S pred is the initial slump value, rpm is the current tank speed, rpm base is the set reference speed, a and b are experimental calibration parameters, the value range of a is [0.02, 0.05], the value range of b is [2, 5], and θ is the roll angle;

[0013] Output smoothed slump values ​​at a preset time period and mark the confidence level.

[0014] Furthermore, delineating the region of interest on the concrete surface in the redirected video frame image includes the following steps:

[0015] Determining an initial reference area boundary size, and laterally adjusting the reference area boundary size according to the roll angle and the rotation speed to obtain a first area;

[0016] Calculate the sinking velocity of the concrete based on the optical flow method, and adjust the longitudinal boundary of the first area according to the sinking velocity to obtain the second area;

[0017] The Kalman filter is used to process the mutation position of the second area so that the range of the processed area is smaller than the boundary of the video frame image, thereby obtaining the region of interest.

[0018] Furthermore, the neural network model is a 3D-CNN network model, and the video features are input into the trained neural network model to output the preliminary slump value of the concrete in the current mixer tank, including:

[0019] The 3D-CNN network model performs the following steps:

[0020] The continuously extracted surface ripple wavelength, aggregate settlement gradient, and flow motion vector are aligned according to the time step to form a time series feature matrix with a dimension of T×3, where T is the number of video frames. Each feature dimension is then normalized to obtain a standardized time series feature matrix.

[0021] The normalized temporal feature matrix is ​​passed through a bidirectional LSTM layer to extract temporal dependency features, and the key frames are weighted by an attention mechanism to output a first-dimensional feature vector. The region of interest is passed through an expanded convolution layer and a spatiotemporal GRU layer in sequence to output a second-dimensional feature vector.

[0022] The first-dimensional feature vector and the second-dimensional feature vector are concatenated to obtain a fused feature vector, which is input into a fully connected network. After regularization processing with random neuron discarding, a preliminary slump prediction value is output.

[0023] Furthermore, the video frame image is redirected according to the affine transformation matrix, and the video frame image after the redirection processing is obtained includes:

[0024] Applying the affine transformation matrix to all pixel coordinates of the current frame, and performing coordinate mapping using bilinear interpolation;

[0025] Adopt edge pixel replication strategy to fill the vacant area after the boundary mapping of the current frame image;

[0026] The affine transformation matrix corresponding to multiple consecutive frames of video frame images is subjected to sliding average filtering to smooth the motion compensation trajectory.

[0027] Furthermore, the concrete surface corrugation wavelength is calculated according to the following method:

[0028] Perform Fourier transform or wavelet transform on the video frame image sequence to extract the spatial wavelength corresponding to the main frequency component;

[0029] Identifying corrugation peaks and valleys by edge detection or ridge tracking algorithms, calculating the average of the distances between adjacent peaks, and obtaining the wavelength of the corrugations on the concrete surface;

[0030] The aggregate settlement gradient is calculated according to the following method:

[0031] The HSV color space threshold segmentation algorithm was used to separate the aggregate and paste areas, and the noise was removed through morphological opening operation, retaining the outline of aggregate with a particle size greater than 2 mm.

[0032] The vertical displacement field of aggregate during collapse is calculated based on the optical flow method, and the sinking velocity v of the center of mass of the aggregate is calculated according to the time series. z (t);

[0033] Fitting the curve of aggregate sinking depth z(t) versus time t, and taking the maximum value of its first-order derivative as the aggregate settlement gradient;

[0034] The flow motion vector is calculated according to the following method:

[0035] Apply the dense optical flow algorithm to adjacent video frame images to obtain the motion vector (Δx, Δy) of each pixel in the current frame and the previous frame;

[0036] Perform principal component analysis on the motion vector, take the first principal component direction as the dominant flow direction, and calculate the mean modulus of the vector in the dominant flow direction. and standard deviation σ v , normalized to obtain the flow motion vector, the calculation formula is as follows:

[0037] ;

[0038] in, is the angle between the dominant flow direction and the gravity direction, F flow is the flow motion vector.

[0039] Furthermore, the smoothed slump value is output at a preset time period and marked with a confidence level including:

[0040] The slump value is output in a preset time period, and is smoothed using first-order lag filtering and median filtering to obtain the final slump value;

[0041] The marking confidence levels include:

[0042] A weighted calculation is performed based on the completeness of video feature extraction, the credibility of IMU posture data, and the variance of the slump value output by the neural network model to obtain the confidence level, and the data with a confidence level exceeding the preset level are screened out as valid data.

[0043] Furthermore, before collecting the video stream data of the concrete in the mixer tank, the method further includes:

[0044] The reference image of the calibration plate is captured by the visual sensor, and the clarity index Q of the reference image is calculated. clarity and stain coverage R dirt ;

[0045] The clarity index Q clarity Calculated by Laplace variance method, the calculation formula is as follows:

[0046] ;

[0047] Where I is the grayscale image, is the Laplace operator;

[0048] The stain coverage R dirt is the area ratio of the dirty area on the reference image;

[0049] When the preset cleaning conditions are met, the cleaning alarm of the visual sensor is triggered and the subsequent slump value detection process is suspended. The preset cleaning conditions include the clarity index Q clarity Less than the preset clarity or stain coverage R dirt Greater than the preset coverage rate;

[0050] When the preset cleaning condition is not met, the pixels in the dirty area are shielded during the subsequent extraction of video features, and only the pixels in the non-dirty area are selected for calculation.

[0051] Furthermore, when the preset cleaning conditions are met, the automatic film changing device is started to replace the lens film of the visual sensor, and the automatic film changing device includes:

[0052] a feed reel for storing unused cleaning membranes and driven to rotate by a first servo motor electrically connected to the control board;

[0053] The scrap reel is used to recycle the contaminated film. A constant tension of 0.5-1N is provided by a magnetic powder brake, which is connected to the tension sensor signal. A stripping blade is also provided on the film movement path between the scrap reel and the lens.

[0054] A guide roller assembly, comprising at least two rollers supported by bearings, for maintaining the flatness of the film conveying path, the two rollers being symmetrically distributed between the feed reel and the waste reel;

[0055] The diaphragm positioning mechanism includes an infrared radiation sensor and a main drive wheel. The infrared radiation sensor is installed at a preset distance in front of the lens and is used to detect the edge position of the diaphragm. The preset distance ranges from 3 to 5 mm. The main drive wheel is coaxially connected to the second servo motor and is used to control the stepping conveyance of the cleaning diaphragm according to the feedback signal of the infrared radiation sensor.

[0056] The hot pressing bonding head includes an annular heating plate and a pressure sensor. The annular heating plate is maintained at a preset temperature range by a temperature control module, and the pressure sensor is used to provide real-time feedback of the bonding pressure to the control board.

[0057] The ultrasonic cutting knife is driven to rise and fall by a thin cylinder, and the distance between the cutting track and the edge of the lens is in the range of [1.8, 2.2] mm. The thin cylinder is connected to the air circuit of the solenoid valve.

[0058] The control panel is used to issue a material shortage alarm when the amount of waste reel recovered reaches a preset proportional threshold of the initial length of the feed reel, or to terminate the film replacement process when the pressure sensor detects that the pressure abnormality lasts for a preset time.

[0059] In a second aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for real-time detection of concrete slump in a mixer truck tank as described in the first aspect of the present application.

[0060] In a third aspect, the present application provides an electronic device on which a computer program is stored, comprising a processor and a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, the method for real-time detection of the slump of concrete in a mixer truck tank as described in the first aspect of the present application is implemented.

[0061] Different from the existing technology, the above technical solution provides a real-time detection method, medium and equipment for the slump of concrete in a mixer truck tank, which realizes high-precision dynamic detection by fusing visual and inertial measurement data. The method specifically includes: collecting video streams and IMU posture data of concrete in the mixing tank; detecting at least three positioning markers preset on the inside of the tank mouth, constructing an affine transformation matrix based on the pixel displacement of the markers between adjacent frames, and performing motion compensation redirection on the video frames; dynamically delineating the area of ​​interest on the concrete surface in the redirected stable image, extracting the surface ripple wavelength, aggregate sedimentation gradient and flow motion vector features; inputting these video features into a trained neural network model to output a preliminary slump value, and combining it with dynamic calibration of the tank body motion state, and finally outputting a smoothed slump value with a confidence mark. The present invention solves the problem that traditional methods are greatly disturbed by tank body shaking and have measurement lags, and is suitable for real-time quality monitoring during the driving process of mixer trucks.

[0062] The above-mentioned description of the invention content is only an overview of the technical solution of the present invention. In order to enable ordinary technicians in this field to more clearly understand the technical solution of the present invention, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned objects and other objects, features and advantages of the present invention easier to understand, the following is an explanation in combination with the specific implementation methods and drawings of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings are only used to illustrate the principles, implementations, applications, features, and effects of the specific embodiments of the present invention and other related contents, and are not to be considered as limiting the present invention.

[0064] In the drawings of the specification:

[0065] Figure 1This is a first flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0066] Figure 2 is a second flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0067] Figure 3 is a third flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0068] Figure 4 is a fourth flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0069] Figure 5 is a fifth flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0070] Figure 6 is a sixth flow chart of a method for real-time detection of concrete slump in a mixer truck tank according to a specific embodiment;

[0071] Figure 7 A schematic diagram of a module of an electronic device involved in a specific embodiment;

[0072] The reference numerals in the above drawings are described as follows:

[0073] 10. Electronic equipment;

[0074] 101. Processor;

[0075] 102. Storage medium. DETAILED DESCRIPTION

[0076] In order to explain in detail the possible application scenarios, technical principles, specific solutions that can be implemented, and the purpose and effects of the present invention, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present invention and are therefore only examples and are not intended to limit the scope of protection of the present invention.

[0077] References to "embodiments" herein mean that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the term "embodiment" in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or relevance to other embodiments. In principle, in the present invention, as long as there are no technical contradictions or conflicts, the various technical features mentioned in the embodiments can be combined in any manner to form a corresponding implementable technical solution.

[0078] Unless otherwise defined, the technical terms used herein have the same meanings as those generally understood by those skilled in the art to which the present invention belongs. The use of relevant terms herein is only for describing specific embodiments and is not intended to limit the present invention.

[0079] In the description of the present invention, the term "and / or" is used to describe a logical relationship between objects, indicating that three possible relationships exist. For example, A and / or B means: A exists, B exists, and both A and B exist. Furthermore, the character " / " generally indicates that the objects are in a logical "or" relationship.

[0080] In the present invention, terms such as "first" and "second" are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0081] Without further restrictions, in the present invention, the words "include", "comprise", "have" or other similar expressions used in sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0082] In the present invention, expressions such as "greater than," "less than," and "exceed" are understood to exclude the number itself; expressions such as "above," "below," and "within" are understood to include the number itself. Furthermore, in the description of the embodiments of the present invention, "multiple" means two or more (including two), and similar expressions related to "multiple," such as "multiple groups" and "multiple times," are also understood in this manner, unless otherwise specifically defined.

[0083] In the description of the embodiments of the present invention, the space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present invention or facilitating the reader's understanding, and do not indicate or imply that the device or component referred to must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as limiting the embodiments of the present invention.

[0084] Unless otherwise expressly specified or limited, in the description of the embodiments of the present invention, the terms "installed", "connected", "connected", "fixed", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be the internal connection between two elements or the interaction relationship between two elements. For those skilled in the art of the technology to which the present invention belongs, the specific meanings of the above terms in the embodiments of the present invention can be understood according to specific circumstances.

[0085] In the first aspect, Figure 1 As shown, the present application provides a method for real-time detection of the slump of concrete in a mixer truck tank, the method comprising the following steps:

[0086] S101: Collecting video stream data of concrete in a mixer truck tank and IMU attitude data of the tank body, wherein the video stream data includes multiple video frame images, and the IMU attitude data includes the roll angle and rotation speed of the tank body.

[0087] In step S101, video stream data from the concrete surface can be collected using a high-frame-rate industrial camera (e.g., 120 fps) mounted on the tank opening to capture rapidly changing flow characteristics. IMU attitude data can be collected using a 6-axis IMU sensor (including a gyroscope and accelerometer) mounted on the tank, recording the roll angle θ and rotational speed (rpm) in real time, with the sampling frequency synchronized with the video.

[0088] S102: Detect at least three positioning mark points preset on the inner wall of the mixing tank as reference corner points, where the positioning mark points are spaced apart on the inner circumference of the tank mouth, construct an affine transformation matrix based on the pixel displacement of the reference corner points between adjacent video frame images, and redirect the video frame images according to the affine transformation matrix to obtain the video frame images after redirection processing.

[0089] In step S102, three high-contrast QR code markers (such as ArUco markers) can be evenly spaced around the inside circumference of the can mouth. The OpenCV library can be used to detect the pixel coordinates of the markers in each frame. For example, the marker diameter is 10 cm, and adjacent markers are spaced 120° apart around the inside circumference of the can mouth, ensuring that at least two markers are visible at any rotation angle. The translation, rotation, and scaling parameters are then calculated based on the pixel displacements (e.g., Δx and Δy) of the markers between adjacent frames to construct the affine transformation matrix M. For example, the expression for M is as follows:

[0090] ;

[0091] Where α represents the relative rotation angle of the tank between adjacent video frames, in radians. Its magnitude can be calculated by the pixel displacement of the positioning marker. Δx and Δy represent the translation of the marker between adjacent video frames (in pixels), which is used to compensate for the tank's translational motion.

[0092] S103: Delineating an area of ​​interest on the concrete surface in the redirected video frame image, and extracting video features corresponding to the concrete from the area of ​​interest, wherein the video features include concrete surface ripple wavelength, aggregate settlement gradient, and flow motion vector.

[0093] By dynamically defining the center area of ​​the tank bottom as the region of interest, the interference of concrete adhered to the tank wall can be avoided when extracting video features, and the accuracy of subsequent slump value calculation can be improved.

[0094] S104: Inputting the video features into the trained neural network model, and outputting a preliminary slump value of the concrete in the current mixer tank.

[0095] Preferably, the neural network model is a 3D-CNN model. In step S104, the preliminary slump value is corrected using formula (1) to obtain the final slump value. The formula (1) is as follows:

[0096] S final =S pred ×(1+a×rpm / rpm base )+b×sin(θ);

[0097] Among them, S final is the final slump value, S pred is the initial slump value, rpm is the current tank speed, rpm base is the set reference speed, a and b are experimental calibration parameters, the value range of a is [0.02, 0.05], the value range of b is [2, 5], and θ is the roll angle. Preferably, a is the speed influence coefficient, which can be 0.03, and b is the inclination influence coefficient, which can be 3, rpm base The value of (reference speed) can be 2rad / s.

[0098] S105: Outputting the smoothed slump value in a preset time period and marking the confidence level.

[0099] Through smoothing, the slump value can be dynamically corrected according to the movement state of the tank, making the measurement result more accurate.

[0100] In some embodiments, as Figure 2As shown, delineating the region of interest on the concrete surface in the redirected video frame image includes the following steps:

[0101] Step S201: determining an initial reference area boundary size, and laterally adjusting the reference area boundary size according to the roll angle and rotation speed to obtain a first area;

[0102] Step S202: Calculate the sinking velocity of the concrete based on the optical flow method, and adjust the longitudinal boundary of the first area according to the sinking velocity to obtain the second area;

[0103] Step S203: applying Kalman filtering to the mutation position of the second region so that the range of the processed region is smaller than the boundary of the video frame image, thereby obtaining the region of interest.

[0104] Through step S201, it is possible to prevent the region of interest from deviating from the actual liquid surface of the concrete due to the tilting or rotation of the tank body; through step S203, it is possible to adapt to the settlement characteristics of different slumps to avoid the region of interest including the solidified underlying concrete due to the sinking of aggregate; through step S203, it is possible to eliminate the influence of single-frame video frame image jitter and ensure a smooth transition of the region of interest.

[0105] In some embodiments, the neural network model is a 3D-CNN network model, and the video features are input into the trained neural network model to output the preliminary slump value of the concrete in the current mixer tank, including:

[0106] like Figure 3 As shown, the 3D-CNN network model performs the following steps:

[0107] Step S301: Align the continuously extracted surface ripple wavelength, aggregate sedimentation gradient, and flow motion vector according to the time step to form a time series feature matrix with a dimension of T×3, where T is the number of video frames, and standardize each feature dimension to obtain a standardized time series feature matrix.

[0108] In step S301, when aligning by time step, a 30×3 matrix can be constructed using a 1-second time window (T=30 frames, assuming a 30 fps video frame rate). Missing frames are supplemented using linear interpolation. During normalization, Z-score normalization can be performed on each feature dimension. Normalization helps eliminate dimensional differences and improves model convergence speed.

[0109] Step S302: The standardized temporal feature matrix is ​​passed through a bidirectional LSTM layer to extract temporal dependency features, and the key frames are weighted by an attention mechanism to output a first-dimensional feature vector, and the region of interest is passed through an expanded convolution layer and a spatiotemporal GRU layer in sequence to output a second-dimensional feature vector.

[0110] In step S302, the first-dimensional feature vector is output through the time-dependent branch, the time-dependent branch bidirectional LSTM layer and the attention mechanism. The input of the bidirectional LSTM layer is a 30×3 matrix, the number of hidden layer units is 64, and the output is a 30×128 time-series feature. The attention mechanism calculates the importance weight of each frame. For example, the weight of the frame with rapid settlement of aggregate at high slump is higher. After weighting, the first-dimensional feature vector (the dimension value can be 256) is obtained.

[0111] The second-dimensional feature vector is output by the spatial dynamic branch, which consists of a dilated convolutional layer and a spatiotemporal GRU layer. The dilated convolutional layer applies a 3×3 convolution kernel with a dilation rate of 2 to the image region of interest to extract multi-scale spatial features. The spatiotemporal GRU layer feeds the convolutional features into the GRU at time steps, capturing local motion patterns and outputting a second-dimensional feature vector.

[0112] Long-term dependencies are modeled through bidirectional LSTM, and dilated convolution is used to enhance the receptive field and overcome local occlusion problems.

[0113] Step S303: concatenate the first-dimensional feature vector and the second-dimensional feature vector to obtain a fused feature vector, input the fused feature vector into a fully connected network, and output a preliminary slump prediction value after regularization processing with random neuron discarding.

[0114] In step S303, the concatenated fused feature vector is compressed to 64 dimensions using a fully connected layer. During the training phase, neurons are randomly dropped with a probability of p = 0.5 to prevent overfitting. Finally, the fully connected layer outputs a scalar value, which is mapped to the range [0, 1] using a sigmoid function. This output is a preliminary slump prediction. By integrating spatiotemporal features, the robustness of the prediction is improved.

[0115] In some embodiments, the video frame image is redirected according to the affine transformation matrix to obtain the redirected video frame image, which includes: applying the affine transformation matrix to all pixel coordinates of the current frame, and using bilinear interpolation to perform coordinate mapping; using an edge pixel replication strategy to fill the vacant area after the boundary mapping of the current frame image, and performing sliding average filtering on the affine transformation matrices corresponding to the video frame images of multiple consecutive frames to smooth the motion compensation trajectory.

[0116] In this embodiment, bilinear interpolation is used for coordinate mapping to eliminate geometric distortion caused by rotation. By processing vacant areas, black edges can be prevented from interfering with subsequent detection of regions of interest. Smoothing filtering can suppress high-frequency jitter and improve the accuracy of subsequent parameter calculations.

[0117] like Figure 4As shown, the concrete surface corrugation wavelength is calculated according to the following method:

[0118] Step S401: performing Fourier transform or wavelet transform on the video frame image sequence to extract the spatial wavelength corresponding to the main frequency component;

[0119] Step S402: Identify corrugation peaks and valleys using an edge detection algorithm or a ridge tracking algorithm, calculate the average of the distances between adjacent peaks, and obtain the wavelength of the corrugations on the concrete surface.

[0120] Before step S401 , Gaussian filtering may be performed on the redirected region of interest to eliminate high-frequency noise.

[0121] In step S401, when calculating the spatial wavelength corresponding to the main frequency component, 2D-FFT can be applied to a single video frame image to calculate the radial power spectrum and extract the main frequency component, where the spatial wavelength is the reciprocal of the main frequency component.

[0122] In step S402, the Steger algorithm can be used to extract corrugation ridges, and the minimum peak-to-valley contrast threshold is set to grayscale level 15. The mean of the distance between adjacent ridges is weighted averaged with the spatial wavelength corresponding to the main frequency component to obtain the concrete surface corrugation wavelength.

[0123] Through the above scheme, the error of the calculated concrete surface corrugation wavelength can be effectively reduced by joint analysis in the frequency domain and spatial domain.

[0124] like Figure 5 As shown, the aggregate settlement gradient is calculated according to the following method:

[0125] Step S501: using the HSV color space threshold segmentation algorithm to separate the aggregate and slurry areas, removing noise through morphological opening operation, and retaining the outline of aggregates with a particle size greater than 2 mm;

[0126] Step S502: Calculate the vertical displacement field of the aggregate during the collapse process based on the optical flow method, and calculate the sinking velocity v of the center of mass of the aggregate in time series. z (t);

[0127] Step S503: fitting a curve of the aggregate sinking depth z(t) versus time t, and taking the maximum value of its first-order derivative as the aggregate settlement gradient.

[0128] Through the above scheme, concrete with different slumps can be further distinguished.

[0129] like Figure 6 As shown, the flow motion vector is calculated according to the following method:

[0130] Step S601: Apply a dense optical flow algorithm to adjacent video frame images to obtain the motion vector (Δx, Δy) of each pixel in the current frame and the previous frame;

[0131] Step S602: Perform principal component analysis on the motion vector, take the first principal component direction as the dominant flow direction, and calculate the average modulus of the vector in the dominant flow direction. and standard deviation σ v , normalized to obtain the flow motion vector.

[0132] In step S602, the calculation formula is as follows:

[0133] ;

[0134] in, is the angle between the dominant flow direction and the gravity direction, F flow is the flow motion vector.

[0135] Through the above scheme, when calculating the flow motion vector, only the slurry area can be analyzed to exclude aggregate interference. At the same time, the computational efficiency of flow characterization can be effectively improved through principal component analysis.

[0136] In some embodiments, outputting the smoothed slump value in a preset time period and marking the confidence level includes: outputting the slump value in a preset time period and smoothing it using a first-order lag filter and a median filter to obtain a final slump value.

[0137] Preferably, when the IMU sensor detects that the current tank body is vibrating violently, the coefficient of the first-order lag filter can be selected as 0.2 to enhance smoothness, while in a stable state the filter coefficient can be selected as 0.5 to improve the response speed.

[0138] The marking confidence levels include:

[0139] A weighted calculation is performed based on the completeness of video feature extraction, the credibility of IMU posture data, and the variance of the slump value output by the neural network model to obtain the confidence level, and the data with a confidence level exceeding the preset level are screened out as valid data.

[0140] The confidence level ranges from [0,1]. The confidence level can be expressed as a numerical value. Different processing strategies can be used for slump values ​​with different confidence levels output by the model. Specifically, when the confidence level is below 0.6, the slump data is discarded; when the confidence level is above 0.8, the data is directly output; and when the confidence level is between [0.6,0.8], a prompt message is issued, prompting manual review to confirm the data validity.

[0141] In some embodiments, before collecting the video stream data of the concrete in the mixer truck tank, the method further includes:

[0142] The reference image of the calibration plate is captured by the visual sensor, and the clarity index Q of the reference image is calculated. clarity and stain coverage R dirt ;

[0143] The clarity index Q clarity Calculated by Laplace variance method, the calculation formula is as follows:

[0144] ;

[0145] Where I is the grayscale image, is the Laplace operator;

[0146] The stain coverage R dirt is the area ratio of the dirty area on the reference image;

[0147] When the preset cleaning conditions are met, the cleaning alarm of the visual sensor is triggered and the subsequent slump value detection process is suspended. The preset cleaning conditions include the clarity index Q clarity Less than the preset clarity or stain coverage R dirt Greater than the preset coverage rate;

[0148] When the preset cleaning condition is not met, the pixels in the dirty area are shielded during the subsequent extraction of video features, and only the pixels in the non-dirty area are selected for calculation.

[0149] Through the above method, it can be ensured that the model will only calculate the slump when the camera can capture a clear image of the concrete, so that the calculated slump is more accurate. If there is dirt in the captured image, the cleaning alarm of the visual sensor will be triggered, and the subsequent slump value detection process will be suspended. The slump value calculation will not be restarted until the dirtiness of the captured image is effectively resolved.

[0150] Preferably, when the preset cleaning conditions are met, the automatic film changing device is started to replace the lens film of the visual sensor, and the automatic film changing device includes:

[0151] a feed reel for storing unused cleaning membranes and driven to rotate by a first servo motor electrically connected to the control board;

[0152] The scrap reel is used to recycle the contaminated film. A constant tension of 0.5-1N is provided by a magnetic powder brake, which is connected to the tension sensor signal. A stripping blade is also provided on the film movement path between the scrap reel and the lens.

[0153] A guide roller assembly, comprising at least two rollers supported by bearings, for maintaining the flatness of the film conveying path, the two rollers being symmetrically distributed between the feed reel and the waste reel;

[0154] The diaphragm positioning mechanism includes an infrared radiation sensor and a main drive wheel. The infrared radiation sensor is installed at a preset distance in front of the lens and is used to detect the edge position of the diaphragm. The preset distance ranges from 3 to 5 mm. The main drive wheel is coaxially connected to the second servo motor and is used to control the stepping conveyance of the cleaning diaphragm according to the feedback signal of the infrared radiation sensor.

[0155] The hot pressing bonding head includes an annular heating plate and a pressure sensor. The annular heating plate is maintained at a preset temperature range by a temperature control module, and the pressure sensor is used to provide real-time feedback of the bonding pressure to the control board.

[0156] The ultrasonic cutting knife is driven to rise and fall by a thin cylinder, and the distance between the cutting track and the edge of the lens is in the range of [1.8, 2.2] mm. The thin cylinder is connected to the air circuit of the solenoid valve.

[0157] The control panel is used to issue a material shortage alarm when the amount of waste reel recovered reaches a preset proportional threshold of the initial length of the feed reel, or to terminate the film replacement process when the pressure sensor detects that the pressure abnormality lasts for a preset time.

[0158] During the use of the automatic film changing device, the control board first sends a pulse signal to the first servo motor to drive the feed reel to rotate. The unused clean film is smoothly unfolded by the guide roller group. After the infrared radiation sensor detects that the clean film reaches a position about 3 mm in front of the lens, the main drive wheel intervenes and achieves a repeat positioning accuracy of 0.05 mm through the 17-bit absolute encoder.

[0159] The magnetic powder brake dynamically adjusts the current based on feedback from the tension sensor (sampling period 10ms). The scrap reel rotates counterclockwise under the 0.5-1N tension provided by the magnetic powder brake. The stripping blade contacts the contaminated film at a 30° angle, with a peeling force controlled within the range of 1.2-1.5N. The stripping blade is fixed to the film path between the lens and the scrap reel.

[0160] The control board then controls the hot pressing laminating head to press downward, activating the annular heating plate for 2 seconds to bond the unused cleaning film to the edge of the lens. A pressure sensor monitors the laminating pressure in real time, ensuring it remains constant at around 5N. The control board then controls a micro-cylinder to push the ultrasonic cutting blade to sever the remaining contaminated film. Once cutting is complete, the limit switch resets the position.

[0161] Preferably, the automatic film changing device is entirely encapsulated in an aluminum alloy protective shell, and a detachable dust box is provided at the bottom of the shell for collecting film scraps generated by cutting.

[0162] In a second aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for real-time detection of concrete slump in a mixer truck tank as described in the first aspect of the present invention.

[0163] The computer-readable storage medium may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories.

[0164] The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a ferromagnetic random access memory (FRAM), a flash memory, a magnetic surface storage, an optical disc, or a compact disc read-only memory (CD ROM); the magnetic surface storage may be a magnetic disk storage or a magnetic tape storage.

[0165] The volatile memory may be a random access memory (RAM) that is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronized dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM). The computer-readable storage medium described in the embodiments of the present invention is intended to include these and any other suitable types of memory.

[0166] like Figure 7 As shown, in a third aspect, the present invention provides an electronic device 10, comprising a processor 101 and a storage medium 102, wherein the storage medium stores a computer program, and when the computer program is executed by the processor, the method for real-time detection of the slump of concrete in a mixer truck tank as described in the first aspect of the present invention is implemented.

[0167] In some embodiments, the processor can be implemented by software, hardware, firmware or a combination thereof, and can use at least one of a circuit, a single or multiple application-specific integrated circuits (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor, so that the processor can execute some or all of the steps or any combination of the steps in the real-time detection method of the slump of concrete in the mixer truck tank described in the various embodiments of the present application.

[0168] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concepts of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A real-time detection method for the slump of concrete in a mixer truck, characterized in that: The method comprises the following steps: Collect video stream data of concrete in the mixer tank and IMU attitude data of the tank, the video stream data includes multiple video frame images, and the IMU attitude data includes the roll angle and rotation speed of the tank; Detecting at least three positioning mark points preset on the inner wall of the mixer truck tank as reference corner points, the positioning mark points being spaced apart on the inner circumference of the tank mouth, constructing an affine transformation matrix based on the pixel displacement of the reference corner points between adjacent video frame images, and redirecting the video frame images according to the affine transformation matrix to obtain redirected video frame images; Delineating a region of interest on the concrete surface in the redirected video frame image, and extracting video features corresponding to the concrete from the region of interest, the video features including concrete surface ripple wavelength, aggregate settlement gradient, and flow motion vector; The video features are input into the trained neural network model, and the preliminary slump value of the concrete in the current mixer tank is output. The preliminary slump value is corrected using formula (1) to obtain the final slump value. The formula (1) is as follows: S final =S pred ×(1+a×rpm / rpm base )+b×sin(θ); Among them, S final is the final slump value, S pred is the initial slump value, rpm is the current tank speed, rpm base is the set reference speed, a and b are experimental calibration parameters, the value range of a is [0.02, 0.05], the value range of b is [2, 5], and θ is the roll angle; Output smoothed slump values ​​at a preset time period and mark the confidence level.

2. The method for real-time detection of concrete slump in a mixer truck according to claim 1, wherein: Delineating the region of interest on the concrete surface in the redirected video frame image comprises the following steps: Determining an initial reference area boundary size, and laterally adjusting the reference area boundary size according to the roll angle and the rotation speed to obtain a first area; Calculate the sinking velocity of the concrete based on the optical flow method, and adjust the longitudinal boundary of the first area according to the sinking velocity to obtain the second area; The Kalman filter is used to process the mutation position of the second area so that the range of the processed area is smaller than the boundary of the video frame image, thereby obtaining the region of interest.

3. The method for real-time detection of concrete slump in a mixer truck tank according to claim 1 or 2, wherein: The neural network model is a 3D-CNN network model. The video features are input into the trained neural network model, and the output of the preliminary slump value of the concrete in the current mixer tank includes: The 3D-CNN network model performs the following steps: The continuously extracted surface ripple wavelength, aggregate settlement gradient, and flow motion vector are aligned according to the time step to form a time series feature matrix with a dimension of T×3, where T is the number of video frames. Each feature dimension is then normalized to obtain a standardized time series feature matrix. The normalized temporal feature matrix is ​​passed through a bidirectional LSTM layer to extract temporal dependency features, and the key frames are weighted by an attention mechanism to output a first-dimensional feature vector. The region of interest is passed through an expanded convolution layer and a spatiotemporal GRU layer in sequence to output a second-dimensional feature vector. The first-dimensional feature vector and the second-dimensional feature vector are concatenated to obtain a fused feature vector, which is input into a fully connected network. After regularization processing with random neuron discarding, a preliminary slump prediction value is output.

4. The method for real-time detection of concrete slump in a mixer truck tank according to claim 1, wherein: The video frame image is redirected according to the affine transformation matrix, and the video frame image after redirection processing includes: Applying the affine transformation matrix to all pixel coordinates of the current frame, and performing coordinate mapping using bilinear interpolation; Adopt edge pixel replication strategy to fill the vacant area after the boundary mapping of the current frame image; The affine transformation matrix corresponding to the continuous multi-frame video frame images is subjected to sliding average filtering to smooth the motion compensation trajectory.

5. The method for real-time detection of concrete slump in a mixer truck tank according to claim 1, wherein: The concrete surface ripple wavelength is calculated according to the following method: Perform Fourier transform or wavelet transform on the video frame image sequence to extract the spatial wavelength corresponding to the main frequency component; Identifying corrugation peaks and valleys by edge detection or ridge tracking algorithms, calculating the average of the distances between adjacent peaks, and obtaining the wavelength of the corrugations on the concrete surface; The aggregate settlement gradient is calculated according to the following method: The HSV color space threshold segmentation algorithm was used to separate the aggregate and paste areas, and the noise was removed through morphological opening operation, retaining the outline of aggregate with a particle size greater than 2 mm. The vertical displacement field of aggregate during collapse is calculated based on the optical flow method, and the sinking velocity v of the center of mass of the aggregate is calculated according to the time series. z (t); Fitting the curve of aggregate sinking depth z(t) versus time t, and taking the maximum value of its first-order derivative as the aggregate settlement gradient; The flow motion vector is calculated according to the following method: Apply the dense optical flow algorithm to adjacent video frame images to obtain the motion vector (Δx, Δy) of each pixel in the current frame and the previous frame; Perform principal component analysis on the motion vector, take the first principal component direction as the dominant flow direction, and calculate the mean modulus of the vector in the dominant flow direction. and standard deviation σ v , normalized to obtain the flow motion vector, the calculation formula is as follows: ； in, is the angle between the dominant flow direction and the gravity direction, F flow is the flow motion vector.

6. The method for real-time detection of concrete slump in a mixer truck tank according to claim 1, wherein: Output smoothed slump values ​​at preset time intervals and mark confidence levels including: The slump value is output in a preset time period, and is smoothed using first-order lag filtering and median filtering to obtain the final slump value; The marking confidence levels include: A weighted calculation is performed based on the completeness of video feature extraction, the credibility of IMU posture data, and the variance of the slump value output by the neural network model to obtain the confidence level, and the data with a confidence level exceeding the preset level are screened out as valid data.

7. The method for real-time detection of concrete slump in a mixer truck tank according to claim 1, wherein: Before collecting the video stream data of the concrete in the mixer tank, the method further includes: The reference image of the calibration plate is captured by the visual sensor, and the clarity index Q of the reference image is calculated. clarity and stain coverage R dirt ; The clarity index Q clarity Calculated by Laplace variance method, the calculation formula is as follows: ； Where I is the grayscale image, is the Laplace operator; The stain coverage R dirt is the area ratio of the dirty area on the reference image; When the preset cleaning conditions are met, the cleaning alarm of the visual sensor is triggered and the subsequent slump value detection process is suspended. The preset cleaning conditions include the clarity index Q clarity Less than the preset clarity or stain coverage R dirt Greater than the preset coverage rate; When the preset cleaning condition is not met, the pixels in the dirty area are shielded during the subsequent extraction of video features, and only the pixels in the non-dirty area are selected for calculation.

8. The method for real-time detection of concrete slump in a mixer truck according to claim 7, wherein: When the preset cleaning conditions are met, the automatic film changing device is started to replace the lens film of the visual sensor, and the automatic film changing device includes: a feed reel for storing unused cleaning membranes and driven to rotate by a first servo motor electrically connected to the control board; The scrap reel is used to recycle the contaminated film. A constant tension of 0.5-1N is provided by a magnetic powder brake, which is connected to the tension sensor signal. A stripping blade is also provided on the film movement path between the scrap reel and the lens. A guide roller assembly, comprising at least two rollers supported by bearings, for maintaining the flatness of the film conveying path, the two rollers being symmetrically distributed between the feed reel and the waste reel; The diaphragm positioning mechanism includes an infrared radiation sensor and a main drive wheel. The infrared radiation sensor is installed at a preset distance in front of the lens and is used to detect the edge position of the diaphragm. The preset distance ranges from 3 to 5 mm. The main drive wheel is coaxially connected to the second servo motor and is used to control the stepping conveyance of the cleaning diaphragm according to the feedback signal of the infrared radiation sensor. The hot pressing bonding head includes an annular heating plate and a pressure sensor. The annular heating plate is maintained at a preset temperature range by a temperature control module, and the pressure sensor is used to provide real-time feedback of the bonding pressure to the control board. The ultrasonic cutting knife is driven up and down by a thin cylinder, and the distance between the cutting track and the edge of the lens is in the range of [1.8, 2.2] mm. The thin cylinder is connected to the air circuit of the solenoid valve; The control panel is used to issue a material shortage alarm when the amount of waste reel recovered reaches a preset proportional threshold of the initial length of the feed reel, or to terminate the film replacement process when the pressure sensor detects that the pressure abnormality lasts for a preset time.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the real-time detection method for the slump of concrete in a mixer truck tank is implemented.

10. An electronic device having a computer program stored thereon, characterized in that: The method comprises a processor and a storage medium, wherein a computer program is stored on the storage medium, and when the computer program is executed by the processor, the method for real-time detection of the slump of concrete in a mixer truck tank according to any one of claims 1 to 8 is implemented.

Images