Slump detection method, device, engineering machine and equipment
By combining a multi-dimensional detection model with video frame sequences and total power difference, the problem of existing concrete slump detection relying on human experience and sensor non-deterministic factors is solved, achieving high-precision online detection and abnormal state identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHANGDE SANY MACHINERY CO LTD
- Filing Date
- 2022-12-29
- Publication Date
- 2026-06-12
Smart Images
Figure CN116030386B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of building engineering data processing technology, and in particular to a slump detection method, device, engineering machinery and equipment. Background Technology
[0002] Concrete is a general term for engineering composite materials that bind aggregates together with cementing materials. It is widely used in civil engineering. An important parameter for judging whether concrete meets the needs of a construction site is the slump of the concrete.
[0003] Currently, there are several main methods for detecting slump: First, inspectors manually judge the slump by observing the unloading video and the main unit current, or by directly observing the flowability of the concrete in the mixing drum of the mixer truck on the inspection platform; second, slump is predicted using physical quantities measured by sensors. However, manual slump judgment requires a high level of experience from inspectors, and the accuracy of slump detection is susceptible to subjective influence. Furthermore, sensors typically measure the relationship between concrete viscous resistance and slump, pressure and slump, or current value and slump. These three factors can only be considered as influencing factors, not determining factors, leading to lower accuracy in slump judgment. Summary of the Invention
[0004] This invention provides a slump detection method, apparatus, engineering machinery and equipment, aiming to improve the accuracy of concrete slump detection.
[0005] This invention provides a slump detection method, comprising:
[0006] The total power difference of the mixing shaft drive motor in the mixing host is obtained, and the video frame sequence of the concrete to be tested in the unloading area is collected.
[0007] The unloading video frame sequence is input into the first slump detection model to obtain the first slump detection result output by the first slump detection model;
[0008] The total power difference is input into the second slump detection model to obtain the second slump detection result output by the second slump detection model;
[0009] Based on the first slump test result and the second slump test result, the first target slump test result of the concrete to be tested is obtained by fusion calculation;
[0010] The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples.
[0011] The second slump detection model is obtained by probability distribution fitting of the total work difference characteristics corresponding to different slumps under different concrete strengths.
[0012] According to the slump detection method provided by the present invention, the first slump detection model is trained based on the following steps:
[0013] Acquire several sets of video frame sequence samples during the unloading of the mixing host;
[0014] For any set of video frame sequence samples, the video frame sequence samples are input into the initial detection model to obtain the detection result output by the initial detection model;
[0015] Based on the detection results and the sample labels corresponding to the video frame sequence samples, the model loss value is calculated.
[0016] Based on the model loss value obtained in each iteration, the model parameters of the initial detection model are updated to obtain the first slump detection model.
[0017] According to the slump detection method provided by the present invention, the first slump detection model includes a feature extraction module and a detection output module;
[0018] The step of inputting the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model includes:
[0019] The unloading video frame sequence is input into the feature extraction module to obtain the temporal spatial feature information output by the feature extraction module;
[0020] The temporal spatial feature information is input into the detection output module to obtain the first slump detection result output by the detection output module.
[0021] According to the slump detection method provided by the present invention, the second slump detection model is trained based on the following steps:
[0022] For any concrete strength, obtain the total work difference features to be fitted for different slumps under the concrete strength;
[0023] The total work difference features to be fitted are dimensionless to obtain the dimensionless feature data set with different collapse degrees.
[0024] By fitting the dimensionless feature data set corresponding to any slump under any concrete strength to a probability distribution, a second slump detection model corresponding to any slump under any concrete strength is obtained.
[0025] According to a slump detection method provided by the present invention, the step of calculating a first target slump detection result of the concrete to be tested based on the first slump detection result and the second slump detection result includes:
[0026] Based on the first slump detection result, the second slump detection result, the first fusion weight corresponding to the first slump detection model, and the second fusion weight corresponding to the second slump detection model, the first target slump detection result is obtained by fusion calculation;
[0027] The first fusion weight and the second fusion weight are obtained by training the weight parameters between the first collapse detection model and the second collapse detection model based on the historical total power difference corresponding to the mixing host, several sets of video frame sequences to be trained, and the true collapse labels corresponding to each of the video frame sequences to be trained.
[0028] According to the slump detection method provided by the present invention, before acquiring the unloading video frame sequence corresponding to the unloading area of the concrete to be tested, the method further includes:
[0029] Acquire the multimodal characteristic data of the concrete to be tested in the mixing host during the mixing process;
[0030] The multimodal feature data is input into the third slump detection model to obtain the third slump detection result output by the third slump detection model;
[0031] Based on the third slump detection result and the second slump detection result, the second target slump detection result is obtained by fusion calculation;
[0032] Based on the second target slump detection result, determine whether the slump of the concrete to be tested is in an abnormal state during the mixing process;
[0033] If not, return to the step of collecting the unloading video frame sequence corresponding to the concrete to be tested in the unloading area;
[0034] If so, slump adjustment information is generated to make adjustments based on the slump adjustment information.
[0035] According to the slump detection method provided by the present invention, the third slump detection model is trained based on the following steps:
[0036] Acquire training multidimensional feature data for several types of concrete, wherein the training multidimensional feature data includes at least concrete material mix proportion data and humidity data;
[0037] Based on the multidimensional feature data to be trained and the collapse labels corresponding to the multidimensional feature data to be trained, the detection model to be trained is iteratively trained to obtain the third collapse detection model.
[0038] The present invention also provides a slump detection device, comprising:
[0039] The acquisition module is used to acquire the total power difference of the mixing shaft drive motor in the mixing host, and to collect the unloading video frame sequence corresponding to the concrete to be tested;
[0040] The first detection module is used to input the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model.
[0041] The second detection module is used to input the total power difference into the second slump detection model to obtain the second slump detection result output by the second slump detection model;
[0042] The calculation module is used to calculate the first target slump test result of the concrete to be tested by fusing the first slump test result and the second slump test result.
[0043] The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples.
[0044] The second slump detection model is obtained by probability distribution fitting based on the total work difference characteristics corresponding to different slumps under various concrete strengths.
[0045] The present invention also provides an engineering machine, including a power acquisition device, a camera, and a server, wherein:
[0046] A power acquisition device, installed on the mixing host, is used to collect current and voltage data of the mixing shaft drive motor in the mixing host. The current and voltage data are used to determine the total power difference.
[0047] The camera is used to capture the unloading video frame sequence corresponding to the unloading area;
[0048] The server includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the program, it implements the slump detection method.
[0049] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the slump detection method as described above.
[0050] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the slump detection method as described above.
[0051] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the slump detection method as described above.
[0052] The slump detection method, apparatus, engineering machinery, and equipment provided by this invention utilize a first slump detection model to perform image recognition on a video frame sequence, thereby extracting the slump flowability and surface detail features of concrete. Furthermore, by combining a second slump detection model to estimate the slump probability based on the total work difference, the slump value of each batch of concrete can be detected online from multiple dimensions during the unloading process, based on the unloading video frame sequence and the total work difference, effectively improving the accuracy and precision of slump detection. Attached Figure Description
[0053] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.
[0054] Figure 1 This is one of the flowcharts of the slump detection method provided by the present invention;
[0055] Figure 2 This is a schematic diagram of one possible structure of the unloading process of the mixing host provided by the present invention;
[0056] Figure 3 This is the second flowchart of the slump detection method provided by the present invention;
[0057] Figure 4 This is the third flowchart of the slump detection method provided by the present invention;
[0058] Figure 5 This is the fourth flowchart of the slump detection method provided by the present invention;
[0059] Figure 6 This is the fifth flowchart of the slump detection method provided by the present invention;
[0060] Figure 7 This is a system flowchart of the slump detection system provided by the present invention;
[0061] Figure 8This is a schematic diagram of the slump detection device provided by the present invention;
[0062] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention.
[0063] Explanation of reference numerals in the attached diagram: 1: Throttle valve; 2: Limit switch; 3: Camera; 4: Unloading hopper; 5: Cylinder. Detailed Implementation
[0064] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.
[0065] The terminology used in one or more embodiments of the present invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The singular forms “a,” “the,” and “the” used in one or more embodiments of the invention are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” used in one or more embodiments of the invention refers to and includes any or all possible combinations of one or more associated listed items.
[0066] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of the present invention, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of the present invention, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "when".
[0067] Figure 1 This is one of the flowcharts illustrating the slump detection method provided by this invention. For example... Figure 1 As shown, the slump detection method includes:
[0068] Step 11: Obtain the total power difference of the mixing shaft drive motor in the mixing host, and collect the unloading video frame sequence corresponding to the concrete to be tested in the unloading area.
[0069] It should be noted that the mixed concrete is fed into the mixer truck through the discharge port of the mixing host and the receiving hopper of the mixer truck. This process is called the unloading process. The unloading video frame sequence in the embodiment of the present invention is the image sequence corresponding to the unloading process. The unloading video frame sequence can be a time-continuous video frame in the unloading video, or it can be a number of video frames extracted from the unloading video at a preset time interval. The preset time interval can be set according to the actual situation. Preferably, in order to effectively monitor the fluidity of concrete during the unloading process, the preset time interval can be set to be small, for example, 0.2 seconds.
[0070] Furthermore, such as Figure 2 As shown, Figure 2 This is a schematic diagram of one possible structure of the unloading process of the mixing host provided by the present invention. The unloading video frame sequence can be obtained by capturing images from a pre-set camera, and the camera's installation position can be set according to actual conditions. In one embodiment, when the mixing host's unloading port directly unloads concrete into the mixer truck's receiving hopper, the camera can capture the feeding video frames of the mixer truck's receiving hopper. In another embodiment, the mixing host first unloads concrete into the waiting hopper of the throttle valve at the mixing host's unloading port. In this case, the camera can capture the feeding video of the waiting hopper of the throttle valve. The unloading video frame sequence can then be extracted from the feeding video.
[0071] Furthermore, the total power difference represents the total power difference per unit time between the drive motors of all stirring shafts in the mixing host during the mixing process and when unloaded. For example, a twin-shaft mixing host includes two drive motors, and the total power difference represents the total power difference between the two drive motors during the mixing process and when unloaded. The total power difference includes information such as the work done to overcome concrete resistance and the work done to overcome gravity.
[0072] Specifically, the process involves acquiring the current and voltage data of all drive motors of the mixing shaft in the mixing host when they are unloaded, and also acquiring the current and voltage data of all drive motors during the mixing process. Additionally, the material mix proportions of the concrete need to be determined, whereby the material mix proportions represent the ratios of raw materials added to the concrete during the mixing process. Furthermore, the total power difference is calculated based on the current and voltage data during unload and mixing. The current and voltage data can be measured using a power acquisition device or similar measuring equipment. Additionally, the process involves acquiring the unloading video frame sequence of the concrete in the unloading area, captured by a camera. It should be noted that the execution order of acquiring the unloading video frame sequence and calculating the total power difference is not specifically limited here; preferably, the unloading video frame sequence captured by the camera can be acquired first, followed by the calculation of the total power difference.
[0073] Step 12: Input the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model;
[0074] The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples.
[0075] Specifically, the video frame sequence is input into the first slump detection model to extract concrete morphology features such as concrete fluidity and surface features, thereby obtaining the first slump detection result output by the first slump detection model. The first slump detection result includes multiple probability values corresponding to slump.
[0076] It should be noted that the first slump detection model is obtained through iterative training based on video frame sequence samples carrying sample labels. Understandably, in order to effectively learn the temporal and spatial characteristics of concrete, this embodiment of the invention uses a deep learning temporal spatial modeling algorithm (Video Classification), and then iteratively trains the first slump detection model based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples. Thus, based on the video frame sequence of the unloading video, the first slump detection model can extract the flowability characteristics and surface characteristics of concrete, effectively improving the accuracy of the first slump detection model in online slump detection.
[0077] Step 13: Input the total power difference into the second slump detection model to obtain the second slump detection result output by the second slump detection model;
[0078] The second slump detection model is obtained by fitting the probability distribution of the total work difference characteristics corresponding to different slumps under various concrete strengths.
[0079] Specifically, the strength of the concrete to be tested in the mixing host is determined. Further, the total work difference is processed into a dimensionless form, and the dimensionless feature of the total work difference is input into each second slump detection model corresponding to the strength to obtain the probability of the second slump detection result output by each second slump detection model.
[0080] Understandably, firstly, for each concrete strength, the total power difference features corresponding to different slumps at that strength are pre-collected. These features represent the total power difference of all drive motors of the mixing shafts in the mixing host. Based on these features, a set of dimensionless feature data corresponding to each feature is calculated. Then, for any slump at any concrete strength, the set of dimensionless feature data corresponding to that slump is fitted and modeled to obtain a second slump detection model for that concrete strength and slump. Understandably, the same concrete strength may contain multiple second slump detection models corresponding to different slumps. Therefore, when determining the strength of the concrete to be tested, the dimensionless features corresponding to the total power difference are input into the multiple second slump detection models corresponding to different slumps at that strength, obtaining the second slump detection results output by each second slump detection model.
[0081] Step 14: Based on the first slump test result and the second slump test result, the first target slump test result of the concrete to be tested is obtained by fusion calculation.
[0082] Specifically, in one feasible implementation, the model weights corresponding to the first slump detection model and the second slump detection model are pre-set, and the first target slump detection result of the concrete to be tested is obtained by fusion calculation based on the first slump detection result, the second slump detection result and the corresponding weights.
[0083] In another embodiment, to improve the accuracy of the target slump detection result, after training the first slump detection model and the second slump detection model, additional verification data samples are obtained. These verification data samples include historical total power differences, several sets of video frame sequences to be trained corresponding to unloading areas, and real slump labels corresponding to each set of video frame sequences to be trained. Based on these historical total power differences, several sets of video frame sequences to be trained, and real slump labels corresponding to each set of video frame sequences to be trained, the weight parameters between the first slump detection model and the second slump detection model are fused and trained to obtain a first fusion weight corresponding to the first slump detection model and a second fusion weight corresponding to the second slump detection model. Then, based on the first slump detection result, the second slump detection result, the first fusion weight corresponding to the first slump detection model, and the second fusion weight corresponding to the second slump detection model, the first target slump detection result is calculated.
[0084] This invention utilizes a first slump detection model to perform image recognition on a video frame sequence, enabling the extraction of concrete slump flowability and surface detail features. Furthermore, it combines a second slump detection model to perform slump detection on the total work difference. This allows for online detection of the slump value of each batch of concrete from multiple dimensions during the unloading process, based on the unloading video frame sequence and the total work difference, effectively improving the accuracy and precision of slump detection.
[0085] Figure 3 This is the second flowchart of the slump detection method provided by the present invention, as shown below. Figure 3 As shown, in one embodiment of the present invention, the first slump detection model is trained based on the following steps:
[0086] Step 31: Obtain several sets of video frame sequence samples during the unloading of the mixing host;
[0087] Step 32: For any set of video frame sequence samples, input the video frame sequence samples into the initial detection model to obtain the detection result output by the initial detection model;
[0088] Step 33: Calculate the model loss value based on the detection results and the sample labels corresponding to the video frame sequence samples;
[0089] Step 34: Based on the model loss value obtained in each iteration, update the model parameters of the initial detection model to obtain the first slump detection model.
[0090] Specifically, several sets of video frame sequence samples of the mixing host during unloading are collected in advance. Each set of video frame sequence samples is configured with its own corresponding sample label. Then, for each set of video frame sequence samples, the following steps are performed: the video frame sequence samples are input into an initial detection model to determine the detection result based on the output of the initial detection model. Then, based on the detection result and the sample label corresponding to the video frame sequence sample, a model loss value is calculated using a preset loss function, including L1 loss function and Focal Loss function, etc. Further, after calculating the model loss value, the model parameters in the initial detection model are updated using the error backpropagation algorithm. This training process ends, and then the next training is performed. During the training process, it is determined whether the updated initial detection model meets the preset training termination condition. If it does, the updated initial detection model is used as the first collapse detection model. If it does not meet the condition, the model training continues. The preset training termination condition includes loss convergence and reaching the maximum number of iterations threshold, etc.
[0091] This invention trains a first slump detection model based on video frame sequence samples, which helps to control the loss value of the first slump detection model within a preset range. Thus, the trained first slump detection model can be used to identify the flowability and surface details of concrete during the unloading process, effectively improving the accuracy of slump detection.
[0092] Figure 4 This is the third flowchart of the slump detection method provided by the present invention, as shown below. Figure 4 As shown, in one embodiment of the present invention, the second slump detection model is trained based on the following steps:
[0093] Step 41: For any concrete strength, obtain the total work difference features to be fitted for different slumps under the concrete strength.
[0094] Step 42: Perform dimensionless processing on each of the total work difference features to be fitted to obtain a set of dimensionless feature data for different collapse degrees;
[0095] Step 43: Perform probability distribution fitting on the dimensionless feature data set corresponding to any slump under any concrete strength to obtain the second slump detection model corresponding to any slump under any concrete strength.
[0096] Specifically, for any concrete strength, the following steps are performed: First, the total power difference features to be fitted corresponding to different slumps under the concrete strength are collected in advance. The total power difference features to be fitted represent the total power difference of all the driving motors of the mixing shaft in the mixing host. Then, the total power difference features to be fitted are dimensionless to obtain the dimensionless feature data set of different slumps. Further, in the modeling process, for each dimensionless feature data set corresponding to any slump under any concrete strength, the probability distribution of each dimensionless feature data set corresponding to any slump under any concrete strength is fitted to obtain the second slump detection model corresponding to any slump under any concrete strength.
[0097] Understandably, assuming the concrete strength is C25, and the slump includes 160mm, 170mm, 180mm, and 190mm, for C25 concrete strength, several total work difference features to be fitted are collected for each slump of 160mm, 170mm, 180mm, and 190mm. Then, for each total work difference feature to be fitted for each slump, a dimensionless feature data set for different slumps is calculated. For example, for a 160mm slump, the dimensionless feature dataset corresponding to the 160mm slump is fitted with a probability distribution to obtain the second slump detection model corresponding to the 160mm slump under C25 strength. Thus, the second slump detection models corresponding to the four slumps under C25 strength are constructed.
[0098] This invention constructs second slump detection models corresponding to various slump values under different concrete strengths. Thus, when the strength of the concrete to be tested is determined, the second slump detection models under that strength can be used to perform the test based on the total work difference of the concrete to be tested, effectively improving the accuracy of slump detection.
[0099] In one embodiment of the present invention, the first slump detection model includes a feature extraction module and a detection output module; the unloading video frame sequence is input into the first slump detection model to obtain a first slump detection result output by the first slump detection model, including:
[0100] The unloading video frame sequence is input to the feature extraction module to obtain the temporal and spatial feature information output by the feature extraction module; the temporal and spatial feature information is input to the detection output module to obtain the first slump detection result output by the detection output module.
[0101] It should be noted that the first slump detection model includes a feature extraction module and a detection output module. Specifically, the video frame sequence is input to the feature extraction module to extract features such as the flowability and surface detail dimensions of the concrete during unloading, corresponding to the video frame sequence, to obtain the spatiotemporal feature information. This spatiotemporal feature information is then input to the detection output module to obtain the probability values corresponding to each slump, and these probability values are used as the first slump detection result. For example, the probability value for a slump of 160mm is 30%, for 170mm it is 50%, and for 180mm it is 20%.
[0102] According to the embodiments of the present invention, based on the video frame sequence, the first slump detection model can identify dimensional feature information such as the fluidity and surface details of concrete during the unloading process, which greatly improves the accuracy of slump detection.
[0103] In one embodiment of the present invention, the step of calculating the first target slump test result of the concrete to be tested based on the first slump test result and the second slump test result includes:
[0104] Based on the first slump detection result, the second slump detection result, the first fusion weight corresponding to the first slump detection model, and the second fusion weight corresponding to the second slump detection model, the first target slump detection result is obtained by fusion calculation; wherein, the first fusion weight and the second fusion weight are obtained by training the weight parameters between the first slump detection model and the second slump detection model based on the historical total power difference corresponding to the mixing host, several sets of video frame sequences to be trained, and the true slump labels corresponding to each of the video frame sequences to be trained.
[0105] Specifically, after the training of the second slump detection model and the first slump detection model is completed, the historical total power difference corresponding to the mixing host, several sets of video frame sequences to be trained, and the slump real labels corresponding to each of the video frame sequences to be trained are collected to jointly train the fusion weight parameters between the first slump detection model and the second slump detection model to obtain the first fusion weight and the second fusion weight. More specifically: for any historical total power difference, and each video frame sequence to be trained when the mixing host is unloading at the corresponding time and space, the video frame sequence to be trained is input into the first slump detection model to obtain... The first collapse detection result output by the first collapse detection model and the historical total power difference input to the second collapse detection model are used to obtain the second collapse detection result output by the second collapse detection model. Further, based on the true collapse labels corresponding to the video frame sequence to be trained, the first collapse detection result and the second collapse detection result are fused and trained using a preset estimation algorithm. The preset estimation algorithm includes maximum likelihood estimation, least squares estimation, and Bayesian estimation, etc. Preferably, maximum likelihood estimation is selected to obtain the first fusion weight and the second fusion weight. Therefore, based on the first collapse detection result, the second collapse detection result, the first fusion weight, and the second fusion weight, the first target collapse detection result is calculated.
[0106] Understandably, the detection results of the first slump detection model include a probability value of 30% for a slump of 160mm, a probability value of 50% for a slump of 170mm, and a probability value of 20% for a slump of 180mm. The detection results of the second slump detection model include a probability value of 50% for a slump of 160mm, a probability value of 20% for a slump of 170mm, and a probability value of 30% for a slump of 180mm. Assuming the first fusion weight is 0.7 and the second fusion weight is 0.3, the final probability of a slump of 160mm is 30% × 0.7 + 50% × 0.3 = 0.36. The calculation process for slumps of 170mm and 180mm is basically the same as the calculation process for the final probability of a slump of 160mm. Furthermore, the slump with the highest final probability is selected as the first target slump detection result.
[0107] This invention calculates the first target slump detection result based on the first slump detection result and the second slump detection result, combined with the first fusion weight and the second fusion weight obtained from fusion training. This enables online detection of the slump value of each batch of concrete from multiple dimensions, thereby improving the accuracy and precision of slump detection.
[0108] Figure 5 This is the fourth flowchart of the slump detection method provided by the present invention, as shown below. Figure 5 As shown, in one embodiment of the present invention, before acquiring the unloading video frame sequence corresponding to the unloading area of the concrete to be tested, the method further includes:
[0109] Step 51: Obtain multimodal feature data of the concrete to be tested in the mixing host during the mixing process;
[0110] Step 52: Input the multimodal feature data into the third slump detection model to obtain the third slump detection result output by the third slump detection model;
[0111] Step 53: Based on the third slump detection result and the second slump detection result, the second target slump detection result is obtained by fusion calculation;
[0112] Step 54: Based on the second target slump detection result, determine whether the slump of the concrete to be tested is in an abnormal state during the mixing process;
[0113] Step 55: If not, return to the step of collecting the unloading video frame sequence corresponding to the concrete to be tested in the unloading area;
[0114] Step 56: If yes, generate slump adjustment information to make adjustments based on the slump adjustment information.
[0115] It should be noted that the multimodal characteristic data includes the material mix proportion data and humidity data of the concrete to be tested. The humidity data can be obtained by using a humidity sensor. In addition, the multimodal characteristic data may also include characteristic data such as the density, apparent density, water reduction rate of admixtures, crushing value, gradation, sand ratio and bulk density of raw materials.
[0116] Specifically, when the mixing host starts mixing, the production control system sends a mixing start signal to the server. The production control system is a control system used to monitor the mixing, pausing, and unloading operations of the mixing host. The server stores algorithm programs such as the first slump detection model algorithm, the second slump detection result, and the third slump detection result. Then, the server collects multimodal feature data of the concrete to be tested during the mixing process, and inputs the multimodal feature data into the third slump detection model. Based on the output of the third slump detection model, the probability prediction value corresponding to each slump is determined, and the probability prediction value corresponding to each slump is used as the third slump detection result. Further, based on the third slump detection result, the second slump detection result output by the second slump detection model, the third fusion weight corresponding to the second slump detection model, and the fourth fusion weight corresponding to the third slump detection model, a second target slump detection result is obtained by fusion calculation. The calculation process of the second target slump detection result is basically the same as that of the first target slump detection result, and will not be described again here. It should also be noted that the third and fourth fusion weights are obtained by training the weight parameters between the second and third collapse detection models based on the total historical power difference, the multi-dimensional feature data, and the true collapse labels corresponding to the multi-dimensional feature data. The fusion training process of the third and fourth fusion weights is basically the same as that of the first and second fusion weights, and will not be described in detail here.
[0117] Furthermore, based on the second target slump detection result and the preset slump threshold, it is determined whether the slump of the concrete under test is in an abnormal state during the mixing process. The preset slump threshold is set according to the actual concrete strength and is not specifically limited here. If it is in an abnormal state, the discharge from the mixing host's unloading port is paused, and slump adjustment information is generated. This slump adjustment information can be information on adjusting humidity data, or information on adjusting the water reduction rate of admixtures, crushing value, etc. In addition, when the slump is in an abnormal state, a warning message can be generated to prompt the operator to make abnormal adjustments, such as adjusting according to the current material ratio. If not, it is determined that the slump during the mixing process is normal, allowing the mixing host to discharge. Then, a discharge signal is sent to the preset server through the production control system, thus returning to the step of collecting the discharge video frame sequence of the concrete under test in the discharge area to execute the subsequent online slump detection process during discharge.
[0118] This invention collects multimodal characteristic data such as material ratio data and humidity data corresponding to the mixing host during the mixing process, and then uses a third slump detection model to obtain a third slump detection result. In addition, it combines the second slump detection result of the second slump detection model to calculate a second target slump detection result, which effectively improves the accuracy of slump anomaly detection during the mixing process.
[0119] Figure 6 This is the fifth flowchart of the slump detection method provided by the present invention, as shown below. Figure 6 As shown, in one embodiment of the present invention, the third slump detection model is trained based on the following steps:
[0120] Step 61: Obtain training multidimensional feature data for several types of concrete. The training multidimensional feature data includes at least the material mix ratio data and humidity data of the concrete.
[0121] Step 62: Based on the multidimensional feature data to be trained and the collapse label corresponding to each multidimensional feature data to be trained, the detection model to be trained is iteratively trained to obtain the third collapse detection model.
[0122] It should be noted that the detection model to be trained can be a model such as SVM, logistic regression, decision boosting tree, and random forest.
[0123] Specifically, several types of concrete corresponding to training multidimensional feature data are collected. The training multidimensional feature data includes at least the material proportion data and humidity data of the concrete, and may also include feature data such as the density of raw materials, apparent density, water reduction rate of admixtures, crushing value, gradation, sand ratio and bulk density. Then, during the model iteration process, for any training multidimensional feature data, the training multidimensional feature data is input into the training detection model to obtain the predicted value output by the training detection model. Further, based on the slump label corresponding to the training multidimensional feature data and the predicted value, the loss value is calculated. Thus, based on the loss value calculated in each iteration, the model parameters of the training detection model are updated to obtain the third slump detection model.
[0124] This invention provides an embodiment of the invention that trains a third collapse detection model based on multidimensional feature data to be trained. This helps to control the loss value of the third collapse detection model within a preset range and effectively improves the accuracy of the third collapse detection model in collapse detection.
[0125] like Figure 7 Show, Figure 7 This is a system flowchart of the slump detection method provided by the present invention, wherein weight a1 represents the first fusion weight, weight a2 represents the second fusion weight, weight a3 represents the third fusion weight, and weight a4 represents the fourth fusion weight.
[0126] More specifically, regarding the mixing stage: the lower-level machine of the production control system sends a mixing start signal to a preset server. When the server receives the mixing start signal, it collects multi-modal characteristic data corresponding to the mixing process, as well as current and voltage data of the mixing shaft drive motor collected by the power acquisition device. The multi-modal characteristic data includes at least the concrete material ratio data and humidity data collected by the humidity sensor. The multi-modal characteristic data is input into the third slump detection model to obtain the third slump detection result output by the third slump detection model. In addition, based on the current data and the material ratio data in the multi-modal characteristic data, a calculation is performed... The total power difference corresponding to all the drive motors of the mixing shaft in the mixing host is input into the second slump detection model to obtain the second slump detection result output by the second slump detection model. Then, based on the third slump detection result, the second slump detection result, the third fusion weight, and the fourth fusion weight, the second target slump detection result is calculated. Further, based on the second target slump detection result, it is determined whether the slump of the concrete to be tested is in an abnormal state. If so, a warning message is issued. If not, a discharge signal is sent to the preset server through the production control system, thereby entering the slump detection process of the discharge process.
[0127] For the unloading stage: When the server receives the unloading signal, the server collects the video frame sequence corresponding to the concrete to be tested in the unloading area, and then inputs the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model. It should be noted that the second slump detection result used to determine the first target slump detection result and the second target slump detection result are the same. The first target slump detection result can be calculated by directly combining the second slump detection result output by the second slump detection model, the first slump detection result, the first fusion weight, and the second fusion weight.
[0128] Optionally, in other embodiments, the first target slump detection result can also be calculated based on the first slump detection result output by the first slump detection model, the second slump detection result output by the second slump detection model, the third slump detection result output by the third slump detection model, and the weights corresponding to each model. The weights corresponding to each model are obtained by fusing and training the weight parameters between the first slump detection model, the second slump detection model, and the third slump detection model based on multi-dimensional feature data such as historical total work difference, several sets of video frame sequences to be trained, material ratio data, and humidity data, as well as the true slump labels of each model.
[0129] The slump detection device provided by the present invention is described below. The slump detection device described below can be referred to in correspondence with the slump detection method described above.
[0130] Figure 8 This is a schematic diagram of the slump detection device provided by the present invention, as shown below. Figure 8 As shown, an embodiment of the present invention provides a slump detection device, which includes:
[0131] The acquisition module 81 is used to acquire the total power difference of the mixing shaft drive motor in the mixing host, and to collect the unloading video frame sequence corresponding to the concrete to be tested in the unloading area.
[0132] The first detection module 82 is used to input the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model.
[0133] The second detection module 83 is used to input the total power difference into the second slump detection model to obtain the second slump detection result output by the second slump detection model;
[0134] Calculation module 84 is used to calculate the first target slump test result of the concrete to be tested by fusing the first slump test result and the second slump test result.
[0135] The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples.
[0136] The second slump detection model is obtained by probability distribution fitting based on the total work difference characteristics corresponding to different slumps under various concrete strengths.
[0137] The slump detection device also includes:
[0138] Acquire several sets of video frame sequence samples during the unloading of the mixing host;
[0139] For any set of video frame sequence samples, the video frame sequence samples are input into the initial detection model to obtain the detection result output by the initial detection model;
[0140] Based on the detection results and the sample labels corresponding to the video frame sequence samples, the model loss value is calculated.
[0141] Based on the model loss value obtained in each iteration, the model parameters of the initial detection model are updated to obtain the first slump detection model.
[0142] The slump detection device also includes:
[0143] For any concrete strength, obtain the total work difference features to be fitted for different slumps under the concrete strength;
[0144] The total work difference features to be fitted are dimensionless to obtain the dimensionless feature data set with different collapse degrees.
[0145] By fitting the probability distribution of each dimensionless feature data set corresponding to any slump under any concrete strength, a second slump detection model corresponding to any slump under any concrete strength is obtained.
[0146] The first detection module 82 is also used for:
[0147] The first slump detection model includes a feature extraction module and a detection output module;
[0148] The unloading video frame sequence is input into the feature extraction module to obtain the temporal spatial feature information output by the feature extraction module;
[0149] The spatiotemporal feature information is input into the detection output module to obtain the first slump detection result output by the detection output module.
[0150] The computing module 84 is also used for:
[0151] Based on the first slump detection result, the second slump detection result, the first fusion weight corresponding to the first slump detection model, and the second fusion weight corresponding to the second slump detection model, the first target slump detection result is obtained by fusion calculation;
[0152] The first fusion weight and the second fusion weight are obtained by training the weight parameters between the first collapse detection model and the second collapse detection model based on the historical total power difference corresponding to the mixing host, several sets of video frame sequences to be trained, and the true collapse labels corresponding to each of the video frame sequences to be trained.
[0153] The slump detection device also includes:
[0154] Acquire the multimodal characteristic data of the concrete to be tested in the mixing host during the mixing process;
[0155] The multimodal feature data is input into the third slump detection model to obtain the third slump detection result output by the third slump detection model;
[0156] Based on the third slump detection result and the second slump detection result, the second target slump detection result is obtained by fusion calculation;
[0157] Based on the second target slump detection result, determine whether the slump of the concrete to be tested is in an abnormal state during the mixing process;
[0158] If not, return to the step of collecting the unloading video frame sequence corresponding to the concrete to be tested in the unloading area;
[0159] If so, slump adjustment information is generated to make adjustments based on the slump adjustment information.
[0160] The slump detection device also includes:
[0161] Acquire training multidimensional feature data for several types of concrete, wherein the training multidimensional feature data includes at least concrete material mix proportion data and humidity data;
[0162] Based on the multidimensional feature data to be trained and the collapse labels corresponding to the multidimensional feature data to be trained, the detection model to be trained is iteratively trained to obtain the third collapse detection model.
[0163] It should be noted that the apparatus provided in this embodiment of the invention can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Therefore, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail here.
[0164] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9 As shown, the electronic device may include: a processor 910, a memory 920, a communication interface 930, and a communication bus 940, wherein the processor 910, the memory 920, and the communication interface 930 communicate with each other through the communication bus 940. The processor 910 can call logic instructions in the memory 920 to execute a slump detection method. This method includes: acquiring the total power difference of the mixing shaft drive motor in the mixing host, and collecting a sequence of unloading video frames corresponding to the unloading area of the concrete to be tested; inputting the unloading video frame sequence into a first slump detection model to obtain a first slump detection result output by the first slump detection model; inputting the total power difference into a second slump detection model to obtain a second slump detection result output by the second slump detection model; and calculating a first target slump detection result for the concrete to be tested based on the first slump detection result and the second slump detection result. The first slump detection model is obtained through iterative training based on video frame sequence samples and the corresponding sample labels; the second slump detection model is obtained by probability distribution fitting based on the total power difference characteristics corresponding to different slumps under various concrete strengths.
[0165] Furthermore, the logical instructions in the aforementioned memory 920 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0166] In another aspect, an engineering machine according to an embodiment of the present invention includes a power acquisition device, a camera, and a server, wherein: the power acquisition device is installed on the mixing host and is used to collect current data and voltage data of the mixing shaft drive motor in the mixing host, the current data and the voltage data being used to determine the total power difference; the camera is used to collect a sequence of unloading video frames corresponding to the unloading area; the server includes a memory, a processor, and a computer program stored in the memory and running on the processor, the processor implementing the slump detection method when executing the program.
[0167] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the slump detection method provided by the above methods. The method includes: acquiring the total power difference of the mixing shaft drive motor in the mixing host, and acquiring a sequence of unloading video frames corresponding to the unloading area of the concrete to be tested; inputting the unloading video frame sequence into a first slump detection model to obtain a first slump detection result output by the first slump detection model; inputting the total power difference into a second slump detection model to obtain a second slump detection result output by the second slump detection model; and calculating a first target slump detection result of the concrete to be tested based on the first slump detection result and the second slump detection result. The first slump detection model is obtained through iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples; the second slump detection model is obtained by probability distribution fitting based on the total power difference characteristics corresponding to different slumps under various concrete strengths.
[0168] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the slump detection method provided by the above methods. The method includes: obtaining the total power difference of the mixing shaft drive motor in the mixing host, and acquiring the unloading video frame sequence corresponding to the unloading area of the concrete to be tested; inputting the unloading video frame sequence into a first slump detection model to obtain a first slump detection result output by the first slump detection model; inputting the total power difference into a second slump detection model to obtain a second slump detection result output by the second slump detection model; and fusion calculation based on the first slump detection result and the second slump detection result to obtain a first target slump detection result of the concrete to be tested; wherein, the first slump detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples; and the second slump detection model is obtained by probability distribution fitting based on the total power difference characteristics corresponding to different slumps under various concrete strengths.
[0169] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.
[0170] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.
[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A slump detection method, characterized in that, include: The total power difference of the mixing shaft drive motor in the mixing host is obtained, and the video frame sequence of the concrete to be tested in the unloading area is collected. The acquisition of the unloading video frame sequence corresponding to the concrete to be tested in the unloading area includes: capturing the feeding video of the unloading area through a camera, wherein the unloading area is the receiving hopper of the mixer truck or the waiting hopper of the throttle valve; extracting video frames with continuous time from the feeding video or extracting video frames according to a preset time interval to obtain the unloading video frame sequence. The unloading video frame sequence is input into a first slump detection model to obtain a first slump detection result output by the first slump detection model; the total power difference is input into a second slump detection model to obtain a second slump detection result output by the second slump detection model; the second slump detection model is trained based on the following steps: for any concrete strength, obtain each total power difference feature to be fitted corresponding to different slumps under the concrete strength; perform dimensionless processing on each of the total power difference features to be fitted to obtain a dimensionless feature data set for different slumps; perform probability distribution fitting on the dimensionless feature data set corresponding to any slump under any concrete strength to obtain a second slump detection model corresponding to any slump under any concrete strength. Based on the first slump test result and the second slump test result, the first target slump test result of the concrete to be tested is obtained by fusion calculation; The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples. The second slump detection model is obtained by probability distribution fitting of the total work difference characteristics corresponding to different slumps under different concrete strengths.
2. The slump detection method according to claim 1, characterized in that, The first slump detection model was trained based on the following steps: Acquire several sets of video frame sequence samples during the unloading of the mixing host; For any set of video frame sequence samples, the video frame sequence samples are input into the initial detection model to obtain the detection result output by the initial detection model; Based on the detection results and the sample labels corresponding to the video frame sequence samples, the model loss value is calculated. Based on the model loss value obtained in each iteration, the model parameters of the initial detection model are updated to obtain the first slump detection model.
3. The slump detection method according to claim 1, characterized in that, The first slump detection model includes a feature extraction module and a detection output module; The step of inputting the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model includes: The unloading video frame sequence is input into the feature extraction module to obtain the temporal spatial feature information output by the feature extraction module; The temporal spatial feature information is input into the detection output module to obtain the first slump detection result output by the detection output module.
4. The slump detection method according to claim 1, characterized in that, The step of calculating the first target slump test result of the concrete to be tested based on the first slump test result and the second slump test result includes: Based on the first slump detection result, the second slump detection result, the first fusion weight corresponding to the first slump detection model, and the second fusion weight corresponding to the second slump detection model, the first target slump detection result is obtained by fusion calculation; The first fusion weight and the second fusion weight are obtained by training the weight parameters between the first collapse detection model and the second collapse detection model based on the historical total power difference corresponding to the mixing host, several sets of video frame sequences to be trained, and the true collapse labels corresponding to each of the video frame sequences to be trained.
5. The slump detection method according to claim 1, characterized in that, Before acquiring the unloading video frame sequence corresponding to the unloading area of the concrete to be tested, the method further includes: Acquire the multimodal characteristic data of the concrete to be tested in the mixing host during the mixing process; The multimodal feature data is input into the third slump detection model to obtain the third slump detection result output by the third slump detection model; Based on the third slump detection result and the second slump detection result, the second target slump detection result is obtained by fusion calculation; Based on the second target slump detection result, determine whether the slump of the concrete to be tested is in an abnormal state during the mixing process; If not, return to the step of collecting the unloading video frame sequence corresponding to the concrete to be tested in the unloading area; If so, slump adjustment information is generated to make adjustments based on the slump adjustment information.
6. The slump detection method according to claim 5, characterized in that, The third slump detection model is trained based on the following steps: Acquire training multidimensional feature data for several types of concrete, wherein the training multidimensional feature data includes at least concrete material mix proportion data and humidity data; Based on the multidimensional feature data to be trained and the collapse labels corresponding to the multidimensional feature data to be trained, the detection model to be trained is iteratively trained to obtain the third collapse detection model.
7. A slump detection device, characterized in that, include: The acquisition module is used to acquire the total power difference of the mixing shaft drive motor in the mixing host, and to collect the unloading video frame sequence corresponding to the concrete to be tested; The acquisition of the unloading video frame sequence corresponding to the concrete to be tested in the unloading area includes: capturing the feeding video of the unloading area through a camera, wherein the unloading area is the receiving hopper of the mixer truck or the waiting hopper of the throttle valve; extracting video frames with continuous time from the feeding video or extracting video frames according to a preset time interval to obtain the unloading video frame sequence. The first detection module is used to input the unloading video frame sequence into the first slump detection model to obtain the first slump detection result output by the first slump detection model. The second detection module is used to input the total work difference into the second slump detection model to obtain the second slump detection result output by the second slump detection model. The second slump detection model is trained based on the following steps: for any concrete strength, obtain each total work difference feature to be fitted corresponding to different slumps under the concrete strength; perform dimensionless processing on each of the total work difference features to be fitted to obtain a dimensionless feature data set for different slumps; perform probability distribution fitting on the dimensionless feature data set corresponding to any slump under any concrete strength to obtain the second slump detection model corresponding to any slump under any concrete strength. The calculation module is used to calculate the first target slump test result of the concrete to be tested by fusing the first slump test result and the second slump test result. The first collapse detection model is obtained by iterative training based on video frame sequence samples and the sample labels corresponding to the video frame sequence samples. The second slump detection model is obtained by probability distribution fitting based on the total work difference characteristics corresponding to different slumps under various concrete strengths.
8. An engineering machinery, characterized in that, Includes a power collector, camera, and server, among which: A power acquisition device, installed on the mixing host, is used to collect current and voltage data of the mixing shaft drive motor in the mixing host. The current and voltage data are used to determine the total power difference. The camera is used to capture the unloading video frame sequence corresponding to the unloading area; The server includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the slump detection method as described in any one of claims 1 to 6.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the slump detection method as described in any one of claims 1 to 6.