Image detection slump model migration use method based on different working conditions

By using the image detection model migration method under different working conditions, the problem of inaccurate slump monitoring caused by the differences in the concrete mixing process is solved, and the rapid model update and real-time detection in the new mixing station are realized, which improves the concrete production efficiency and quality.

CN119048966AInactive Publication Date: 2024-11-29FUJIAN SOUTHERN HIGHWAY MECHANICAL CO LTD +1

Patent Information

Application Number
CN202411531592.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-30
Publication Date
2024-11-29
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The difference in the concrete mixing process under different working conditions leads to different image feature areas, making it difficult to achieve stable and accurate concrete slump monitoring. In addition, traditional methods require a lot of manual labeling and data collection, which is time-consuming and cost-effective.

Method used

Using a model migration method based on image detection, the initial concrete slump detection model is established by obtaining previous production data for labeling and preprocessing, and searching for the best data enhancement strategy using 3D convolutional neural network and NAS to migrate the model to the new stirring The station conducts training to reduce the workload of manual labeling and improve detection efficiency.

Benefits of technology

It realizes stable and accurate concrete slump monitoring under different working conditions, reduces the workload of manual test marking, saves time and costs, and improves the efficiency and quality of concrete production.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119048966A_ABST
    Figure CN119048966A_ABST
Patent Text Reader

Abstract

The invention provides an image detection slump model migration use method based on different working conditions. The image detection slump model migration use method comprises the following steps: acquiring previous concrete production data and labeling, acquiring previous concrete stirring video data and preprocessing, establishing an initial concrete slump detection model and migrating the initial concrete slump detection model. The invention relates to a method. The method comprises the following steps of: acquiring production data of previous concrete and labeling, acquiring previous concrete mixing video data and preprocessing, and establishing an initial concrete slump detection model to carry out image processing on a uniform mixing video, establishing an initial model and applying the model to a new mixing station; the model can be quickly updated, the workload of manual test marking is reduced, time and cost are saved, the slump degree is detected in real time in the concrete production process, the production formula can be adjusted in time, and the concrete production efficiency and quality are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of concrete slump prediction, and more particularly to a method for migrating and using a slump model detected by images under different working conditions. Background Art

[0002] In the concrete production process, the quality control of concrete is very important. Among them, the slump of concrete is one of the important indicators for evaluating the performance of concrete, and it needs to be tested during the production process. However, the traditional method of testing the slump of concrete has some defects, such as cumbersome process, large measurement error, hysteresis, etc.

[0003] In recent years, with the development of computer vision technology, image-based concrete slump detection methods have gradually attracted attention. This method can realize real-time monitoring of concrete slump by analyzing the video of the concrete mixing process.

[0004] However, due to the differences in the concrete mixing process under different working conditions, the captured image feature areas are quite different, making it difficult to achieve stable and accurate monitoring. At the same time, for different concrete mixing stations, the mixing video data collected are diverse, and a large number of concrete video samples need to be manually labeled and data collected, and the slump prediction model needs to be re-created, which is time-consuming and costly. Summary of the invention

[0005] The present invention discloses a method for migrating a slump model based on images of different working conditions, mainly to solve the problem that differences in concrete mixing processes under different working conditions lead to large differences in captured image feature areas, making it difficult to achieve stable and accurate monitoring.

[0006] To achieve the above object, the technical solution of the present invention is implemented as follows: The present invention provides a method for migrating and using a slump model detected by an image based on different working conditions, comprising the following steps: S1. Obtain previous concrete production data and label it: obtain previous concrete mixing video data, production requirements data, current curve data of the mixing process, and use the distribution of current curve characteristics to calibrate the slump, and assign slump labels to the mixing video data; S2. Obtaining previous concrete mixing video data and preprocessing: Obtaining previous concrete mixing video data, and filtering out concrete mixing uniform video data from the concrete mixing video data, intercepting the concrete mixing uniform video and processing it into an image sequence, and intercepting the image sequence to extract the ROI area as a data set; S3. Establish an initial concrete slump detection model: obtain the image sequence data set after the ROI area is cut and input into the 3D convolutional neural network model, use the NAS (Neural Architecture Search) method to search for the best data enhancement strategy and apply it to the 3D convolutional neural network model for training, and obtain the initial concrete slump detection model of the mapping relationship between image data and corresponding slump labels; S4. Migration of the initial concrete slump detection model: Apply the initial concrete slump detection model to the new mixing station, and obtain a partial image sequence of the new mixing station after being processed, assigned with slump labels and after ROI areas are captured using the steps of S1 and S2, and input it into the initial concrete slump detection model for training to obtain the migrated concrete slump detection model.

[0007] In one embodiment, the method for intercepting the ROI area is to locate the position of the stirring blade of the mixer in the image sequence, extract the boundary contour line of the stirring blade, and obtain a coordinate point on the boundary contour line as the absolute coordinate, and intercept it based on the absolute coordinate to obtain the ROI area.

[0008] In one implementation, the absolute coordinate position is the coordinate point of the lower left corner on the boundary contour line.

[0009] In one embodiment, the first 80 frames of images in the image sequence after the ROI area is intercepted are used as initial frames, and 160 frames of images are selected backward as an image sequence, and the image sequence is downsampled using the time resolution to reduce the computational complexity of the model.

[0010] In one embodiment, the NAS (Neural Architecture Search) method includes a search space, which includes multiple data enhancement strategies, each data enhancement strategy includes an image processing function and a probability of using the image processing function, and an image processing function is selected using a predefined probability of using the image processing function, and is applied to an image sequence to obtain multiple data subsets, and a cross-validation method is used to evaluate the performance of various data enhancement strategies on multiple data subsets, and the probability distribution parameters of the search space are adjusted or new image processing functions are added according to the evaluation results until the best data enhancement strategy is searched.

[0011] In one embodiment, the 3D convolutional neural network model includes an input layer, a hidden layer and an output layer, the input layer is used to receive three-dimensional input data of time step, width and height, the hidden layer includes two 3D convolution layers, two batch normalization layers, two ReLU activation functions, a maximum pooling layer and a fully connected layer, the 3D convolution layer, the batch normalization layer, the ReLU activation function, the maximum pooling layer and the fully connected layer are connected in sequence, the first 3D convolution layer uses 32 3D convolution kernels, the 3D convolution kernel size is 5×5×5, the second convolution layer uses 48 3D convolution kernels, the 3D convolution kernel size is 3×3×3, the maximum pooling layer includes a 3D pooling kernel, and the 3D pooling kernel size is 2×2×2.

[0012] In one implementation, the image processing function includes functions such as image shearing, translation, and color shift.

[0013] In one embodiment, in the step S4, the slump of actual concrete is detected by sampling, and the error is compared with the concrete slump predicted by the migrated concrete slump detection model. An error threshold is preset. When the error is less than the error threshold, the migrated concrete slump detection model can continue to be used. Otherwise, the steps S1 and S2 are repeated to update the model until the error is less than the threshold.

[0014] The advantages or beneficial effects of the above technical solution include at least: image processing of the mixing uniformity video and establishment of the initial model by obtaining and labeling the previous concrete production data, obtaining and preprocessing the previous concrete mixing video data and establishing the initial concrete slump detection model; the method of migrating the initial concrete slump detection model can be used to process the mixing uniformity video of the new mixing station, and make the processed image sequence consistent with that used when building the model, so as to facilitate its application to the initial model, thereby enabling the model to be quickly updated and a suitable model to be obtained, reducing the workload of manual test labeling, saving time and cost, and adjusting the production formula in time through real-time detection of slump during concrete production to improve the efficiency and quality of concrete production. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings illustrate exemplary embodiments of the present invention and together with the description serve to explain the principles of the present invention. These drawings are included to provide a further understanding of the present invention and are incorporated in and constitute a part of this specification.

[0016] Figure 1 A flowchart of an image sequence acquisition method according to an exemplary embodiment of the present invention is shown; Figure 2 A schematic diagram of the structure of a 3D convolutional neural network model according to an exemplary embodiment of the present invention is shown; Figure 3 A flow chart of a method for migrating a slump model using an image detection method according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0017] Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

[0018] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0019] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0020] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

[0023] See also Figures 1 to 3 The embodiment of the present invention provides a method for migrating and using a slump model detected by an image based on different working conditions, comprising the following steps: S1. Obtaining and labeling previous concrete production data: Obtain previous concrete mixing video data, production requirement data, and current curve data of the mixing process, and use the distribution of current curve characteristics to calibrate the slump, and assign a slump label to the mixing video data.

[0024] The stirring video data includes a stirring uniformity video.

[0025] S2. Obtaining previous concrete mixing video data and preprocessing: Obtaining previous concrete mixing video data, and filtering out concrete mixing uniformity video data from the concrete mixing video data, intercepting the concrete mixing uniformity video and processing it into an image sequence, and intercepting the ROI area from the image sequence as a data set.

[0026] The method for capturing the ROI area is to locate the position of the mixer blade in the image sequence, extract the boundary contour of the mixer blade, obtain a coordinate point on the boundary contour as the absolute coordinate, and capture it based on the absolute coordinate to obtain the ROI area.

[0027] The absolute coordinates are the coordinates of the lower left corner of the boundary contour line. By intercepting the ROI area of ​​the image sequence in this way, it is possible to prevent the interception position of the ROI area from being biased, thereby ensuring the accuracy and consistency of the image sequence after intercepting the ROI area.

[0028] The first 80 frames of images in the image sequence after the ROI area is intercepted are used as the initial frames, and 160 frames of images are selected backward as an image sequence to achieve the temporal expansion of the data set, and the image sequence is downsampled using the time resolution to reduce the computational complexity of the model.

[0029] S3. Establish an initial concrete slump detection model: Obtain a sequence of image data sets after cutting out the ROI area and input them into the 3D convolutional neural network model. Use the NAS (Neural Architecture Search) method to search for the best data enhancement strategy and apply it to the 3D convolutional neural network model for training, and obtain an initial concrete slump detection model with a mapping relationship between image data and corresponding slump labels. The data set can be divided into a training set and a test set. The training set is used to train the 3D convolutional neural network model to improve the generalization ability of the model, and the test set is used to test the performance of the initial concrete slump detection model, and select the optimal model as the initial concrete slump detection model for subsequent migration.

[0030] Among them, the NAS (Neural Architecture Search) method includes a search space, which includes multiple data enhancement strategies. Each data enhancement strategy includes an image processing function and the probability of using the image processing function. The image processing function is selected using the predefined probability of using the image processing function and applied to the image sequence to obtain multiple data subsets. The cross-validation method is used to evaluate the performance of various data enhancement strategies on multiple data subsets, and the probability distribution parameters of the search space are adjusted or new image processing functions are added according to the evaluation results until the best data enhancement strategy is searched. The best data enhancement strategy is obtained by this method, and the best data enhancement strategy is applied to the training of the 3D convolutional neural network model, which can generate more samples to expand the scale and diversity of the data set, thereby helping the 3D convolutional neural network model to learn better and improve the quality of image data and the performance of the initial concrete slump detection model.

[0031] The 3D convolutional neural network model includes an input layer, a hidden layer, and an output layer. The input layer is used to receive three-dimensional input data of time step, width, and height. The hidden layer includes two 3D convolutional layers, two batch normalization layers, two ReLU activation functions, a maximum pooling layer, and a fully connected layer. The 3D convolutional layer, batch normalization layer, ReLU activation function, maximum pooling layer, and fully connected layer are connected in sequence. The first 3D convolutional layer uses 32 3D convolution kernels with a size of 5×5×5 to extract low-level features of the input data. After the first batch normalization layer and the first ReLU activation layer, the training process is accelerated and the model performance is improved. The second convolutional layer uses 48 3D convolution kernels with a size of 3×3×3, and passes through the second batch normalization layer and the second ReLU activation layer. The maximum pooling layer includes a 3D pooling kernel with a size of 2×2×2.

[0032] Image processing functions include but are not limited to image shearing, translation, color shift and other functions.

[0033] S4. Migration of the initial concrete slump detection model: Apply the initial concrete slump detection model to the new mixing station, and obtain a sequence of images with slump labels and ROI regions after processing the part of the new mixing station by the steps of S1 and S2, and input it into the initial concrete slump detection model for training to obtain the migrated concrete slump detection model. When the initial concrete slump detection model is migrated to the new mixing station, and the obtained sequence of images with slump labels and ROI regions after processing the part of the new mixing station is obtained, the data used is less than when the model was established, and after training, the model can be quickly updated to obtain a concrete slump detection model that conforms to the new mixing station.

[0034] The slump of actual concrete is detected by sampling, and the error is compared with the concrete slump predicted by the migrated concrete slump detection model. An error threshold is preset. When the error is less than the error threshold, the migrated concrete slump detection model can continue to be used. Otherwise, steps S1 and S2 are repeated to update the model until the error is less than the threshold.

[0035] By adopting the above method, the image processing of the mixing uniformity video and the establishment of the initial model are carried out by obtaining and labeling the previous concrete production data, obtaining and preprocessing the previous concrete mixing video data and establishing the initial concrete slump detection model. The method of migrating the initial concrete slump detection model can be used to process the mixing uniformity video of the new mixing station, and make the processed image sequence consistent with the one used when building the model, so as to facilitate its application to the initial model, so as to quickly update the model and obtain a suitable model, reducing the workload of manual test labeling, saving time and cost, and through real-time detection of slump in the concrete production process, the production formula can be adjusted in time to improve the efficiency and quality of concrete production. Example

[0036] Obtain monitoring videos, production order data, and main shaft current curve data of all mixing processes of the mixing station from 2021 to 2023. According to the different concrete compressive strength requirements in the production order (including concrete from C15 to C50), make corresponding concrete slump detection models. Since there is a correlation between the mixing current and the measured slump, the characteristic distribution of the current curve data can be used and a small amount of slump test calibration can be performed to give the mixing video data a slump label.

[0037] Reference Figure 1 , preprocess the mixing video data, intercept 10 seconds of video of each plate of concrete mixing stage, and process it into an image sequence. Select the moment when the mixing blade leaves the material surface as the initial frame, and perform threshold segmentation on the frame to obtain the edge contour of the mixing blade for analysis, and intercept the ROI area based on the coordinate point of the lower left corner of the edge contour of the mixing blade. The first 80 frames of the image sequence after intercepting the ROI area are used as the initial frame, and 160 frames of images are selected as an image sequence to achieve the temporal expansion of the data set; at the same time, the time resolution is used to downsample the samples to reduce the computational complexity of the model.

[0038] The processed image data set is divided into a training set and a test set. A 3D convolutional neural network model is built using an industrial computer for training and testing, and an initial concrete slump detection model with a mapping relationship between input image data and corresponding slump labels is constructed. Figure 2The 3D convolutional neural network mainly includes an input layer, a hidden layer and an output layer. The input layer receives three-dimensional input data including time steps, width, height and channels; the hidden layer learns the representation features of the input data, including two 3D convolutional layers, two batch normalization layers, and two ReLU layers. Activation function, maximum pooling layer and fully connected layer; the first 3D convolution layer uses 32 3D convolution kernels with a size of 5×5×5 to extract low-level features of the input data, and passes through a batch normalization layer and a ReLU activation layer to accelerate the training process and improve model performance. The second 3D convolution layer uses 48 3D convolution kernels with a size of 3×3×3, and also passes through a batch normalization layer and a ReLU activation layer; then connect the maximum pooling layer to perform the maximum pooling operation with a pooling kernel of 2×2×2, flatten the output of the maximum pooling layer and connect it to a fully connected layer; finally, regress the weighted results of the neurons through a linear connection layer with a unique output to complete the mapping of features to output results, that is, slump.

[0039] The NAS (Neural Architecture Search) method is used to search for the best data augmentation strategy. The training sample expansion data set is generated according to the best data augmentation strategy to improve the generalization ability of the model during the training process. After the training is completed, the test set is used to evaluate the final performance of the model and the best model is selected as the initial model for subsequent migration.

[0040] Specifically, the above detection program is deployed to an industrial computer, and the mixing host camera, the PLC device storing the current curve data of the mixing spindle, and the formula data storage computer are connected through a network cable. Before the production of each order task, the formula data and mixing video data are obtained; according to the strength information in the formula, the corresponding initial concrete slump detection model is called to perform real-time detection of concrete slump. Example

[0041] Reference Figure 3, deploy the initial concrete slump detection model in Example 1 to different mixing station production lines, ensure that the target deployment environment has installed the necessary software and libraries that the initial concrete slump detection model relies on, and connect the mixing host camera and the formula data storage computer through a network cable to obtain the mixing video data and formula data; and similarly to the method for collecting and processing mixing video data in Example 1, that is, after collecting part of the original mixing video data set to be predicted for the new mixing station, the image sequence data of the concrete slump is intercepted in the ROI area, and expanded using the time series expansion method of the data set, and then the initial concrete slump detection model trained in Example 1 is called to predict the slump of the image sequence samples, and an artificial sampling test is performed to compare the error between the prediction result and the actual slump value to verify the accuracy of the model.

[0042] The quality of the above migration effect depends on the consistency of the source domain characteristics and the target domain characteristics. By adding random samples to the training data set, retraining the model, and adjusting the model weights to adapt to the characteristics of the target domain, the initial concrete slump detection model is migrated to different mixing plant production lines, and a concrete slump detection model that conforms to the new mixing plant is obtained, thereby ensuring stability and accuracy.

[0043] When the present invention is actually used, it is required that the mixing station deployed in use can obtain the mixing process monitoring video, the main machine mixing current data and the production formula data, and a small amount of mixing video data and slump calibration test are required to complete the model update.

[0044] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0045] It should be understood by those skilled in the art that the above embodiments are only for the purpose of clearly illustrating the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, other changes or modifications may be made based on the above invention, and these changes or modifications are still within the scope of the present invention.

Claims

1. A method for migrating and using a slump model based on images of different working conditions, characterized in that: The following steps are included: S1. Obtain previous concrete production data and label it: obtain previous concrete mixing video data, production requirements data, current curve data of the mixing process, and use the distribution of current curve characteristics to calibrate the slump, and assign slump labels to the mixing video data; S2. Obtaining previous concrete mixing video data and preprocessing: Obtaining previous concrete mixing video data, and filtering out concrete mixing uniform video data from the concrete mixing video data, intercepting the concrete mixing uniform video and processing it into an image sequence, and intercepting the image sequence to extract the ROI area as a data set; S3. Establish an initial concrete slump detection model: obtain the image sequence data set after the ROI area is cut and input into the 3D convolutional neural network model, use the NAS (Neural Architecture Search) method to search for the best data enhancement strategy and apply it to the 3D convolutional neural network model for training, and obtain the initial concrete slump detection model of the mapping relationship between image data and corresponding slump labels; S4. Migration of the initial concrete slump detection model: Apply the initial concrete slump detection model to the new mixing station, and obtain a partial image sequence of the new mixing station after being processed, assigned with slump labels and after ROI areas are captured using the steps of S1 and S2, and input it into the initial concrete slump detection model for training to obtain the migrated concrete slump detection model.

2. The method for migrating and using the image detection slump model based on different working conditions according to claim 1, characterized in that: The method for intercepting the ROI area is to locate the position of the stirring blade of the mixer in the image sequence, extract the boundary contour line of the stirring blade, and obtain a coordinate point on the boundary contour line as the absolute coordinate, and intercept with the absolute coordinate as the reference to obtain the ROI area.

3. The method for migrating and using the image detection slump model based on different working conditions as claimed in claim 2, characterized in that: The position of the absolute coordinates is the coordinate point of the lower left corner on the boundary contour line.

4. The method for migrating and using the image detection slump model based on different working conditions as claimed in claim 2, characterized in that: The first 80 frames of images in the image sequence after the ROI area is intercepted are used as the initial frames, and 160 frames of images are selected backward as an image sequence, and the image sequence is downsampled using the time resolution to reduce the computational complexity of the model.

5. The method for migrating and using the image detection slump model based on different working conditions according to claim 1, characterized in that: The NAS (Neural Architecture Search) method includes a search space, which includes multiple data enhancement strategies. Each data enhancement strategy includes an image processing function and a probability of using the image processing function. The image processing function is selected using the predefined probability of using the image processing function and applied to the image sequence to obtain multiple data subsets. A cross-validation method is used to evaluate the performance of various data enhancement strategies on the multiple data subsets, and the probability distribution parameters of the search space are adjusted or new image processing functions are added according to the evaluation results until the best data enhancement strategy is searched.

6. The method for migrating and using the image detection slump model based on different working conditions according to claim 1, characterized in that: The 3D convolutional neural network model includes an input layer, a hidden layer and an output layer. The input layer is used to receive three-dimensional input data of time step, width and height. The hidden layer includes two 3D convolutional layers, two batch normalization layers, two ReLU activation functions, a maximum pooling layer and a fully connected layer. The 3D convolutional layer, the batch normalization layer, the ReLU activation function, the maximum pooling layer and the fully connected layer are connected in sequence. The first 3D convolutional layer uses 32 3D convolution kernels with a 3D convolution kernel size of 5×5×5. The second convolutional layer uses 48 3D convolution kernels with a 3D convolution kernel size of 3×3×3. The maximum pooling layer includes a 3D pooling kernel with a 3D pooling kernel size of 2×2×2.

7. The method for migrating and using the image detection slump model based on different working conditions as claimed in claim 5, characterized in that: The image processing function includes image shearing, translation, and color shift functions.

8. The method for migrating and using the image detection slump model based on different working conditions as described in claim 1, wherein in the step S4, the slump of actual concrete is detected by sampling, and the error is compared with the concrete slump predicted by the migrated concrete slump detection model, and an error threshold is preset. When the error is less than the error threshold, the migrated concrete slump detection model can continue to be used, otherwise, the steps S1 and S2 are repeated to update the model until the error is less than the threshold.

Citation Information

Patent Citations

  • Button surface defect detection method based on transfer learning

    CN111161228A

  • Slump monitoring method and device based on image sequence and readable medium

    CN117218118A

  • Slump monitoring method and device based on multi-modal data and readable medium

    CN117390586A

  • Small sample fabric defect detection method based on transfer learning

    CN118411565A

  • Zero-order learning intelligent early warning method based on multi-modal deep learning driving

    CN118429899A

Cited By

  • Concrete slump prediction method and system

    CN121437938A