Method and device for monitoring discharging of mixing plant and storage medium

Abstract

The invention discloses a method and device for monitoring discharging of a mixing plant and a storage medium, and relates to the technical field of concrete detection.The method comprises the steps that a plurality of discharging images of the mixing plant are obtained; determining an aggregate area of each blanking image; determining at least one feature of each aggregate region, wherein the at least one feature comprises at least one of an aggregate area proportion, an aggregate distribution density and an aggregate movement speed; and determining the blanking density of the mixing plant according to at least one feature corresponding to all the aggregate areas so as to adjust the blanking speed of the mixing plant according to the blanking density. The blanking density is determined by identifying the features of the image, the blanking speed of the mixing plant can be adjusted, waste of aggregate caused by misoperation of an operator of the mixing plant is reduced, the production process is optimized, and the production efficiency and the product quality are improved.

Description

Technical Field

[0001] This application relates to the technical field of concrete detection, and specifically to a method, device, industrial robot and storage medium for monitoring the feeding of a mixing station. Background Art

[0002] In the construction industry, the production quality and efficiency of concrete are crucial. The precise monitoring of the aggregate metering process in a mixing station is a key link to ensure the quality of concrete and control costs. Traditional aggregate metering in mixing stations mainly relies on manual observation and empirical judgment, which has the disadvantages of strong subjectivity, low accuracy, and low efficiency. With the progress of automation technology, some sensor-based aggregate metering methods have been applied, but these methods are easily affected by environmental interference, and it is also difficult to accurately judge the density of the aggregate feeding process. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a method, device, industrial robot and storage medium for monitoring the feeding of a mixing station, so as to solve the technical problems in the prior art that it is difficult to judge the density of aggregate feeding and the feeding speed cannot be accurately regulated.

[0004] To achieve the above purpose, the first aspect of this application provides a method for monitoring the feeding of a mixing station, and the method includes:

[0005] Obtain multiple feeding images of the mixing station;

[0006] Determine the aggregate area of each feeding image;

[0007] Determine at least one feature of each aggregate area, and the at least one feature includes at least one of the aggregate area ratio, aggregate distribution density, and aggregate movement speed;

[0008] Determine the feeding density of the mixing station according to at least one feature corresponding to all aggregate areas;

[0009] Adjust the feeding speed of the mixing station according to the feeding density.

[0010] In the embodiments of this application, determining at least one feature of each aggregate area includes at least one of the following: determining the aggregate area ratio corresponding to each aggregate area; determining the aggregate distribution density corresponding to each feeding image according to the pixel values of each pixel point included in each aggregate area; determining the aggregate movement speed corresponding to each feeding image.

[0011] In an embodiment of the present application, determining the feeding density of a mixing station according to at least one feature corresponding to all aggregate areas includes: determining a first statistical value of the proportion of the aggregate area corresponding to all aggregate areas, a second statistical value of the aggregate distribution density corresponding to all aggregate areas, and a third statistical value of the aggregate movement speed corresponding to all aggregate areas; when any two of the following conditions are met, determining that the feeding density is dense: First, the first statistical value is greater than or equal to a first preset proportion threshold; Second, the second statistical value is greater than or equal to a first preset density threshold; Third, the third statistical value is greater than or equal to a first preset speed threshold; when any two of the following conditions are met, determining that the feeding density is sparse: Fourth, the first statistical value is less than or equal to a second preset proportion threshold; Fifth, the second statistical value is less than or equal to a second preset density threshold; Sixth, the third statistical value is less than or equal to a second preset speed threshold; when none of the above conditions are met, determining that the feeding density is normal; wherein, the first preset proportion threshold is greater than the second preset proportion threshold, the first preset density threshold is greater than the second preset density threshold, and the first preset speed threshold is greater than the second preset speed threshold.

[0012] In an embodiment of the present application, adjusting the feeding speed of a mixing station according to the feeding density includes: when the feeding density is normal, determining the target feeding speed of the mixing station according to the first statistical value, the first preset density threshold, the second preset density threshold, the first feeding speed, and the second feeding speed, so as to adjust the feeding speed to the target feeding speed; wherein, the first feeding speed is less than the second feeding speed, the first feeding speed refers to the minimum feeding speed of the mixing station, and the second feeding speed is the maximum feeding speed of the mixing station.

[0013] In an embodiment of the present application, adjusting the blanking speed of the mixing station according to the blanking density further includes: determining a first weight corresponding to the aggregate area ratio, a second weight corresponding to the aggregate distribution density, and a third weight corresponding to the aggregate movement speed; determining a first speed adjustment coefficient corresponding to the blanking speed according to the first statistical value, the first preset ratio threshold, and the second ratio threshold; determining a second speed adjustment coefficient corresponding to the blanking speed according to the second statistical value, the first preset density threshold, and the second preset density threshold; determining a third speed adjustment coefficient corresponding to the blanking speed according to the third statistical value, the first preset speed threshold, and the second preset speed threshold; when the blanking density is normal, updating the target blanking speed in any of the following ways to adjust the blanking speed to the updated target blanking speed: First, updating the target blanking speed according to the first weight, the second weight, the first speed adjustment coefficient, and the second speed adjustment coefficient; Second, updating the target blanking speed according to the first weight, the third weight, the first speed adjustment coefficient, and the third speed adjustment coefficient; Third, updating the target blanking speed according to the second weight, the third weight, the second speed adjustment coefficient, and the third speed adjustment coefficient; Fourth, updating the target blanking speed according to the first weight, the second weight, the third weight, the first speed adjustment coefficient, the second speed adjustment coefficient, and the third speed adjustment coefficient.

[0014] In an embodiment of the present application, determining the aggregate distribution density corresponding to each blanking image according to the pixel value of each pixel point included in each aggregate area includes: determining the pixel value of all pixel points in each aggregate area to determine the gray value variance of each aggregate area; determining the aggregate distribution density of each blanking image according to the gray value variance of each aggregate area.

[0015] In an embodiment of the present application, determining the aggregate movement speed corresponding to each blanking image includes: determining the target aggregate in each aggregate area, and determining the first aggregate position and the aggregate characteristics of the target aggregate; determining the target aggregate area where the target aggregate exists in other aggregate areas according to the aggregate characteristics; determining the second aggregate position of the target aggregate in the target aggregate area; determining the aggregate movement speed corresponding to each blanking image according to the first aggregate position and the second aggregate position.

[0016] In an embodiment of the present application, determining the aggregate area of each blanking image includes: processing each blanking image to obtain a processed blanking image; inputting each processed blanking image into a target detection model, and determining a high-level semantic feature map and a low-level detail feature map corresponding to each processed blanking image through the target detection model, where the high-level semantic feature map includes aggregate type information of each processed blanking image, and the low-level detail feature map includes aggregate edge information and aggregate texture information of each processed blanking image; performing upsampling processing on the high-level semantic feature map of each processed blanking image to obtain a first target feature map with the same image size as the low-level detail feature map corresponding to each processed blanking image; performing convolution processing on the low-level detail feature map corresponding to each processed blanking image to obtain a second target feature map with the same number of channels as the high-level semantic feature map corresponding to each processed blanking image; performing fusion processing on the first target feature map and the second target feature map to obtain a third target feature map corresponding to each processed blanking image; processing the third target feature map corresponding to each processed blanking image to output the aggregate area of each processed blanking image through the target detection model.

[0017] A second aspect of the present application provides a device for monitoring the blanking of a mixing plant, including:

[0018] A memory configured to store instructions;

[0019] A processor configured to call the instructions from the memory and capable of implementing the method for monitoring the blanking of a mixing plant according to the above.

[0020] A third aspect of the present application is a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the method for monitoring the blanking of a mixing plant according to the above.

[0021] A fourth aspect of the present application provides an industrial robot, including the device for monitoring the blanking of a mixing plant according to the above.

[0022] Through the above technical solutions, a plurality of blanking images of the mixing plant are obtained; the aggregate area of each blanking image is determined; at least one feature of each aggregate area is determined, and the at least one feature includes at least one of the aggregate area ratio, the aggregate distribution density, and the aggregate movement speed; the blanking density of the mixing plant is determined according to the at least one feature corresponding to all the aggregate areas, so as to adjust the blanking speed of the mixing plant according to the blanking density. By identifying the features of the images to determine the blanking density, the blanking speed of the mixing plant can be adjusted, reducing the waste of aggregates caused by the misoperation of the mixing plant operator, optimizing the production process, and improving the production efficiency and product quality.

[0023] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation section. Brief Description of the Drawings

[0024] The drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the following specific implementation, they are used to explain the embodiments of the present application, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0025] Figure 1 Schematically shows a flowchart of a method for monitoring the discharging of a mixing plant according to an embodiment of the present application;

[0026] Figure 2 Schematically shows a structural block diagram of a device for monitoring the discharging of a mixing plant according to an embodiment of the present application;

[0027] Figure 3 Schematically shows a structural diagram of a computer device according to an embodiment of the present application. Specific Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It should be understood that the specific implementation described herein is only used to explain and illustrate the embodiments of the present application and does not limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application.

[0029] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present application, the directional indications are only used to explain the relative positional relationship and movement conditions between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0030] In addition, if there are descriptions involving "first", "second", etc. in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0031] Figure 1 The following schematically shows a flow chart of a method for monitoring material discharge in a mixing station according to an embodiment of the present application. Figure 1 As shown, an embodiment of the present application provides a method for monitoring material unloading in a mixing station, which may include the following steps.

[0032] S102, acquiring multiple unloading images of the mixing station.

[0033] S104, determining the aggregate area of ​​each blanking image.

[0034] S106, determining at least one feature of each aggregate region, where the at least one feature includes at least one of aggregate area ratio, aggregate distribution density, and aggregate movement speed.

[0035] S108, determining the material discharge density of the mixing station according to at least one feature corresponding to all aggregate areas.

[0036] S110, adjusting the material discharge speed of the mixing station according to the density of the material discharge.

[0037] It can be understood that aggregate refers to granular loose materials that act as a skeleton or filler in concrete, such as sand, gravel, etc. The unloading image refers to the image of the aggregate unloading port of the mixing plant taken by the image acquisition device. The camera that collects images or videos can be installed near the aggregate unloading port of the mixing plant. The lighting conditions should be sufficient and uniform to avoid the impact of lighting problems on image quality and algorithm judgment.

[0038] Aggregate area refers to the area where the aggregate is located in the unloading image. The feature of the aggregate area refers to the image feature of the aggregate area. The feature includes at least one of the aggregate area ratio, aggregate distribution density and aggregate movement speed. Among them, the aggregate area ratio refers to the area ratio of the aggregate area in the unloading image. Aggregate distribution density refers to the distribution of pixels where aggregate exists in the aggregate area. Aggregate movement speed refers to the movement speed of the aggregate in the unloading image at the time corresponding to each unloading image. The density of unloading refers to the density of the aggregate unloading process of the mixing station. The denser the unloading density, the faster the aggregate unloading speed and the larger the unloading amount. Since traditional sensors can usually only measure a single parameter, such as weight, it is difficult to fully reflect the density of aggregate unloading. For example, the sensor cannot distinguish whether the aggregate is unloaded uniformly or concentratedly, which is not conducive to optimizing the production process and controlling quality. Therefore, the unloading density of the mixing station can be determined by extracting the video captured by the camera frame by frame to obtain a series of image frames, extracting the aggregate area of ​​each frame of the unloading image, and determining at least one feature of each aggregate area. According to the material discharge density feedback from the image, the material discharge speed of the mixing station can be adjusted to reduce the waste of aggregates caused by misoperation of the mixing station operator, optimize the production process, and improve production efficiency and product quality.

[0039] In an embodiment of the present application, determining at least one feature of each aggregate area includes at least one of the following: determining the proportion of the aggregate area corresponding to each aggregate area; determining the aggregate distribution density corresponding to each blanking image according to the pixel values of each pixel point included in each aggregate area; determining the aggregate movement speed corresponding to each blanking image.

[0040] It can be understood that in the aggregate area of each blanking image, the area where there is aggregate is the aggregate area, and the area where there is no aggregate can be considered as the background area. The pixel values of the pixel points in the aggregate area are different from those of the pixel points in the background area. In the blanking image, the obvious shadow area compared with the background area is the aggregate area. Then, the aggregate distribution density corresponding to each blanking image can be determined according to the pixel values of each pixel point included in each aggregate area. The aggregate movement speed can be determined by tracking the aggregates in the aggregate area.

[0041] In an embodiment of the present application, the ratio between the number of pixel points of each aggregate area and the number of all pixel points included in the blanking image can be calculated and determined as the proportion of the aggregate area. Alternatively, if the aggregate area is a rectangle with a length of a pixels and a width of b pixels, then the area of the aggregate area is a×b. The image area of the blanking image can be calculated through the resolution of the image (with a length of L pixels and a width of W pixels), that is, the image area is L×W. Finally, the area of the aggregate area is divided by the area of the blanking image to obtain the proportion of the aggregate area.

[0042] In an embodiment of the present application, determining the aggregate distribution density corresponding to each blanking image according to the pixel values of each pixel point included in each aggregate area includes: determining whether each pixel point is a target pixel point according to the pixel value of each pixel point in each aggregate area, where the target pixel point refers to a pixel point where aggregates are detected; determining the aggregate distribution density of each blanking image according to the number of all target pixel points included in each aggregate area.

[0043] Specifically, since the aggregate blanking may be uneven and not every pixel point in the aggregate area has aggregates, a pixel threshold can be set, and the pixel points greater than or equal to the pixel threshold in each aggregate area are determined as target pixel points. Assuming that the aggregate area includes N pixel points and the number of target pixel points in the aggregate area is M, then the aggregate distribution density is M / N.

[0044] In an embodiment of the present application, determining the aggregate distribution density corresponding to each blanking image according to the pixel values of each pixel point included in each aggregate region includes: determining the pixel values of all pixel points in each aggregate region to determine the gray value variance of each aggregate region; determining the aggregate distribution density of each blanking image according to the gray value variance of each aggregate region.

[0045] Specifically, the gray value of each pixel point in the aggregate region can be calculated to determine the average gray value of the aggregate region. Then, for each pixel point in each aggregate region, according to the difference between the gray value of the pixel point and the average gray value, the gray value variance of the aggregate region can be calculated based on the differences between the gray values of all pixel points and the average gray value. The aggregate distribution density of the blanking image is represented by the gray value variance. A larger variance indicates that the gray values of the pixel points vary greatly, and the aggregate distribution density is relatively dispersed. A smaller variance indicates that the gray values of the pixel points are relatively close, and the aggregate distribution density is relatively dense.

[0046] In an embodiment of the present application, determining the aggregate movement speed corresponding to each blanking image includes: determining the target aggregate in each aggregate region, and determining the first aggregate position and aggregate characteristics of the target aggregate; determining the target aggregate region where the target aggregate exists in other aggregate regions according to the aggregate characteristics; determining the second aggregate position of the target aggregate in the target aggregate region; determining the aggregate movement speed corresponding to each aggregate region according to the first aggregate position and the second aggregate position.

[0047] It can be understood that the target aggregate can be the gravel with a relatively large volume in the aggregate region. The aggregate characteristics refer to the area, texture characteristics, shape characteristics, etc. presented by the target aggregate in the blanking image, which are used to track the position of the aggregate. Determine the target aggregate region where the target aggregate exists in other aggregate regions according to the aggregate characteristics; determine the second aggregate position of the target aggregate in the target aggregate region. According to the first aggregate position, the second aggregate position, and in combination with the acquisition time difference between the two images corresponding to the aggregate region and the corresponding target aggregate region, the aggregate movement speed corresponding to the aggregate region can be determined.

[0048] In an embodiment of the present application, determining the blanking density of the mixing station according to at least one characteristic corresponding to all aggregate regions includes: determining the blanking density of the mixing station according to the aggregate area ratio, aggregate distribution density, and aggregate movement speed corresponding to all aggregate regions. Analyzing the blanking density of the mixing station through multiple characteristics such as the aggregate area ratio, aggregate distribution density, and aggregate movement speed is more accurate.

[0049] In an embodiment of the present application, determining the feeding density of the mixing plant according to at least one feature corresponding to all aggregate areas includes: determining a first statistical value of the aggregate area ratio corresponding to all aggregate areas, a second statistical value of the aggregate distribution density corresponding to all aggregate areas, and a third statistical value of the aggregate movement speed corresponding to all aggregate areas; when any two of the following conditions are met, determining that the feeding density is dense: First, the first statistical value is greater than or equal to the first preset ratio threshold; Second, the second statistical value is greater than or equal to the first preset density threshold; Third, the third statistical value is greater than or equal to the first preset speed threshold; when any two of the following conditions are met, determining that the feeding density is sparse: Fourth, the first statistical value is less than or equal to the second preset ratio threshold; Fifth, the second statistical value is less than or equal to the second preset density threshold; Sixth, the third statistical value is less than or equal to the second preset speed threshold; when none of the above conditions are met, determining that the feeding density is normal; where the first preset ratio threshold is greater than the second preset ratio threshold, the first preset density threshold is greater than the second preset density threshold, and the first preset speed threshold is greater than the second preset speed threshold.

[0050] Specifically, by identifying multiple feeding images, multiple aggregate area ratios, multiple aggregate distribution densities, and multiple aggregate movement speeds can be obtained. Since the recognition results of single-frame images are unstable, the first statistical value of the aggregate area ratios of multiple frames of images, the second statistical value of multiple aggregate distribution densities, and the third statistical value of multiple aggregate movement speeds can be statistically calculated. The statistical value can be an average value, which can neutralize the problem of inaccurate recognition of single-frame images. Specifically, the aggregate movement speed can be calculated by dividing the change in the aggregate position between consecutive frames by the time interval. Assuming that within the time interval t, the aggregate movement speed in the image changes from the first aggregate position (X1, Y1) to the second aggregate position (X2, Y2), then the aggregate movement speed is calculated according to the following formula (1):

[0051]

[0052] where Velocity refers to the aggregate movement speed, and t refers to the acquisition time difference between the two images corresponding to the first aggregate position and the second aggregate position respectively.

[0053] The first preset proportion threshold is a threshold set by technicians based on technical experience for determining dense feeding according to the proportion of aggregate area. The second preset proportion threshold is a threshold set by technicians based on technical experience for determining sparse feeding according to the proportion of aggregate area. Similarly, the first preset density threshold is a threshold set by technicians based on technical experience for determining dense feeding according to the distribution density of aggregates, and the second preset density threshold is a threshold set by technicians based on technical experience for determining sparse feeding according to the distribution density of aggregates. The first preset speed threshold is a threshold set by technicians based on technical experience for determining dense feeding according to the movement density of aggregates, and the second preset speed threshold is a threshold set by technicians based on technical experience for determining sparse feeding according to the movement density of aggregates. For the proportion of aggregate area, through the analysis of images and experiments of a large number of different feeding situations, it is found that when the proportion of aggregate area exceeds 50%, it is generally considered that the feeding is relatively dense. The first preset density threshold is a threshold set by technicians based on technical experience for the distribution density of aggregates. Specifically, the first preset density threshold and the second preset density threshold can be adjusted according to the actual situation, such as setting according to factors such as the production scale of the mixing plant and the type of aggregates. For the movement speed of aggregates, considering that a faster feeding speed usually means denser feeding. After testing, it is set that when the position change speed of aggregates between consecutive frames exceeds a certain value, it is judged as a faster speed. The first preset speed threshold and the second preset speed threshold can be set by observing and analyzing the normal feeding speed, sparse feeding speed, and dense feeding speed. The distribution density of aggregates can be evaluated by calculating the distribution of pixel points in the aggregate area. Specifically, the ratio of the number of pixel points in the aggregate area to the area of the aggregate area can be used, or statistical quantities such as the variance of the gray values of pixel points in the aggregate area can be calculated. In order to accurately judge the degree of dense feeding of aggregates, when feeding aggregates, when any two of the following conditions are met, the degree of dense feeding is determined to be dense: First, the first statistical value is greater than or equal to the first preset proportion threshold; Second, the second statistical value is greater than or equal to the first preset density threshold; Third, the third statistical value is greater than or equal to the first preset speed threshold. When any two of the following conditions are met, the degree of dense feeding is determined to be sparse: Fourth, the first statistical value is less than or equal to the second preset proportion threshold; Fifth, the second statistical value is less than or equal to the second preset density threshold; Sixth, the third statistical value is less than or equal to the second preset speed threshold. And if only one statistical value is greater than or less than the threshold, but other statistical values are within the normal range, it can be considered that the degree of dense feeding is normal.

[0054] In the experiment, multiple groups of images were collected for different types of aggregates (such as stones and sands with different particle sizes) and different feeding methods (uniform feeding and concentrated feeding), and the above two distribution density indexes were calculated respectively. For example, through the analysis of 500 groups of images, it was found that when the ratio of the number of pixel points to the area of the aggregate region was used as the index, if this ratio exceeded a certain value (such as 0.6) and at the same time the variance of the gray value was small (indicating that the pixel points were relatively evenly distributed), the feeding situation was usually relatively dense and prone to accumulation; while when the ratio was low and the variance was large, the feeding was relatively sparse. For the movement speed, we considered that a faster feeding speed usually means a denser feeding. After installing a high-speed camera in the actual production environment of the mixing plant, recording the movement of the aggregates at a high frame rate (such as 60 frames per second) and accurately calculating the movement speed, the collected video data was analyzed, and it was found that when the position change speed of the aggregates between consecutive frames exceeded 10 pixels / frame, the feeding speed was fast, and problems such as aggregate splashing and inaccurate metering were likely to occur, and production parameters such as the belt transmission rate needed to be adjusted in a timely manner. Therefore, 10 pixels / frame was set as the threshold for judging the density of aggregate feeding (based on the movement speed). In practical applications, monitoring based on this threshold can timely detect abnormal feeding speeds, effectively reduce aggregate waste, improve metering accuracy, and ensure the production quality of concrete.

[0055] In an embodiment of the present application, adjusting the feeding speed of the mixing plant according to the feeding density includes: when the feeding density is normal, determining the target feeding speed of the mixing plant according to the first statistical value, the first preset density threshold, the second preset density threshold, the first feeding speed, and the second feeding speed, so as to adjust the feeding speed to the target feeding speed; wherein, the first feeding speed is less than the second feeding speed, the first feeding speed refers to the minimum feeding speed of the mixing plant, and the second feeding speed is the maximum feeding speed of the mixing plant.

[0056] Specifically, when the feeding density is normal, in order to make the feeding speed of the aggregates stable, the feeding speed can be adjusted according to the first statistical value, the first preset density threshold, the second preset density threshold, the first feeding speed, and the second feeding speed to obtain the current optimal target feeding speed. Specifically, the target feeding speed can be calculated according to the following formula (2):

[0057]

[0058] wherein, V refers to the calculated target feeding speed, V max refers to the first feeding speed, V minRefers to the second feeding speed, D refers to the aggregate distribution density obtained by calculating the distribution of pixel points in the aggregate area (such as the ratio of the number of pixel points in the aggregate area to the area of the aggregate area, or statistical quantities such as the variance of the gray values of pixel points in the aggregate area), D max Refers to the first preset density threshold set by combining a large number of experiments and analyzing the actual production situation of the mixing plant, D nin Refers to the second preset density threshold.

[0059] In the embodiments of the present application, adjusting the feeding speed of the mixing plant according to the feeding density further includes: determining a first weight corresponding to the aggregate area ratio, a second weight corresponding to the aggregate distribution density, and a third weight corresponding to the aggregate movement speed; determining a first speed adjustment coefficient corresponding to the feeding speed according to the first statistical value, the first preset ratio threshold, and the second ratio threshold; determining a second speed adjustment coefficient corresponding to the feeding speed according to the second statistical value, the first preset density threshold, and the second preset density threshold; determining a third speed adjustment coefficient corresponding to the feeding speed according to the third statistical value, the first preset speed threshold, and the second preset speed threshold; when the feeding density is normal, updating the target feeding speed in any of the following ways to adjust the feeding speed to the updated target feeding speed: First, update the target feeding speed according to the first weight, the second weight, the first speed adjustment coefficient, and the second speed adjustment coefficient; Second, update the target feeding speed according to the first weight, the third weight, the first speed adjustment coefficient, and the third speed adjustment coefficient; Third, update the target feeding speed according to the second weight, the third weight, the second speed adjustment coefficient, and the third speed adjustment coefficient; Fourth, update the target feeding speed according to the first weight, the second weight, the third weight, the first speed adjustment coefficient, the second speed adjustment coefficient, and the third speed adjustment coefficient.

[0060] It can be understood that the aggregate area ratio P and the aggregate distribution density D play an important role in judging the feeding situation. Therefore, the two can be combined to calculate the target feeding speed. For the adjustment of the feeding speed, the aggregate area ratio P and the aggregate distribution density D can have different influence degrees respectively. Let the first weight corresponding to the aggregate area ratio be W p , the second weight corresponding to the aggregate distribution density be W d , and W p +W p =1. First, calculate the second speed adjustment coefficient k p according to the area ratio, and the second speed adjustment coefficient k p is calculated according to the following formula (3):

[0061]

[0062] Where k prefers to the second speed adjustment coefficient, P refers to the first statistical value of the aggregate distribution density, P max refers to the first preset proportion threshold, P min refers to the second preset proportion threshold.

[0063] Specifically, when P ≤ P min , the blanking situation is judged according to the area proportion. At this time, the blanking is relatively sparse, k p = 1; when P min < P < P max , it is within the normal blanking range but has a tendency to develop towards densification. When P ≥ P max (when the area proportion exceeds a certain value, the blanking is relatively dense), kp = 0.

[0064] Then, calculate the first speed adjustment coefficient k d according to the aggregate distribution density. The first speed adjustment coefficient k d is calculated according to the following formula (4):

[0065]

[0066] Among them, k d refers to the first speed adjustment coefficient, D refers to the first statistical value of the aggregate area proportion, D max refers to the first preset proportion threshold, D min refers to the second preset proportion threshold.

[0067] When D ≤ D min , the aggregate distribution density is low and the blanking is not dense, k d = 1; when D min < D < D max , the distribution density has an increasing trend. When D ≥ D max , the distribution density is large and the blanking may be dense, k d = 0. The comprehensive speed adjustment coefficient k = W p × k p + W d × k d . Then, the blanking speed V = k × Vdefault, where Vdefault is the default blanking speed.

[0068] The movement speed S of the aggregate is also one of the important characteristics for judging the blanking density. Therefore, on the basis of combining the aggregate area proportion and the aggregate distribution density mentioned above, adding the influence of the aggregate movement speed can calculate the blanking speed more comprehensively. Let the third weight corresponding to the aggregate movement speed be W s , and W p + W p + W s = 1. Calculate the third speed adjustment coefficient k according to the aggregate movement speeds , the third speed adjustment coefficient k s is calculated according to the following formula (5):

[0069]

[0070] wherein, k S refers to the third speed adjustment coefficient, S refers to the first statistical value of the aggregate movement speed, S max refers to the first preset speed threshold, S min refers to the second preset speed threshold.

[0071] When S ≤ S min , the aggregate movement speed is slow, the material discharge is not dense, k s = 1. When S min < S < S max , the movement speed has an accelerating trend. When S ≥ S max , the movement speed exceeds a certain value and problems are likely to occur in the material discharge, ks = 0. For the comprehensive speed adjustment coefficient, k = W p × k p + W d × k d + W s × k s , then the material discharge speed V = k × Vdefault. In the actual application scenario, each threshold value (such as P min , P max , D min , D max , S min , S max , etc.) and weights (W p , W d , W s ) in the above formula need to be accurately determined based on the actual production situation of the mixing plant through a large number of experimental tests and data statistical analyses, so as to achieve precise control of the material discharge speed and ensure the high efficiency and stability of the mixing plant production.

[0072] In an embodiment of the present application, determining the aggregate area of each blanking image includes: processing each blanking image to obtain a processed blanking image; inputting each processed blanking image into a target detection model, and determining a high-level semantic feature map and a low-level detail feature map corresponding to each processed blanking image through the target detection model, where the high-level semantic feature map includes aggregate type information of each processed blanking image, and the low-level detail feature map includes aggregate edge information and aggregate texture information of each processed blanking image; performing upsampling processing on the high-level semantic feature map of each processed blanking image to obtain a first target feature map with the same image size as the low-level detail feature map corresponding to each processed blanking image; performing convolution processing on the low-level detail feature map corresponding to each processed blanking image to obtain a second target feature map with the same number of channels as the high-level semantic feature map corresponding to each processed blanking image; performing fusion processing on the first target feature map and the second target feature map to obtain a third target feature map corresponding to each processed blanking image; processing the third target feature map corresponding to each processed blanking image to output the aggregate area of each processed blanking image through the target detection model.

[0073] Specifically, the video captured by the camera can be extracted frame by frame to obtain a series of image frames, that is, multiple blanking images. Denoising processing can be performed on each frame of the image to reduce noise interference in the image, and methods such as median filtering and Gaussian filtering can be used. Then, image enhancement is performed to improve the contrast and clarity of the image for subsequent analysis, such as using techniques like histogram equalization. Further, a target detection model in the target detection algorithm, such as a model in the YOLO series, is used to identify the position and range of the aggregate in the image. Specifically, during the training process, a large amount of image data containing the aggregate blanking scenario of the mixing plant can be collected. The aggregate area in the image is labeled, and the position and category information of the aggregate area are marked. According to the computing resources and accuracy requirements, a suitable YOLO version, such as YOLOv5, YOLOv7, etc., is selected. The required deep learning framework, such as PyTorch, is installed. The dependency libraries and tools corresponding to the selected YOLO version are installed. The labeled data is divided into a training set, a validation set, and a test set according to a certain ratio. The training parameters of the model, such as the learning rate, the number of training epochs, the batch size, etc., are configured. The model is trained on the training set and evaluated on the validation set, and the parameters are adjusted according to the evaluation results.

[0074] Among them, the YOLO algorithm adopts a relatively simple network structure with a reasonable design of the number of network layers, which can effectively extract image features while ensuring computational efficiency. For example, the network structure of YOLOv5 contains multiple convolutional layers and pooling layers, and feature extraction is performed through convolutional kernels of different scales. Shallow convolutional layers can capture detailed information in the image, such as the edges and textures of aggregates; deep convolutional layers can learn more abstract semantic features, which helps to distinguish different types of aggregates. This hierarchical feature extraction method enables the YOLO algorithm to comprehensively understand the image content when processing the aggregate images in the mixing plant, improving the recognition ability of aggregates. The YOLO algorithm divides the input image into multiple grids, and each grid is responsible for predicting the targets whose centers fall within that grid. This grid division method has good adaptability to the detection of aggregates of different sizes in the mixing plant. In the mixing plant, the sizes of aggregates vary, from fine sand grains to larger stones. The grid division of YOLO can detect aggregates of different sizes in grids of different scales according to the actual distribution of aggregates. For example, smaller sand grains may be detected in smaller grids, while larger stones will be recognized in larger grids. In this way, YOLO can accurately locate and identify aggregates of various sizes, improving the accuracy and comprehensiveness of detection.

[0075] In a specific embodiment, the OpenCV library is used for image processing operations (such as cv2.medianBlur for median filter denoising and cv2.equalizeHist for histogram equalization to enhance the image), and the TensorFlow deep learning framework is selected (its computational graph execution mode is efficient, and it can flexibly construct network structures and support various optimization algorithms). New aggregate feeding images are regularly collected every week to form a test data set. The judgment results of the algorithm are compared with the true results of manual annotation, and indicators such as accuracy and recall are calculated. The performance of the algorithm is analyzed according to the changes in the indicators, and the corresponding parameters are adjusted. For example, the confidence threshold of the object detection model and the parameters related to the feature extraction algorithm are used to randomly crop the original data set (the cropping ratio is 0.5 - 0.8), rotate (-10° to 10°), flip (horizontal and vertical), scale (0.8 - 1.2), and perform data augmentation operations such as color transformation (brightness -30 to 30, contrast and saturation 0.8 - 1.2). For multi-scale training of the YOLO model, three input image sizes of 320×320, 416×416, and 608×608 are selected, appropriate stride and padding parameters are set, and the weights of the prediction results at different scales are calculated to balance the model performance. The high-level semantic feature map and low-level detail feature map of the YOLO model are fused (the high-level feature map is upsampled and then added to the corresponding elements of the low-level feature map. The number of channels can be adjusted by a 1×1 convolution before addition), providing a richer feature representation for object detection and feature extraction. Figure 1 ×1 convolution to adjust the number of channels), providing a richer feature representation for object detection and feature extraction.

[0076] Furthermore, a spatial attention mechanism is adopted to calculate the attention weights at each spatial position on the feature map. The specific calculation method is as follows: First, average pooling and max pooling operations are performed on the feature map in the channel dimension to obtain two two-dimensional feature descriptions. Then, these two feature descriptions are processed through a shared convolutional layer and then passed through an activation function (such as the sigmoid function) to obtain the attention weight map. Multiply the attention weight map by the original feature map to make the model pay more attention to the area where the aggregates are located. At the same time, a channel attention mechanism is used to calculate the attention weights of each channel. First, average pooling and max pooling operations are performed on the feature map in the spatial dimension to obtain two one-dimensional feature descriptions, which are processed through a multi-layer perceptron (MLP), and finally, the channel attention weights are obtained through an activation function. Multiply the channel attention weights by the original feature map in the channel dimension to enable the model to focus on important feature channels, improve the ability to extract aggregate features, and thus enhance the algorithm performance.

[0077] Moreover, three YOLO models with different structures or parameter initializations (such as YOLOv5small, YOLOv5medium, and YOLOv5 large) can be trained. During the testing phase, the prediction results are weighted and averaged for fusion (weights are assigned according to the performance of the models on the validation set) to comprehensively utilize the advantages of multiple models to improve the accuracy and stability of the algorithm. FocalLoss is used to solve the problem of unbalanced positive and negative samples. The loss is calculated based on the predicted confidence of the samples and the true labels, and the model parameters are updated through backpropagation. The initial learning rate is set to 0.001, and an exponential decay strategy is adopted (multiplied by 0.9 every 5 training epochs). L2 regularization is selected, and the regularization coefficient is tried starting from 0.0001. By evaluating the model performance on the validation set, observe the effect of the regularization coefficient on preventing overfitting of the model. If it is found that the model shows overfitting on the validation set (such as the accuracy no longer improving and the loss value starting to rise, etc.), appropriately increase the regularization coefficient; if the model is underfitting (such as low accuracy and large loss value), then appropriately decrease the regularization coefficient to find the optimal regularization parameter value and improve the generalization ability of the model.

[0078] Furthermore, the K-Means++ algorithm is used to optimize the size and ratio of the prior boxes. First, randomly select K initial clustering centers from the aggregate feeding images (the value of K is set according to experience, such as 5 or 9, etc.). Calculate the distance between each aggregate area and these clustering centers (the distance metric can use the intersection over union IoU). Assign each aggregate area to the cluster where the nearest clustering center belongs. Then update the clustering centers, calculate the average size and ratio of the aggregate areas in each cluster as the new clustering centers. Repeat the above assignment and update steps until the clustering centers no longer change significantly. The size and ratio of the prior boxes optimized by the K-Means++ algorithm are more in line with the actual size distribution of the aggregates, enabling the YOLO model to more accurately predict the position and size of the aggregates during object detection and improving the accuracy of the algorithm. Collect more aggregate feeding image data at different times and working conditions, use data augmentation techniques to expand the data, and cooperate with other mixing plants to obtain more scenario data to improve the generalization ability of the model.

[0079] In addition to accuracy, recall, and precision, the F1-Score metric is introduced. It is the harmonic mean of precision and recall and can comprehensively reflect the balanced performance of the algorithm between the two. Calculate the F1-Score according to the following formula (6):

[0080]

[0081] Among them, F1 is an index used in statistics to measure the accuracy of a binary classification model. It takes into account both the precision and recall of the classification model. Precision refers to precision, and Recall refers to recall.

[0082] When evaluating the algorithm, also pay attention to the change of the F1-Score to avoid the situation where only the accuracy is optimized while the recall drops significantly, and ensure that the algorithm achieves a good balance between accurately detecting the aggregate feeding situation and detecting all situations that should be detected as much as possible.

[0083] Calculate the mean intersection over union (mIoU) to measure the overlapping degree between the aggregate areas predicted by the algorithm and the real aggregate areas. For each predicted aggregate area and the corresponding real aggregate area, calculate their intersection over union (IoU = the intersection area of the predicted area and the real area / the union area of the predicted area and the real area), and then take the average of the IoUs of all aggregate areas to get the mIoU. The higher the mIoU, the more accurate the algorithm is in positioning and segmenting the aggregate areas, which helps to evaluate the accuracy of the algorithm in dealing with the shape and position judgment of the aggregates.

[0084] The confusion matrix is used to analyze the prediction results of the algorithm in detail. The confusion matrix shows the situations where the algorithm predicts different categories of aggregates (such as stones and sands of different particle sizes) as other categories. By observing the confusion matrix, it is possible to clearly understand in which categories the algorithm is prone to misjudgment, and thus improve the algorithm targeted, for example, for easily confused aggregate categories, adjusting the feature extraction method or retraining the model to improve the discrimination ability.

[0085] Abnormal data processing strategy: During the data collection process, establish a data screening mechanism to identify and exclude abnormal data. For example, the image data collected when the camera has a short-term failure or is strongly interfered with may be abnormal (such as severely blurred images, sudden brightness changes, etc.). By setting image quality evaluation indicators (such as clarity threshold, brightness change range threshold, etc.), these abnormal data can be automatically screened out and not used for algorithm training and testing to avoid affecting the algorithm performance.

[0086] For abnormal results that occur during the operation of the algorithm (such as suddenly appearing extremely low or high aggregate feeding density judgment values, etc.), establish an anomaly detection algorithm. A method based on a statistical model can be used, such as calculating the mean and standard deviation of the algorithm output results over a period of time. When a result exceeds a certain multiple of the standard deviation range, it is determined as an abnormal result. Once an abnormal result is detected, record relevant information in a timely manner (such as the time when the anomaly occurred, the corresponding image data, etc.), and analyze the cause of the anomaly. It may be a sudden situation in the production process (such as abnormal feeding caused by equipment failure) or a problem with the algorithm itself (such as sudden changes in model parameters). Take corresponding measures according to the cause, such as rechecking the equipment status, adjusting the algorithm parameters, or retraining the model, etc.

[0087] Model adaptive adjustment: Establish a model adaptive mechanism to enable the algorithm to automatically adjust model parameters according to the actual situation in the production process. For example, when the aggregate supplier in the mixing plant is changed, resulting in changes in the characteristics of the aggregates such as color, shape, or texture, the algorithm can automatically adjust the parameters in the feature extraction algorithm or retrain some layers of the model by real-time monitoring of the new aggregate image data to adapt to the new aggregate characteristics.

[0088] Adopt online learning technology to enable the model to continuously learn new data during the production process. During the normal production process of the mixing plant, at regular time intervals (such as after every certain number of production batches), select a part of the new aggregate feeding image data as incremental data and use this data to perform online training on the model to update the model parameters, so that the model can continuously learn new production patterns and changes in aggregate characteristics and maintain the accuracy and adaptability of the algorithm.

[0089] Environmental monitoring and algorithm linkage: Install environmental monitoring sensors at the mixing plant to continuously monitor environmental parameters such as dust concentration, humidity, light intensity, etc. Integrate the environmental monitoring data with the algorithm operation. When the environmental parameters change (such as a sudden increase in dust concentration), the algorithm automatically adjusts the image preprocessing strategy. For example, increasing the denoising intensity or adjusting the image enhancement parameters to adapt to the impact of environmental changes on image quality and ensure that the algorithm can accurately judge the aggregate feeding situation under different environmental conditions.

[0090] Based on the environmental monitoring data, establish a correlation model between environmental conditions and algorithm performance. By analyzing a large amount of historical data, understand the performance of the algorithm under different environmental conditions. When the environmental parameters are within a specific range, predict in advance the possible performance changes of the algorithm, such as a downward trend in accuracy, and take corresponding preventive measures, such as switching to a backup algorithm model or adjusting algorithm parameters, to ensure stable monitoring of the production process.

[0091] Multi-modal data fusion: In addition to the image data collected by the camera, fuse other types of data, such as the aggregate weight data and belt transmission speed data collected by sensors. Establish a data fusion model to conduct correlation analysis on the aggregate characteristics (such as area ratio, distribution density, etc.) obtained from the image data, the weight data, and the transmission speed data. For example, by establishing a mathematical model, estimate the weight of the aggregate based on its characteristics in the image and compare it with the weight data measured by the sensor for mutual supplementation and correction to improve the accuracy of aggregate measurement.

[0092] Use multi-modal data to make a more comprehensive judgment on the feeding situation. For example, when the image data shows a high density of aggregate feeding, but the weight data measured by the sensor increases slowly, it may indicate problems such as aggregate blockage or metering equipment failure. Combining the information of the two types of data can more accurately judge abnormal situations in the production process, improve the algorithm's ability to handle complex production scenarios, and enhance the accuracy and stability of the algorithm's judgment.

[0093] Manual evaluation: Invite experts with rich experience in the field of mixing plants (such as engineers with many years of mixing plant production management experience, familiar with aggregate characteristics and metering requirements, etc.) to form an evaluation group. Regularly display the judgment results of the algorithm in the actual production process to the evaluation group, including the judgment of the aggregate feeding density, the feature extraction data, and the production suggestions generated based on these results (such as suggestions for adjusting the belt transmission rate, etc.). The experts comprehensively evaluate the judgment results of the algorithm based on their professional knowledge and practical experience. They will consider the rationality of the algorithm in different production scenarios, such as whether the accuracy judgment of aggregate metering by the algorithm under special aggregate ratios and different mixing processes meets the actual production requirements. The experts will also evaluate whether the production suggestions provided by the algorithm are practical and effective, and whether they can truly optimize the production process and improve product quality.

[0094] The evaluation team provides detailed written opinions and improvement suggestions based on the evaluation results. For example, it points out the possible reasons for misjudgment of the algorithm in some complex situations, and proposes directions for improving the feature extraction algorithm or adjusting the judgment threshold. At the same time, experts can share special cases and experiences in actual production to provide practical reference for the optimization of the algorithm. Through manual evaluation, the algorithm can be comprehensively tested from a professional and practical perspective, so that the algorithm can better meet the actual production needs of the mixing plant and improve its value in actual applications.

[0095] The embodiment of the present application further provides an industrial robot, including the device for monitoring the feeding of the mixing plant described above.

[0096] The above solution can provide a solution for optimizing and adjusting the aggregate feeding speed through the technical principles and ideas of embodied intelligence. Specifically, this solution can be deployed on an embodied robot to realize the supervision and control of the feeding of the mixing plant by the robot. The embodied robot can include modules such as intelligent perception and detection, intelligent decision-making and control, adaptation and optimization. Among them, intelligent perception and monitoring: Embodied intelligence emphasizes obtaining environmental information through the perception modules of intelligent agents (such as vision, hearing, etc.). During the aggregate production process, similar intelligent perception technologies can be introduced to real-time monitor the feeding speed, flow rate and other related parameters of the aggregate. This real-time monitoring can provide data support for subsequent control decisions. Intelligent decision-making and control: The embodied intelligence system can make decisions based on the perceived information and execute corresponding actions. For example, after obtaining the required adjusted feeding speed according to this solution, the robot controls itself to move to the feeding control unit of the mixing plant, operates the control unit and adjusts it to the required feeding speed. Adaptation and optimization: The embodied intelligence system can adapt to the changing environment through empirical learning and optimize its behavior. During the aggregate feeding process, the embodied intelligence system can automatically adjust the feeding speed according to the real-time monitored data to ensure the stability and efficiency of production. With the participation of the embodied robot and the embodied intelligence system relying on the ability of intelligent decision-making and control, it helps to improve the automation and intelligence level of aggregate production. During the aggregate production process, the embodied intelligence system can continuously learn and adapt to different production conditions, so as to continuously optimize parameters such as the feeding speed.

[0097] The above technical solution can significantly reduce aggregate waste and lower production costs (such as reducing the aggregate waste rate from 5% to 2%, saving a large amount of costs annually). Improve the quality of concrete production, making the actual value closer to the target value (reducing the customer complaint rate by 70%). Optimize the production process of the mixing plant and improve production efficiency (an average increase of 30%). Reduce the error between the actual value and the target value of concrete (an average reduction of 40%).

[0098] Figure 1It is a schematic flowchart of a method for monitoring the discharging of a mixing plant in an embodiment. It should be understood that although Figure 1 the steps in the flowchart are shown sequentially according to the indication of the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, Figure 1 at least a part of the steps in

[0099] Figure 2 may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps. Figure 2 As shown in

[0100] a memory, configured to store instructions;

[0101] a processor, configured to call instructions from the memory and be able to implement the above-mentioned method for monitoring the discharging of a mixing plant when executing the instructions.

[0102] Specifically, in the embodiment of the present application, the processor may be configured to:

[0103] acquire multiple discharging images of the mixing plant;

[0104] determine the aggregate area of each discharging image;

[0105] determine at least one feature of each aggregate area, where the at least one feature includes at least one of the aggregate area ratio, aggregate distribution density, and aggregate movement speed;

[0106] determine the discharging density of the mixing plant according to the at least one feature corresponding to all aggregate areas;

[0107] adjust the discharging speed of the mixing plant according to the discharging density.

[0108] In the embodiment of the present application, the processor may further be configured to:

[0109] determining at least one feature of each aggregate area includes at least one of the following: determining the aggregate area ratio corresponding to each aggregate area; determining the aggregate distribution density corresponding to each discharging image according to the pixel values of each pixel point included in each aggregate area; determining the aggregate movement speed corresponding to each discharging image.

[0110] In an embodiment of the present application, the processor may further be configured to:

[0111] Determine the feeding density of the mixing station according to at least one feature corresponding to all aggregate areas, including: determining a first statistical value of the aggregate area ratio corresponding to all aggregate areas, a second statistical value of the aggregate distribution density corresponding to all aggregate areas, and a third statistical value of the aggregate movement speed corresponding to all aggregate areas; determine that the feeding density is dense when any two of the following conditions are met: 1. The first statistical value is greater than or equal to a first preset ratio threshold; 2. The second statistical value is greater than or equal to a first preset density threshold; 3. The third statistical value is greater than or equal to a first preset speed threshold; determine that the feeding density is sparse when any two of the following conditions are met: 4. The first statistical value is less than or equal to a second preset ratio threshold; 5. The second statistical value is less than or equal to a second preset density threshold; 6. The third statistical value is less than or equal to a second preset speed threshold; determine that the feeding density is normal when none of the above conditions are met; wherein, the first preset ratio threshold is greater than the second preset ratio threshold, the first preset density threshold is greater than the second preset density threshold, and the first preset speed threshold is greater than the second preset speed threshold.

[0112] In an embodiment of the present application, the processor may further be configured to:

[0113] Adjust the feeding speed of the mixing station according to the feeding density, including: when the feeding density is normal, determine the target feeding speed of the mixing station according to the first statistical value, the first preset density threshold, the second preset density threshold, the first feeding speed, and the second feeding speed, so as to adjust the feeding speed to the target feeding speed; wherein, the first feeding speed is less than the second feeding speed, the first feeding speed refers to the minimum feeding speed of the mixing station, and the second feeding speed is the maximum feeding speed of the mixing station.

[0114] In an embodiment of the present application, the processor may further be configured to:

[0115] Determine a first weight corresponding to the aggregate area ratio, a second weight corresponding to the aggregate distribution density, and a third weight corresponding to the aggregate movement speed; determine a first speed adjustment coefficient corresponding to the feeding speed according to the first statistical value, the first preset ratio threshold, and the second ratio threshold; determine a second speed adjustment coefficient corresponding to the feeding speed according to the second statistical value, the first preset density threshold, and the second preset density threshold; determine a third speed adjustment coefficient corresponding to the feeding speed according to the third statistical value, the first preset speed threshold, and the second preset speed threshold; when the feeding density is normal, update the target feeding speed in any of the following ways to adjust the feeding speed to the updated target feeding speed: 1. Update the target feeding speed according to the first weight, the second weight, the first speed adjustment coefficient, and the second speed adjustment coefficient; 2. Update the target feeding speed according to the first weight, the third weight, the first speed adjustment coefficient, and the third speed adjustment coefficient; 3. Update the target feeding speed according to the second weight, the third weight, the second speed adjustment coefficient, and the third speed adjustment coefficient; 4. Update the target feeding speed according to the first weight, the second weight, the third weight, the first speed adjustment coefficient, the second speed adjustment coefficient, and the third speed adjustment coefficient.

[0116] In an embodiment of the present application, the processor may further be configured to:

[0117] Determining the aggregate distribution density corresponding to each feeding image according to the pixel value of each pixel point included in each aggregate area includes: determining whether each pixel point is a target pixel point according to the pixel value of each pixel point in each aggregate area, where the target pixel point refers to a pixel point where an aggregate is detected; determining the aggregate distribution density of each feeding image according to the number of all target pixel points included in each aggregate area.

[0118] In an embodiment of the present application, the processor may further be configured to:

[0119] Determining the aggregate distribution density corresponding to each feeding image according to the pixel value of each pixel point included in each aggregate area includes: determining the variance of the gray value of each aggregate area by determining the pixel values of all pixel points in each aggregate area; determining the aggregate distribution density of each feeding image according to the variance of the gray value of each aggregate area.

[0120] In an embodiment of the present application, the processor may further be configured to:

[0121] Determining the aggregate movement speed corresponding to each blanking image includes: determining the target aggregate in each aggregate area, and determining the first aggregate position and aggregate characteristics of the target aggregate; determining the target aggregate areas where the target aggregate exists in other aggregate areas according to the aggregate characteristics; determining the second aggregate position of the target aggregate in the target aggregate areas; and determining the aggregate movement speed corresponding to each blanking image according to the first aggregate position and the second aggregate position.

[0122] In an embodiment of the present application, the processor may also be configured to:

[0123] Determining the aggregate areas of each blanking image includes: processing each blanking image to obtain a processed blanking image; inputting each processed blanking image into a target detection model, and determining a high-level semantic feature map and a low-level detail feature map corresponding to each processed blanking image through the target detection model, where the high-level semantic feature map includes aggregate type information of each processed blanking image, and the low-level detail feature map includes aggregate edge information and aggregate texture information of each processed blanking image; performing upsampling processing on the high-level semantic feature map of each processed blanking image to obtain a first target feature map with the same image size as the low-level detail feature map corresponding to each processed blanking image; performing convolution processing on the low-level detail feature map corresponding to each processed blanking image to obtain a second target feature map with the same number of channels as the high-level semantic feature map corresponding to each processed blanking image; fusing the first target feature map and the second target feature map to obtain a third target feature map corresponding to each processed blanking image; and processing the third target feature map corresponding to each processed blanking image to output the aggregate area of each processed blanking image through the target detection model.

[0124] In an embodiment of the present application, the processor may also be configured to:

[0125] Determining the blanking density of the mixing plant according to at least one feature corresponding to all aggregate areas includes: determining a first statistical value of the proportion of the aggregate area corresponding to all aggregate areas; and determining that the blanking density during the time period is dense when the first statistical value is greater than or equal to a first preset proportion threshold.

[0126] An embodiment of the present application also provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause the machine to execute the above method for monitoring the blanking of the mixing plant.

[0127] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as Figure 3As shown in the figure. The computer device includes a processor A01, a network interface A02, a memory (not shown in the figure), and a database (not shown in the figure) connected through a system bus. Among them, the processor A01 of the computer device is used to provide computing and control capabilities. The memory of the computer device includes an internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown in the figure). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 in the non-volatile storage medium A04. The database of the computer device is used to store data for the method of monitoring the discharging of the mixing station. The network interface A02 of the computer device is used to communicate with an external terminal through a network connection. When the computer program B02 is executed by the processor A01, it implements a method for monitoring the discharging of the mixing station.

[0128] Those skilled in the art can understand that Figure 3 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0129] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0131] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction means that implements the function specified in one or more of the processes and / or blocks Figure 1 in one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the function specified in one or more of the processes and / or blocks Figure 1 in one or more of the processes and / or blocks Figure 1 specified in one or more of the blocks or blocks.

[0133] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0134] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0135] Computer-readable media includes permanent and non-permanent, removable and non-removable media implemented by any method or technology for storing information. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0136] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0137] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included within the scope of the claims of the present application.

Claims

1. A method for monitoring the discharging of a mixing plant, characterized in that The method includes: Obtaining multiple blanking images of the mixing plant; Determining the aggregate area of each blanking image; Determining at least one feature of each aggregate area, where the at least one feature includes at least one of the aggregate area ratio, aggregate distribution density, and aggregate movement speed; Determining the blanking density of the mixing plant according to the at least one feature corresponding to all aggregate areas; Adjusting the blanking speed of the mixing plant according to the blanking density.

2. The method for monitoring the discharging of a mixing plant according to claim 1, characterized in that, The determining of at least one feature of each aggregate area includes at least one of the following: Determining the aggregate area ratio corresponding to each aggregate area; Determining the aggregate distribution density corresponding to each blanking image according to the pixel values of each pixel point included in each aggregate area; Determining the aggregate movement speed corresponding to each blanking image.

3. The method for monitoring the material discharging of a mixing plant according to claim 1, characterized in that The determining of the blanking density of the mixing plant according to the at least one feature corresponding to all aggregate areas includes: Determining a first statistical value of the aggregate area ratio corresponding to all aggregate areas, a second statistical value of the aggregate distribution density corresponding to all aggregate areas, and a third statistical value of the aggregate movement speed corresponding to all aggregate areas; When any two of the following conditions are met, determining that the blanking density is dense: One, the first statistical value is greater than or equal to a first preset ratio threshold; Two, the second statistical value is greater than or equal to a first preset density threshold; Three, the third statistical value is greater than or equal to a first preset speed threshold; When any two of the following conditions are met, determining that the blanking density is sparse: Four, the first statistical value is less than or equal to a second preset ratio threshold; Five, the second statistical value is less than or equal to a second preset density threshold; Six, the third statistical value is less than or equal to a second preset speed threshold; When all of the above conditions are not met, determining that the blanking density is normal; Wherein, the first preset ratio threshold is greater than the second preset ratio threshold, the first preset density threshold is greater than the second preset density threshold, and the first preset speed threshold is greater than the second preset speed threshold.

4. The method for monitoring the discharging of the mixing plant according to claim 3, characterized in that, The adjusting of the blanking speed of the mixing plant according to the blanking density includes: When the blanking density is normal, determining the target blanking speed of the mixing plant according to the first statistical value, the first preset density threshold, the second preset density threshold, the first blanking speed, and the second blanking speed, so as to adjust the blanking speed to the target blanking speed; Wherein, the first blanking speed is less than the second blanking speed, the first blanking speed refers to the minimum blanking speed of the mixing plant, and the second blanking speed is the maximum blanking speed of the mixing plant.

5. The method for monitoring the discharging of a mixing plant according to claim 4, wherein The adjusting of the blanking speed of the mixing plant according to the blanking density further includes: Determining a first weight corresponding to the aggregate area ratio, a second weight corresponding to the aggregate distribution density, and a third weight corresponding to the aggregate movement speed; Determining a first speed adjustment coefficient corresponding to the blanking speed according to the first statistical value, the first preset ratio threshold, and the second preset ratio threshold; Determine a second speed adjustment coefficient corresponding to the blanking speed according to the second statistical value, the first preset density threshold, and the second preset density threshold; Determine a third speed adjustment coefficient corresponding to the blanking speed according to the third statistical value, the first preset speed threshold, and the second preset speed threshold; When the blanking density is normal, update the target blanking speed in any of the following ways to adjust the blanking speed to the updated target blanking speed:

1. Update the target blanking speed according to the first weight, the second weight, the first speed adjustment coefficient, and the second speed adjustment coefficient; 2. Update the target blanking speed according to the first weight, the third weight, the first speed adjustment coefficient, and the third speed adjustment coefficient; 3. Update the target blanking speed according to the second weight, the third weight, the second speed adjustment coefficient, and the third speed adjustment coefficient; 4. Update the target blanking speed according to the first weight, the second weight, the third weight, the first speed adjustment coefficient, the second speed adjustment coefficient, and the third speed adjustment coefficient.

6. The method for monitoring the discharging of the mixing plant according to claim 2, wherein, The determining the aggregate distribution density corresponding to each blanking image according to the pixel values of each pixel point included in each aggregate area includes: Determine the gray value variance of each aggregate area by determining the pixel values of all pixel points in each aggregate area; Determine the aggregate distribution density of each blanking image according to the gray value variance of each aggregate area.

7. The method for monitoring the discharging of the mixing plant according to claim 2, wherein The determining the aggregate movement speed corresponding to each blanking image includes: Determine the target aggregate in each aggregate area and determine the first aggregate position and aggregate characteristics of the target aggregate; Determine the target aggregate area where the target aggregate exists in other aggregate areas according to the aggregate characteristics; Determine the second aggregate position of the target aggregate in the aggregate area; Determine the aggregate movement speed corresponding to each blanking image according to the first aggregate position and the second aggregate position.

8. The method for monitoring the discharging of a mixing plant according to claim 1, characterized in that The determining the aggregate area of each blanking image includes: Process each blanking image to obtain a processed blanking image; Input each processed blanking image into a target detection model, and determine a high-level semantic feature map and a low-level detail feature map corresponding to each processed blanking image through the target detection model, where the high-level semantic feature map includes the aggregate type information of each processed blanking image, and the low-level detail feature map includes the aggregate edge information and aggregate texture information of each processed blanking image; Perform upsampling processing on the high-level semantic feature map of each processed blanking image to obtain a first target feature map with the same image size as the low-level detail feature map corresponding to each processed blanking image; Perform convolution processing on the low-level detail feature map corresponding to each processed blanking image to obtain a second target feature map with the same number of channels as the high-level semantic feature map corresponding to each processed blanking image; Fuse the first target feature map and the second target feature map to obtain a third target feature map corresponding to each processed blanking image; Process the third target feature map corresponding to each processed blanking image, so as to output the aggregate area of each processed blanking image through the target detection model.

9. A device for monitoring the material discharging of a mixing plant, characterized in that, Comprising: A memory configured to store instructions; A processor configured to call the instructions from the memory and capable of implementing the method for monitoring the blanking of a mixing plant according to any one of claims 1 to 8 when executing the instructions.

10. An industrial robot, characterized in that, Comprising the device for monitoring the blanking of a mixing plant according to claim 9.

11. A machine-readable storage medium, characterized in that, Instructions are stored on the machine-readable storage medium, and the instructions are used to cause the machine to execute the method for monitoring the blanking of a mixing plant according to any one of claims 1 to 8.

Images