Bucket wheel machine control method and system based on image monitoring

Abstract

The invention relates to a bucket wheel machine control method and system based on image monitoring. The method comprises the following steps that first image data located in a bucket wheel machine conveying path area are collected through an image collection unit arranged on a bucket wheel machine head platform; inputting the collected first image data into a pre-trained first network model, and obtaining a target recognition frame of a current bucket wheel machine conveying material pile; based on the target recognition frame, segmenting second image data in the range of the target recognition frame from the first image data, preprocessing the second image data, inputting the preprocessed data into a pre-trained second network model, and obtaining material category information and material particle size information of materials conveyed by the current bucket wheel machine; and determining a currently started bucket wheel machine control program based on the obtained material category information and the material particle size information.

Description

Technical Field

[0001] The present invention relates to a bucket wheel machine control method and system based on image monitoring, belonging to the technical field of industrial equipment automation. Background Art

[0002] Bucket wheel excavators are large-scale bulk material handling equipment, widely used in material loading, unloading, and stacking operations in ports, power plants, mines, and other fields. With the advancement of industrial automation, bucket wheel excavator control systems have also continued to evolve, from initial manual operation to today's automated control systems.

[0003] With the development of computer vision technology, the application of image monitoring in bucket wheel machine control is becoming more and more widespread. For example, the Chinese invention patent with publication number CN119888595A discloses an intelligent control system and method for bucket wheel machine based on digital image monitoring. The system processes and analyzes the acquired image of the bucket wheel machine working area, extracts the working area status features, and generates semantic segmentation results based on the working area status feature map highlighting the material target object, thereby achieving tracking of the material target object and adjustment of the bucket wheel machine travel angle.

[0004] However, the existing bucket wheel excavator control system still has some shortcomings. First, the existing bucket wheel excavator control is mainly based on manual control or fixed automation programs, and fails to make intelligent adjustments according to the type and size of the material pile. For different types of materials, such as block materials and granular materials, the processing methods should be different, but the existing technology has not effectively solved this problem. Secondly, when processing block materials, if the travel speed is not well controlled, it may cause the material to fall out of the bucket wheel excavator, causing personnel safety hazards or equipment damage. Similarly, for granular materials, inappropriate travel speed may cause the material to scatter, affecting work efficiency and possibly damaging the equipment. In addition, when the material pile is too large, it may collide with the head platform of the bucket wheel excavator, causing damage to the bucket wheel excavator, but the existing technology lacks an effective collision warning mechanism. Summary of the Invention

[0005] In order to solve the above problems in the prior art, the present invention proposes a bucket wheel excavator control method and system based on image monitoring.

[0006] The technical solutions of the present invention are as follows:

[0007] In one aspect, the present invention provides a bucket wheel excavator control method based on image monitoring, comprising the following steps:

[0008] The image acquisition unit provided on the head platform of the bucket wheel machine collects first image data located in the conveying path area of ​​the bucket wheel machine;

[0009] Inputting the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator;

[0010] Segmenting second image data within the target identification frame from the first image data based on the target identification frame, preprocessing the second image data, and inputting the preprocessed data into a pre-trained second network model to obtain material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator;

[0011] The currently enabled bucket wheel excavator control program is determined based on the acquired material category information and material particle size information.

[0012] As a preferred embodiment, the first network model includes:

[0013] A convolutional layer, configured to extract features from the input first image data and output a feature map;

[0014] The candidate region selection network outputs several candidate region data based on the feature map output by the convolutional layer;

[0015] The pooling layer is used to pool the data of each candidate region and output the corresponding feature vector;

[0016] The region classifier passes the feature vector output by the pooling layer through the fully connected layer and the softmax layer to calculate the probability of whether the corresponding candidate region belongs to the material pile;

[0017] The output layer outputs the target recognition box of the material pile based on the candidate region classification results output by the region classifier.

[0018] As a preferred embodiment, the step of preprocessing the second image data includes:

[0019] performing grayscale conversion on the second image data to generate a grayscale image;

[0020] Perform Gaussian filtering on the grayscale image;

[0021] Randomly select a region in the grayscale image after Gaussian filtering, perform image magnification processing, and obtain an amplified filtered image;

[0022] For the magnified filtered image, the outline of the material particles is extracted using the edge detection algorithm;

[0023] The material particle contour is fitted by an ellipse fitting algorithm, and the ellipse fitting result outputs the ellipse fitting parameters as preprocessed data.

[0024] As a preferred embodiment, the second network model is constructed based on an artificial neural network, and its pre-training process includes:

[0025] Establish a material sample set. Each sample in the material sample set includes the ellipse fitting parameters of the corresponding material contour, the material category label, and the material size label. The material size label includes three types: standard size, below standard size, and above standard size.

[0026] The material sample set is input into the second network model, and the ellipse fitting parameters of the material contour corresponding to each sample are input. The material category and material size prediction results output by the second network model are obtained, and the loss is calculated by comparing with the material category label and material size label of the corresponding sample. The second network model is back-propagated and trained based on the calculated loss.

[0027] As a preferred embodiment, the method further includes a collision warning step, specifically comprising:

[0028] Set the material pile size threshold;

[0029] Confirm the size of the material pile according to the target identification frame of the current bucket wheel conveyor material pile;

[0030] The size of the material pile is compared with a material pile size threshold. When the size of the material pile is greater than the material pile size threshold, a warning message indicating a collision risk is output.

[0031] As a preferred embodiment, it also includes:

[0032] When the host computer receives a warning message indicating a collision risk, it sends a control instruction to the bucket wheel machine control program to suspend the operation of the bucket wheel machine;

[0033] And send control instructions to the sound and light alarm unit to issue sound and light alarm.

[0034] As a preferred embodiment, the step of determining the currently enabled bucket wheel excavator control program based on the acquired material category information and material particle size information includes:

[0035] A program library is pre-set, wherein the program library contains bucket wheel excavator control programs corresponding to different material categories and different material particle sizes;

[0036] When the material category information and material particle size information of the current bucket wheel machine conveying are obtained, the corresponding bucket wheel machine control program is matched in the program library and started.

[0037] On the other hand, the present invention also proposes a bucket wheel excavator control system based on image monitoring, comprising:

[0038] An image acquisition unit, provided on the head platform of the bucket wheel machine, for acquiring first image data located in a conveying path area of ​​the bucket wheel machine;

[0039] A material pile recognition unit is used to input the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator;

[0040] The material particle recognition unit segments the second image data within the target recognition frame from the first image data based on the target recognition frame, preprocesses the second image data, inputs the preprocessed data into the pre-trained second network model, and obtains the material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator;

[0041] The bucket wheel machine control unit determines the currently enabled bucket wheel machine control program based on the acquired material category information and material particle size information.

[0042] On the other hand, the present invention further proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method described in any embodiment of the present invention when executing the program.

[0043] In another aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the method described in any embodiment of the present invention when the program is executed by a processor.

[0044] The beneficial effects of the present invention are: identifying the type and size of materials through image monitoring technology, realizing intelligent control of bucket wheel excavators, overcoming the defect of not considering material properties in the prior art; automatically selecting appropriate control programs according to material properties, avoiding the problem of block material separation or scattering of granular materials due to poor travel speed control, and improving the safety of equipment operation; timely discovering and warning of collision risks caused by excessive material pile size through the collision warning function, preventing equipment damage caused by collision between the material pile and the head platform of the bucket wheel excavator; compared with the prior art, the present invention improves the safety and efficiency of bucket wheel excavator operation, and solves the safety hazards and equipment damage problems caused by not considering material properties in the prior art.

[0045] Additional aspects and advantages of the present invention will be set forth in the following description, and some of them will be obvious from the description, or may be learned by practicing the present invention. In addition, the various aspects and advantages of the present invention may be realized and obtained by the method steps and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 For the present invention. DETAILED DESCRIPTION

[0047] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] It should be understood that the step numbers used herein are only for convenience of description and are not intended to limit the order in which the steps are to be executed.

[0049] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0050] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0051] The term "and / or" refers to and includes any and all possible combinations of one or more of the associated listed items.

[0052] Example 1:

[0053] See also Figure 1 This embodiment proposes a bucket wheel excavator control method based on image monitoring, comprising the following steps:

[0054] S100, collecting first image data located in a conveying path area of ​​the bucket wheel machine by an image acquisition unit provided on a head platform of the bucket wheel machine;

[0055] S200: Inputting the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator;

[0056] S300, segmenting second image data within the target recognition frame from the first image data based on the target recognition frame, preprocessing the second image data, inputting the preprocessed data into a pre-trained second network model, and obtaining material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator;

[0057] S400: Determine the currently enabled bucket wheel excavator control program based on the acquired material category information and material particle size information.

[0058] In this embodiment, a bucket wheel excavator is a large, continuously operating loading and unloading device primarily used for bulk material handling in locations such as ports, power plants, and mines. During operation, the bucket wheel excavator must adjust its operating parameters based on the material type and properties to ensure operational efficiency and equipment safety. Traditional bucket wheel excavator control methods rely primarily on manual experience to determine material type and properties, making precise control difficult and posing safety risks.

[0059] The bucket wheel excavator control method based on image monitoring provided in this embodiment collects image data of the bucket wheel excavator's conveying path area in real time by installing an image acquisition unit on the bucket wheel excavator's head platform, and uses deep learning technology to analyze and process the collected images to automatically identify material types and material characteristics, thereby realizing intelligent control of the bucket wheel excavator.

[0060] Specifically, the image acquisition unit includes a high-definition camera, mounted appropriately on the bucket wheel excavator's head platform, ensuring clear capture of the material pile within the excavator's conveying path. The camera boasts a 4K resolution (3840×2160 pixels), a frame rate of 30fps, and a 120° field of view, covering the entire material conveying area. The camera is secured to the excavator's head platform via a shock-absorbing bracket and equipped with a dust- and waterproof protective cover to ensure operational performance even in harsh environments.

[0061] The first image data captured by the image acquisition unit is transmitted to the processing unit via a wired or wireless network. The processing unit can be an industrial computer installed in the bucket wheel machine control room and equipped with a high-performance GPU for running the deep learning model. After receiving the first image data, the processing unit inputs it into the pre-trained first network model.

[0062] In this embodiment, the first network model adopts an improved Faster R-CNN structure, including: a convolutional layer, which is used to extract features from the input first image data and output a feature map; a candidate region selection network, which outputs a number of candidate region data based on the feature map output by the convolutional layer; a pooling layer, which is used to pool the data of each candidate region and output the corresponding feature vector; a region classifier, which calculates the probability of whether the corresponding candidate region belongs to the material pile by passing the feature vector output by the pooling layer through a fully connected layer and a Softmax layer; and an output layer, which outputs a target recognition box of the material pile based on the candidate region classification result output by the region classifier.

[0063] Specifically, the convolutional layer uses the ResNet50 network structure, replacing the VGG16 structure in Example 1. ResNet50 contains 50 convolutional layers and introduces a residual connection mechanism, which can effectively solve the gradient vanishing problem in deep networks and improve feature extraction capabilities. The output feature map of the convolutional layer has a size of W / 16×H / 16×2048, where W and H are the width and height of the input image, respectively.

[0064] The candidate region selection network (RPN) performs a sliding window operation on the fourth-stage feature map of ResNet50 to generate anchor points. The RPN network contains a 3×3 convolution layer and two 1×1 convolution layers, which are used for classification and regression respectively. The classification convolution layer outputs 2k channels, indicating the probability of whether the k anchor points contain the material pile; the regression convolution layer outputs 4k channels, indicating the bounding box adjustment parameters of the k anchor points. In this embodiment, k is set to 9, corresponding to three scales (128×128, 256×256, 512×512) and three aspect ratios (1:1, 1:2, 2:1) of anchor points.

[0065] The pooling layer uses the ROI Align operation, replacing the ROI pooling operation in Example 1. ROI Align calculates the values ​​of feature points through bilinear interpolation, avoiding the quantization operation in ROI pooling. This preserves more spatial information and improves object detection accuracy. The output size of ROI Align is set to 7×7.

[0066] The region classifier consists of two fully connected layers. The first has an output dimension of 2048, and the second has an output dimension of 1024. The fully connected layers are followed by two parallel output layers: a classification output layer and a bounding box regression output layer. The classification output layer uses a Softmax function to output the probability that a candidate region belongs to the material pile; the bounding box regression output layer outputs the bounding box adjustment parameters of the candidate region.

[0067] The output layer uses an improved non-maximum suppression algorithm (Soft-NMS), replacing the traditional NMS algorithm in Example 1. Soft-NMS does not simply remove candidate boxes with high overlap, but instead reduces their confidence based on the degree of overlap, which is more effective in handling overlapping material piles. The Soft-NMS threshold is set to 0.5, and the attenuation function uses linear attenuation.

[0068] After acquiring the target recognition frame of the material pile, the system segments the first image data into a second image data within the target recognition frame. The second image data only contains the material pile area and excludes the background area, which helps improve the accuracy of subsequent material analysis.

[0069] In this embodiment, the steps of preprocessing the second image data include: performing grayscale conversion on the second image data to generate a grayscale image; performing Gaussian filtering on the grayscale image; randomly selecting a certain area in the grayscale image after Gaussian filtering, performing image magnification processing, and obtaining an enlarged filtered image; for the enlarged filtered image, extracting the material particle contour through an edge detection algorithm; fitting the material particle contour through an ellipse fitting algorithm, and outputting the ellipse fitting parameters as the preprocessed data.

[0070] Specifically, grayscale conversion uses the ITU-R BT.709 standard, with the conversion formula being: Gray = 0.2126 × R + 0.7152 × G + 0.0722 × B, where R, G, and B represent the pixel values ​​of the red, green, and blue channels, respectively, and Gray represents the converted grayscale value. Compared to the weighted averaging method in Example 1, the ITU-R BT.709 standard better aligns with the human eye's perception of brightness and can produce more natural grayscale images.

[0071] Gaussian filtering uses an adaptive Gaussian filtering algorithm, automatically adjusting the standard deviation σ of the Gaussian kernel based on the local variance of the image. Specifically, a larger σ value (e.g., 2.0) is used for strong filtering in noisy areas of the image (where the local variance is large); a smaller σ value (e.g., 1.0) is used for weak filtering in detail-rich areas (where the local variance is small). This adaptive filtering method removes noise while preserving image detail.

[0072] When randomly selecting image regions, a multi-region sampling strategy was used. Five regions, each measuring 200 × 200 pixels, were randomly selected from the grayscale image. Each of these five regions was magnified at a magnification of 2x, resulting in five 400 × 400 pixel magnified filtered images. Multi-region sampling provides more comprehensive information about material particles, improving recognition accuracy.

[0073] The edge detection algorithm uses a modified Canny algorithm and introduces an adaptive threshold selection mechanism. Specifically, the low and high thresholds are automatically calculated based on the image's histogram distribution. The low threshold is set to 0.5 times the histogram mean, and the high threshold is set to 1.5 times the histogram mean. This adaptive threshold selection mechanism adapts to images under varying lighting conditions and improves the robustness of edge detection.

[0074] The ellipse fitting algorithm uses the RANSAC (Random Sample Consensus) method, which effectively handles data containing outliers. Specifically, five points are randomly selected from the material particle contours (an ellipse requires at least five points), the ellipse parameters are calculated, and then the number of contour points that conform to the ellipse model is counted. This process is repeated multiple times (for example, 100 times), and the ellipse model with the most conforming points is selected as the final fitting result. The RANSAC method effectively filters out the influence of outliers and improves the accuracy of ellipse fitting.

[0075] In addition to the ellipse fitting parameters, this example also extracts texture features of the material particles, including statistics such as energy, contrast, correlation, and entropy of the gray-level co-occurrence matrix (GLCM). These texture features, along with the ellipse fitting parameters, serve as preprocessed data and are input into the second network model to further improve the accuracy of material classification and particle size recognition.

[0076] The preprocessed ellipse fitting parameters are input into a pre-trained second network model to obtain the material category and particle size information of the material currently being conveyed by the bucket wheel excavator. The second network model is constructed based on an artificial neural network. Its pre-training process includes: establishing a material sample set, where each sample in the material sample set includes the ellipse fitting parameters corresponding to the material contour, a material category label, and a material size label. The material size labels include standard size, substandard size, and above-standard size; inputting the material sample set into the second network model, along with the ellipse fitting parameters corresponding to the material contour of each sample; obtaining the material category and material size predictions output by the second network model; comparing them with the material category and material size labels of the corresponding samples to calculate the loss; and using the calculated loss to perform backpropagation training on the second network model.

[0077] Specifically, the second network model adopts a multi-layer perceptron (MLP) structure and contains three hidden layers, with the number of neurons in each layer being 128, 64, and 32, respectively. The dimension of the input layer is the dimension of the ellipse fitting parameters (5 dimensions, corresponding to the ellipse center coordinates x0, y0, major axis radius a, minor axis radius b, and rotation angle θ, respectively). The output layer consists of two parts: material category output and material size output. The dimension of the material category output is the number of material categories (for example, 10 common materials such as coal, iron ore, bauxite, limestone, etc.), and the Softmax activation function is used. The dimension of the material size output is 3, corresponding to the three cases of standard size, below standard size, and above standard size, and the Softmax activation function is also used.

[0078] The material sample set contains 10,000 samples, covering 10 common materials and three size labels. During the sample collection process, high-precision cameras were used to capture material particles of different types and sizes. Ellipse fitting parameters were extracted through image processing, and professional personnel annotated the material category and size labels.

[0079] The training process used the Adam optimizer, with a learning rate of 0.001, a batch size of 64, and 100 training epochs. The cross-entropy loss function was used to measure the difference between the model's predictions and the true labels. During training, 5-fold cross-validation was used to evaluate model performance. The model achieved 95% accuracy in material category recognition and 90% accuracy in material size recognition on the validation set.

[0080] The steps of determining the currently enabled bucket wheel excavator control program based on the acquired material category information and material particle size information include: pre-setting a program library, wherein the program library contains bucket wheel excavator control programs corresponding to different material categories and different material particle sizes; when the material category information and material particle size information currently conveyed by the bucket wheel excavator are acquired, matching the corresponding bucket wheel excavator control program in the program library and starting it.

[0081] Specifically, the library includes 30 bucket wheel control programs for 10 material types and three material particle sizes. Each control program includes the following parameters: bucket wheel speed, bucket wheel lift height, boom pitch angle, slew speed, and conveyor speed. These parameters are optimized based on material type and particle size to achieve optimal loading and unloading efficiency and equipment safety.

[0082] For example, for coal, when the material particle size is standard, the bucket wheel speed is set to 30 rpm, the bucket wheel hoist height is set to 2.5 m, the boom pitch angle is set to 30°, the slew speed is set to 0.5 rpm, and the conveyor speed is set to 2.0 m / s. When the material particle size is below the standard size, the bucket wheel speed is reduced to 25 rpm, and other parameters remain unchanged. When the material particle size is above the standard size, the bucket wheel speed is increased to 35 rpm, the bucket wheel hoist height is increased to 3.0 m, and other parameters remain unchanged.

[0083] For iron ore, due to its high density, when the material particle size is standard, the bucket wheel speed is set to 25 rpm, the bucket wheel hoist height is set to 2.0 m, the boom pitch angle is set to 25°, the slewing speed is set to 0.4 rpm, and the conveyor belt speed is set to 1.8 m / s. When the material particle size is below the standard size, the bucket wheel speed is reduced to 20 rpm, the conveyor belt speed is reduced to 1.5 m / s, and other parameters remain unchanged. When the material particle size is above the standard size, the bucket wheel speed is increased to 30 rpm, the bucket wheel hoist height is increased to 2.5 m, the slewing speed is reduced to 0.3 rpm, and other parameters remain unchanged.

[0084] Once the system obtains the material type and particle size information currently being conveyed by the bucket wheel excavator, it searches the program library for a matching control program. This matching process employs an exact match strategy, meaning that the material type and particle size must exactly match before the corresponding control program is selected. If a matching control program is found, the system automatically activates it and adjusts various bucket wheel excavator parameters to achieve optimal loading and unloading efficiency and equipment safety.

[0085] This embodiment also includes a collision warning step, specifically including: setting a material pile size threshold; confirming the size of the material pile based on the target identification frame of the current bucket wheel excavator conveying the material pile; comparing the size of the material pile with the material pile size threshold, and when the size of the material pile is greater than the material pile size threshold, outputting a warning message that there is a collision risk.

[0086] Specifically, the material pile size threshold is set based on the bucket wheel excavator's operating range and safety distance. For example, for a certain bucket wheel excavator model, the material pile height threshold is set at 10 meters, and the material pile width threshold is set at 15 meters. If the detected material pile exceeds 10 meters in height or 15 meters in width, the system will determine that there is a collision risk and issue a warning message.

[0087] The size of the material pile is determined by the size of the target recognition frame. The height of the target recognition frame corresponds to the height of the material pile, and the width corresponds to the width of the material pile. The system converts the pixel size of the target recognition frame to the actual physical size. This conversion process takes into account the camera's intrinsic parameters (focal length, principal point coordinates, etc.) and extrinsic parameters (camera position, attitude, etc.).

[0088] When the host computer receives a warning message indicating a collision risk, it sends a control instruction to the bucket wheel machine control program to suspend the operation of the bucket wheel machine; and sends a control instruction to the sound and light alarm unit to issue a sound and light alarm.

[0089] Specifically, the host computer is the central control unit of the bucket wheel crane's control system, typically installed in the control room. Upon receiving a collision risk warning, the host computer immediately sends a pause command to the bucket wheel crane's control program, halting the current loading and unloading operation. Simultaneously, the host computer also sends an alarm command to the audible and visual alarm unit, triggering an audible and visual alarm.

[0090] The audible and visual alarm unit, consisting of a siren and warning light, is mounted in a prominent position on the bucket wheel excavator for easy observation by the operator and nearby personnel. The siren emits a high-decibel siren (typically 90-110 decibels), and the warning light flashes red (at a rate of 1-2 Hz). The audible and visual alarm lasts for a preset 30 seconds or until the operator manually clears the alarm.

[0091] After receiving the alarm, the operator needs to check the actual situation of the material pile to confirm whether there is a collision risk. If the risk is confirmed, measures need to be taken to adjust the position or height of the material pile. If the risk is confirmed to be non-existent (for example, the system is falsely reporting), the alarm can be manually cleared and the bucket wheel excavator can be restarted.

[0092] The bucket wheel excavator control method based on image monitoring provided in this embodiment can monitor the material conditions in the bucket wheel excavator's conveying path in real time, automatically identify the material type and characteristics, and select the most appropriate control program based on the identification results, thus achieving intelligent control of the bucket wheel excavator. Furthermore, the collision warning function can promptly identify potential safety hazards, avoid equipment collision accidents, and improve the operational safety of the bucket wheel excavator.

[0093] Example 2:

[0094] This embodiment provides a bucket wheel excavator control system based on image monitoring, including:

[0095] An image acquisition unit is provided on the head platform of the bucket wheel excavator and is used to acquire first image data of the conveying path area of ​​the bucket wheel excavator. This module is used to implement the function of step S100 in the first embodiment and will not be described in detail here.

[0096] A material pile recognition unit is used to input the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator; this module is used to implement the function of step S200 in the first embodiment and will not be described in detail here;

[0097] A material particle recognition unit segments second image data within the target recognition frame from the first image data based on the target recognition frame, preprocesses the second image data, inputs the preprocessed data into a pre-trained second network model, and obtains material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator. This module is used to implement the function of step S300 in the first embodiment and will not be described in detail here.

[0098] The bucket wheel machine control unit determines the currently enabled bucket wheel machine control program based on the acquired material category information and material particle size information; this module is used to implement the function of step S400 in Example 1 and will not be repeated here.

[0099] Example 3:

[0100] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and runnable on the processor. When the processor executes the program, the bucket wheel machine control method based on image monitoring as described in any embodiment of the present invention is implemented.

[0101] Example 4:

[0102] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the bucket wheel machine control method based on image monitoring as described in any embodiment of the present invention is implemented.

[0103] In the embodiments of the present application, "at least one" refers to one or more, and "more" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b, c can be single or multiple.

[0104] Those skilled in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented using a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0106] In the several embodiments provided in this application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), magnetic disk or optical disk, and other media that can store program code.

[0107] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention's description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A bucket wheel excavator control method based on image monitoring, characterized in that: The following steps are involved: The image acquisition unit provided on the head platform of the bucket wheel machine collects first image data located in the conveying path area of ​​the bucket wheel machine; Inputting the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator; Segmenting second image data within the target identification frame from the first image data based on the target identification frame, preprocessing the second image data, and inputting the preprocessed data into a pre-trained second network model to obtain material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator; The currently enabled bucket wheel excavator control program is determined based on the acquired material category information and material particle size information.

2. The bucket wheel excavator control method based on image monitoring according to claim 1, characterized in that: The first network model includes: A convolutional layer, configured to extract features from the input first image data and output a feature map; The candidate region selection network outputs several candidate region data based on the feature map output by the convolutional layer; The pooling layer is used to pool the data of each candidate region and output the corresponding feature vector; The region classifier passes the feature vector output by the pooling layer through the fully connected layer and the softmax layer to calculate the probability of whether the corresponding candidate region belongs to the material pile; The output layer outputs the target recognition box of the material pile based on the candidate region classification results output by the region classifier.

3. The bucket wheel excavator control method based on image monitoring according to claim 1, characterized in that: The step of preprocessing the second image data comprises: performing grayscale conversion on the second image data to generate a grayscale image; Perform Gaussian filtering on the grayscale image; Randomly select a region in the grayscale image after Gaussian filtering, perform image magnification processing, and obtain an amplified filtered image; For the magnified filtered image, the outline of the material particles is extracted using the edge detection algorithm; The material particle contour is fitted by an ellipse fitting algorithm, and the ellipse fitting result outputs the ellipse fitting parameters as preprocessed data.

4. The bucket wheel excavator control method based on image monitoring according to claim 3 is characterized in that: The second network model is constructed based on an artificial neural network, and its pre-training process includes: Establish a material sample set. Each sample in the material sample set includes the ellipse fitting parameters of the corresponding material contour, the material category label, and the material size label. The material size label includes three types: standard size, below standard size, and above standard size. The material sample set is input into the second network model, and the ellipse fitting parameters of the material contour corresponding to each sample are input. The material category and material size prediction results output by the second network model are obtained, and the loss is calculated by comparing with the material category label and material size label of the corresponding sample. The second network model is back-propagated and trained based on the calculated loss.

5. The bucket wheel excavator control method based on image monitoring according to claim 1, characterized in that: The method further includes a collision warning step, specifically comprising: Set the material pile size threshold; Confirm the size of the material pile according to the target identification frame of the current bucket wheel conveyor material pile; The size of the material pile is compared with a material pile size threshold. When the size of the material pile is greater than the material pile size threshold, a warning message indicating a collision risk is output.

6. The bucket wheel excavator control method based on image monitoring according to claim 5, characterized in that: Also includes: When the host computer receives a warning message indicating a collision risk, it sends a control instruction to the bucket wheel machine control program to suspend the operation of the bucket wheel machine; And send control instructions to the sound and light alarm unit to issue sound and light alarm.

7. The bucket wheel excavator control method based on image monitoring according to claim 1, characterized in that: The step of determining the currently enabled bucket wheel excavator control program based on the acquired material category information and material particle size information includes: A program library is pre-set, wherein the program library contains bucket wheel excavator control programs corresponding to different material categories and different material particle sizes; When the material category information and material particle size information of the current bucket wheel machine conveying are obtained, the corresponding bucket wheel machine control program is matched in the program library and started.

8. A bucket wheel excavator control system based on image monitoring, characterized in that: include: An image acquisition unit, provided on the head platform of the bucket wheel machine, for acquiring first image data located in a conveying path area of ​​the bucket wheel machine; A material pile recognition unit is used to input the collected first image data into a pre-trained first network model to obtain a target recognition frame of the material pile currently being transported by the bucket wheel excavator; The material particle recognition unit segments the second image data within the target recognition frame from the first image data based on the target recognition frame, preprocesses the second image data, inputs the preprocessed data into the pre-trained second network model, and obtains the material category information and material particle size information of the material currently being conveyed by the bucket wheel excavator; The bucket wheel machine control unit determines the currently enabled bucket wheel machine control program based on the acquired material category information and material particle size information.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images