Bucket wheel machine material taking method, device, equipment and medium based on image processing

Abstract

The application provides a bucket wheel machine material taking method, device, equipment and medium based on image processing, relates to the field of engineering control technology, and comprises the following steps: constructing stereoscopic vision of images in material pile image information according to a deep learning network to obtain a material pile three-dimensional model; constructing a dynamic environment model based on the material pile three-dimensional model and material pile dynamic change parameters, planning a trajectory of an initial position of the bucket wheel machine through reinforcement learning technology to obtain a walking trajectory of the bucket wheel machine; predicting a motion state according to the walking trajectory of the bucket wheel machine, segmented information in the material pile image information and layering height in the material pile three-dimensional model to obtain a slewing trajectory of the bucket wheel machine around a cantilever; and constructing the walking trajectory of the bucket wheel and the slewing trajectory of the bucket wheel machine, combining with the evaluation and optimization of historical material taking data, material type characteristics and material pile stability to obtain a material taking scheme. The application solves the problems of low material taking efficiency, missed or ineffective material taking of part of the material piles.

Description

Technical Field

[0001] This invention relates to the field of engineering control technology, and more specifically, to a method, apparatus, equipment, and medium for material handling in a bucket wheel excavator based on image processing. Background Technology

[0002] In the field of engineering control technology, bucket wheel excavators are widely used as loading and unloading tools for bulk materials in steel raw material yards, power plant coal yards, and port bulk cargo yards. During operation, a layered material handling strategy is often employed, where the bucket wheel excavator extracts material layer by layer from the top down along the height of the stockpile. However, these operating environments generally suffer from irregular stockpile shapes, disordered stacking layouts, and complex and ever-changing environmental interference factors, making it difficult for the bucket wheel excavator to move flexibly and accurately according to the actual conditions of the stockpile. Current technology, which relies on manual material handling plans based on operator experience, leads to low material handling efficiency, the omission of some stockpile material, or even the inability to effectively handle material at all.

[0003] Therefore, there is an urgent need for image processing-based methods, devices, equipment, and media for bucket wheel excavators to solve the problems of low material handling efficiency, missed material piles, or ineffective material handling altogether. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, equipment, and medium for material handling in a bucket wheel excavator based on image processing, in order to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows:

[0005] In a first aspect, this application provides a bucket wheel excavator material handling method based on image processing, including:

[0006] Acquire image information of the material pile and the initial position of the bucket wheel excavator, wherein the image information of the material pile includes complete images of the material pile from at least two angles;

[0007] The images in the material pile image information are processed by multi-directional convolutional kernels and deep learning networks to obtain image features. The image features and the geometric information in the material pile image information are then used to construct a stereo vision to obtain a three-dimensional model of the material pile.

[0008] Based on the three-dimensional model of the material pile and the preset dynamic change parameters of the material pile, a dynamic environment model is constructed. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position, thereby obtaining the travel trajectory of the bucket wheel excavator.

[0009] Based on the walking trajectory of the bucket wheel excavator, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, the motion state is predicted to obtain the swirling trajectory of the bucket wheel excavator around the cantilever.

[0010] The material handling scheme is constructed based on the bucket wheel's traveling trajectory and the bucket wheel machine's slewing trajectory around the cantilever, and then evaluated and optimized by combining historical material handling data, material type characteristics, and stockpile stability.

[0011] Secondly, this application also provides an image processing-based bucket wheel excavator material handling device, including:

[0012] The acquisition module is used to acquire image information of the material pile and the initial position of the bucket wheel excavator. The image information of the material pile includes complete images of the material pile from at least two angles.

[0013] The first construction module is used to process the images in the material pile image information according to multi-directional convolutional kernels and deep learning networks to obtain image features, and to construct a stereo vision by combining the image features and the geometric information in the material pile image information to obtain a three-dimensional model of the material pile.

[0014] The trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the material pile and preset dynamic change parameters of the material pile. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position to obtain the walking trajectory of the bucket wheel excavator.

[0015] The prediction module is used to predict the motion state of the bucket wheel excavator based on its travel trajectory, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, so as to obtain the trajectory of the bucket wheel excavator rotating around the cantilever.

[0016] The second construction module is used to construct a material handling scheme based on the bucket wheel's travel trajectory and the bucket wheel machine's slewing trajectory around the cantilever, and to evaluate and optimize the scheme by combining historical material handling data, material type characteristics, and stockpile stability.

[0017] Thirdly, this application also provides an image processing-based bucket wheel excavator material handling device, including:

[0018] Memory, used to store computer programs;

[0019] A processor is used to implement the steps of the image processing-based bucket wheel excavator material handling method when executing the computer program.

[0020] Fourthly, this application also provides a medium storing a computer program, which, when executed by a processor, implements the steps of the above-described image processing-based bucket wheel excavator material handling method.

[0021] The beneficial effects of this invention are as follows:

[0022] This invention utilizes multi-directional convolutional kernels and deep learning network technology to construct a stereoscopic vision of the material pile image information, thereby generating an accurate 3D model of the material pile. This accurate 3D model can realistically reproduce the shape and structural features of the material pile. Based on the 3D model and dynamic change parameters of the material pile, a dynamic environment model is constructed. This model reflects the dynamic changes of the material pile in real time, achieving adaptive optimization of the bucket wheel excavator's travel trajectory. The bucket wheel excavator's travel trajectory flexibly moves and accurately picks up materials according to the actual situation of the pile. Furthermore, a cantilever-around-the-arm rotation trajectory of the bucket wheel excavator is constructed, and the travel trajectory of the bucket wheel excavator is dynamically adjusted to ensure that the bucket wheel excavator can accurately locate a specific position in the material pile during material picking, thereby achieving efficient and uniform material picking operations. The construction of the cantilever-around-the-arm rotation trajectory of the bucket wheel excavator solves the problem of uneven material picking in traditional bucket wheel excavator methods. This invention solves the problems of low material picking efficiency, missed material, or ineffective material picking.

[0023] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic diagram of the bucket wheel excavator material handling method based on image processing described in an embodiment of the present invention;

[0026] Figure 2 This is a schematic diagram of the structure of the bucket wheel excavator material handling device based on image processing as described in an embodiment of the present invention.

[0027] The diagram is labeled as follows: 800, image processing-based bucket wheel excavator material handling equipment; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0029] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0030] Example 1:

[0031] This embodiment provides a bucket wheel excavator material handling method based on image processing.

[0032] See Figure 1 The figure shows that the method includes steps S1 to S5, including:

[0033] S1: Obtain the image information of the material pile and the initial position of the bucket wheel excavator. The image information of the material pile includes complete images of the material pile from at least two angles.

[0034] In this step, the initial position of the bucket wheel excavator includes the initial position of the trolley, the initial angle of the cantilever pitch, and the initial angle of rotation. The complete images of the material pile at at least two angles ensure multi-angle viewing angles and complete image coverage of the material pile.

[0035] S2: The images in the material pile image information are processed according to multi-directional convolutional kernels and deep learning networks to obtain image features. The image features and the geometric information in the material pile image information are used to construct a stereo vision to obtain a three-dimensional model of the material pile.

[0036] To clarify the specific method for obtaining the 3D model of the stockpile, step S2 includes S21 to S24, specifically:

[0037] S21: Optimize the deep learning network based on the multi-directional convolutional kernel to obtain an optimized deep learning model;

[0038] To clarify the specific method for obtaining the optimized deep learning model, step S21 includes S211 to S214, specifically:

[0039] S211: Obtain dynamic information about features;

[0040] In this step, the dynamic feature information refers to the features of the input data, which are obtained through time series analysis, difference operations, or other dynamic feature extraction methods.

[0041] S212: Based on the multi-directional convolutional kernel, perform feature extraction in multiple directions on the dynamic feature information to obtain multiple information features;

[0042] In this step, the feature extraction in multiple directions includes features in horizontal, vertical, and diagonal directions, to capture multi-dimensional information in the data.

[0043] The information feature expression is:

[0044] F = Conv(I,K) (1)

[0045] In equation (1) above, F represents the information feature, Conv represents the convolution operation, I represents the input image, and K represents the multi-directional convolution kernel;

[0046] The input image is feature dynamic information;

[0047] By performing multi-directional convolution operations, features in different directions are extracted from the dynamic feature information. Using the multi-directional convolution kernel to extract features from the dynamic feature information can capture the structural information in the data more comprehensively.

[0048] S213: The multiple information features are integrated and processed according to the feature fusion method to obtain multi-directional local features;

[0049] S214: The multi-directional local features are weighted and summed, and the weights are assigned by combining the feature dynamic information with the feature intensity and the position information in the network to obtain the optimized deep learning model.

[0050] In this step, the optimized deep learning model is:

[0051]

[0052] In equation (2) above, F 优化 This represents the optimized deep learning model, where Weight represents the weight allocation function, F. 融合 This represents multi-directional local features, and I represents dynamic information of the features.

[0053] The weighted summation and weight allocation dynamically adjust the contribution of features based on their importance and global impact, making the optimized deep learning model pay more attention to important features.

[0054] S22: Based on the optimized deep learning model, feature extraction is performed on the images in the stockpile image information, and the performance evaluation index is improved by combining the attention mechanism to obtain image features;

[0055] In this step, the image feature expression is:

[0056] F′=Attention(F) (3)

[0057] In equation (3) above, F′ represents image features, Attention represents the attention mechanism, and F represents information features;

[0058] By incorporating an attention mechanism into the optimized deep learning model, key features of the material pile image can be accurately extracted, improving the efficiency of feature extraction, reducing redundant information, and enhancing the model's adaptability to complex scenarios. The performance evaluation metrics include accuracy and recall.

[0059] S23: Based on the image features and preset reference features, feature point matching is performed. After removing incorrect matching points, the target matching point is obtained. The reference features are predefined feature points.

[0060] In this step, the reference feature is used as a benchmark for matching, and the removal of erroneous matching points is performed using the RANSAC algorithm.

[0061] The target matching point is:

[0062] M = Filter(Match(F′,R)) (4)

[0063] In equation (4) above, M represents the target matching point, Filter represents the function to remove incorrect matching points, Match represents the matching function, F′ represents the image feature, and R represents the reference feature.

[0064] S24: Calculate depth information based on the target matching point and the geometric information in the material pile image information, construct a three-dimensional point cloud model using the depth information and the target matching point, and optimize the three-dimensional point cloud model using a filtering algorithm to obtain a three-dimensional model of the material pile.

[0065] In this step, the three-dimensional model of the stockpile is as follows:

[0066] D = Depth(M, A 几何 (5)

[0067] In equation (5) above, D represents depth information, Depth represents the depth information calculation function, M represents the target matching point, and A 几何 Represents geometric information;

[0068] P 点云 =PointCloud(D,M) (6)

[0069] In equation (6) above, P 点云 The point cloud model is represented by PointCloud, the point cloud construction function is represented by D, the depth information is represented by M, and the target matching point is represented by M.

[0070] P = Filter(P) 点云 (7)

[0071] In equation (7) above, P represents the three-dimensional model of the stockpile, Filter represents the filtering function, and P 点云 Represents a point cloud model;

[0072] Wherein, the geometric information refers to the geometric information in the stockpile image information;

[0073] The filtering algorithm is used to remove noise and redundant points from the three-dimensional point cloud model, making the three-dimensional model of the material pile smoother and more accurate. The three-dimensional point cloud model constructed based on depth information and target matching points can realistically reflect the shape and structure of the material pile. The three-dimensional model of the material pile solves the problem of inaccurate feature extraction in the prior art.

[0074] S3: Based on the three-dimensional model of the material pile and the preset dynamic change parameters of the material pile, a dynamic environment model is constructed. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position to obtain the walking trajectory of the bucket wheel excavator.

[0075] To clarify the specific method for obtaining the travel trajectory of the bucket wheel excavator, step S3 includes S31 to S35, specifically:

[0076] S31: Obtain the actions performed by the bucket wheel excavator, wherein the actions performed by the bucket wheel excavator include the direction of movement, the speed of movement, and the material handling depth;

[0077] S32: Extract change data based on multiple time steps in the three-dimensional model of the stockpile to obtain dynamic change parameters of the stockpile;

[0078] In this step, the multiple time steps include a first time step and a second time step. Based on the three-dimensional model of the stockpile, change data is extracted at multiple time steps to obtain the dynamic change parameters of the stockpile.

[0079] To clarify the specific method for obtaining the dynamic change parameters of the stockpile, step S32 includes S321 to S323, specifically:

[0080] S321: Based on the first time step in the three-dimensional model of the material pile, feature extraction is performed on the geometric information in the three-dimensional model of the material pile to obtain the first time step features;

[0081] In this step, the geometric information in the three-dimensional model of the material pile is converted into quantifiable feature data.

[0082] S322: Based on the second time step in the three-dimensional model of the material pile, feature extraction is performed on the geometric information in the three-dimensional model of the material pile to obtain the second time step features;

[0083] S323: Parameter calculation is performed based on the first time step features and the second time step features to obtain dynamic change parameters of the material pile. The dynamic change parameters of the material pile include volume change rate, position change, height change, contour change and shape change parameters.

[0084] In this step, by calculating the features of the first time step and the features of the second time step, and comparing the feature data of the two time steps, the changes of the material pile during the operation of the bucket wheel excavator are dynamically reflected, which solves the problem that the dynamic changes of the material pile are difficult to monitor and quantify in real time.

[0085] S33: Based on the height and contour changes in the dynamic change parameters of the stockpile, a dynamic environment model is constructed to obtain the dynamic environment model;

[0086] In this step, the height and contour changes are used to construct a system that reflects the dynamic changes of the stockpile in real time.

[0087] S34: The bucket wheel excavator's movement space is obtained by parametric processing based on its moving direction, moving speed, and material handling depth.

[0088] S35: Based on the real-time state in the dynamic environment model, input the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile into the reinforcement learning model, and train it in conjunction with the action space of the bucket wheel excavator until the model convergence condition is met, and obtain the walking trajectory of the bucket wheel excavator.

[0089] To clarify the specific training process for the bucket wheel excavator's travel trajectory, step S35 includes S351 to S353, specifically:

[0090] S351: Based on the real-time state in the dynamic environment model and the preset actual requirements of the bucket wheel excavator, the reinforcement learning model is optimized in multiple dimensions, and the reward weight of the reinforcement learning model is dynamically adjusted in combination with the preset adaptive reward function adjustment mechanism to obtain the optimized model.

[0091] In this step, the expression for the dynamic adjustment mechanism is:

[0092] R(s,a)=α·R 原始 (s,a)+β·R 原始 (s,a) (8)

[0093] In equation (8) above, R(s,a) represents the reward obtained by taking action a in state s, and α and β represent dynamically adjusted weight coefficients. 原始 This represents the original reward function;

[0094] The actual requirements of the bucket wheel excavator include motion accuracy and energy efficiency. The reinforcement learning model can better adapt to the dynamic environment through multi-dimensional optimization. By reflecting the dynamic changes of the material pile in real time, it realizes adaptive optimization of the walking trajectory of the bucket wheel excavator, so as to solve the problem that the reinforcement learning model is difficult to take into account multiple objectives in complex dynamic environments.

[0095] S352: Input the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile into the optimization model, and train it in combination with the action space of the bucket wheel excavator. Output the action of the bucket wheel excavator according to the real-time state in the dynamic environment model.

[0096] In this step, by training the optimization model, optimal action sequences that adapt to dynamic environments can be generated. These sequences effectively guide the travel trajectory planning of the bucket wheel excavator, solving the problem of efficiently generating optimal action sequences in complex dynamic environments and improving the path planning efficiency and accuracy of the bucket wheel excavator.

[0097] S353: Execute the output bucket wheel excavator action to obtain a new state and reward value. Update the parameters of the optimization model using the new state and reward value until the model convergence condition is met, and obtain the bucket wheel excavator's travel trajectory.

[0098] In this step, by continuously updating and optimizing the optimization model, it can gradually converge to the optimal solution and generate a stable bucket wheel excavator travel trajectory.

[0099] The dynamic environment model and the bucket wheel excavator's motion space are used to adaptively optimize the bucket wheel excavator's travel trajectory, which allows for flexible movement and precise material retrieval based on the actual conditions of the material pile.

[0100] S4: Based on the walking trajectory of the bucket wheel excavator, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, the motion state is predicted to obtain the swirling trajectory of the bucket wheel excavator around the cantilever.

[0101] To clarify the specific method for obtaining the trajectory of the bucket wheel excavator rotating around the cantilever, step S4 includes S41 to S44, specifically:

[0102] S41: Input the walking trajectory of the bucket wheel excavator, the segmentation information in the image information of the material pile, and the layer height in the three-dimensional model of the material pile into the deep learning network for error training to obtain the prediction model;

[0103] In this step, the deep learning network learns the intrinsic relationship between the bucket wheel excavator's travel trajectory, the segmentation information in the stockpile image information, and the layer height in the stockpile's three-dimensional model. By minimizing the error between the predicted value and the actual value, the model's prediction accuracy is improved.

[0104] S42: Based on the prediction model, the trajectory of the bucket wheel excavator, the segmentation information in the material pile image information, and the layering height in the three-dimensional model of the material pile are used to predict the trajectory of the bucket wheel excavator, and the initial motion trajectory of the bucket wheel excavator is obtained.

[0105] S43: Input the initial motion trajectory of the bucket wheel excavator into the reinforcement learning model, select the optimal action from the action space of the bucket wheel excavator based on the initial motion trajectory of the bucket wheel excavator;

[0106] In this step, the optimal action is selected in the complex action space of the bucket wheel excavator through a reinforcement learning model, thereby optimizing the motion trajectory of the bucket wheel excavator.

[0107] S44: Calculate the reward value for the optimal action according to the reward function. When the convergence condition is met, the trajectory of the bucket wheel excavator rotating around the cantilever is obtained.

[0108] In this step, the convergence condition is:

[0109] |R t +1-R t ∣<∈ (9)

[0110] In equation (9) above, R t Let represent the reward value for the t-th iteration, and ∈ represent the threshold.

[0111] When the absolute value of the difference between the reward values ​​of two consecutive iterations is less than ∈, the convergence state is reached. The trajectory of the bucket wheel excavator around the cantilever is the optimal trajectory for the bucket wheel excavator to efficiently and safely complete material grabbing and unloading during the cantilever rotation process. The reward function is calculated based on factors such as trajectory smoothness, energy consumption, and operation time.

[0112] The bucket wheel excavator's gyratory trajectory around the cantilever is dynamically adjusted according to the bucket wheel excavator's travel trajectory, ensuring that the bucket wheel excavator can accurately position itself at a specific location on the material pile when picking up materials, thereby achieving efficient and uniform material picking operations.

[0113] S5: Based on the bucket wheel's travel trajectory and the bucket wheel machine's slewing trajectory around the cantilever, a material handling scheme is constructed, and evaluated and optimized by combining historical material handling data, material type characteristics, and stockpile stability.

[0114] To clarify the specific method for obtaining materials, step S5 includes S51 to S54, specifically:

[0115] S51: The bucket wheel travel trajectory and the bucket wheel machine's slewing trajectory around the cantilever are integrated, and an initial trajectory model is obtained by simulating the integrated result;

[0116] S52: The initial trajectory model is tested based on preset trajectory constraints to obtain the final trajectory model;

[0117] In this step, the verification is to ensure that the initial trajectory model meets various constraints in actual operation, so as to obtain the final trajectory model that meets the requirements;

[0118] The trajectory constraints include safety distance, speed limit, equipment capacity, and / or range of motion.

[0119] S53: Based on the historical material taking data, the characteristics of the material type, the stability of the material stack, and the final trajectory model, multiple material taking parameters are obtained;

[0120] In this step, more accurate material selection parameters are generated by integrating multiple factors, avoiding the problem of unreasonable material selection schemes caused by a single factor;

[0121] The material type characteristics include density and friction coefficient; the stockpile stability includes the shape and stability of the material pile; and the material handling parameters include material handling speed, material handling angle, and / or material handling depth.

[0122] S54: Develop an initial material handling plan based on multiple material handling parameters, and obtain a final material handling plan by evaluating the initial material handling plan.

[0123] In this step, the operation is evaluated to ensure that the final material handling plan has high efficiency and reliability in actual operation, while reducing equipment wear and material waste.

[0124] The assessment includes indicators such as efficiency, safety, and / or energy consumption.

[0125] Example 2:

[0126] This embodiment provides a bucket wheel excavator material handling device based on image processing, the device comprising:

[0127] The acquisition module is used to acquire image information of the material pile and the initial position of the bucket wheel excavator. The image information of the material pile includes complete images of the material pile from at least two angles.

[0128] The first construction module is used to process the images in the material pile image information according to multi-directional convolutional kernels and deep learning networks to obtain image features, and to construct a stereo vision by combining the image features and the geometric information in the material pile image information to obtain a three-dimensional model of the material pile.

[0129] The trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the material pile and preset dynamic change parameters of the material pile. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position to obtain the walking trajectory of the bucket wheel excavator.

[0130] To clarify the specific methods for obtaining the trajectory planning module, the following are included:

[0131] The acquisition unit is used to acquire the actions performed by the bucket wheel excavator, including the direction of movement, the speed of movement, and the material handling depth.

[0132] The extraction unit is used to extract change data based on multiple time steps in the three-dimensional model of the material pile to obtain dynamic change parameters of the material pile;

[0133] A construction unit is used to construct a dynamic environment model based on the height and contour changes in the dynamic change parameters of the stockpile.

[0134] The processing unit is used to perform parameterization processing based on the moving direction, moving speed and material picking depth of the bucket wheel excavator to obtain the bucket wheel excavator's action space;

[0135] The training unit is used to input the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile into the reinforcement learning model based on the real-time state in the dynamic environment model, and to train the model in conjunction with the action space of the bucket wheel excavator until the model convergence condition is met, thereby obtaining the travel trajectory of the bucket wheel excavator.

[0136] The prediction module is used to predict the motion state of the bucket wheel excavator based on its travel trajectory, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, so as to obtain the trajectory of the bucket wheel excavator rotating around the cantilever.

[0137] The second construction module is used to construct a material handling scheme based on the bucket wheel's travel trajectory and the bucket wheel machine's slewing trajectory around the cantilever, and to evaluate and optimize the scheme by combining historical material handling data, material type characteristics, and stockpile stability.

[0138] It should be noted that the specific manner in which each module performs its operation in the apparatus described in the above embodiments has been described in detail in the embodiments of the method, and will not be elaborated here.

[0139] Example 3:

[0140] Corresponding to the above method embodiments, this embodiment also provides a bucket wheel excavator material handling device based on image processing. The bucket wheel excavator material handling device based on image processing described below can be referred to in correspondence with the bucket wheel excavator material handling method based on image processing described above.

[0141] Figure 2 This is a block diagram illustrating an image processing-based bucket wheel excavator material handling device 800 according to an exemplary embodiment. Figure 2 As shown, the image processing-based bucket wheel excavator material handling device 800 may include: a processor 801 and a memory 802. The image processing-based bucket wheel excavator material handling device 800 may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0142] The processor 801 controls the overall operation of the image-processing-based bucket wheel excavator reclaiming device 800 to complete all or part of the steps in the image-processing-based bucket wheel excavator reclaiming method described above. The memory 802 stores various types of data to support the operation of the image-processing-based bucket wheel excavator reclaiming device 800. This data may include, for example, instructions for any application or method operating on the image-processing-based bucket wheel excavator reclaiming device 800, as well as application-related data such as contact data, sent and received messages, images, audio, video, etc. The memory 802 can be implemented using any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in the memory 802 or transmitted via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. I / O interface 804 provides an interface between processor 801 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 805 is used for wired or wireless communication between the image processing-based bucket wheel excavator material handling device 800 and other devices. Wireless communication includes, for example, Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 805 may include a Wi-Fi module, a Bluetooth module, or an NFC module.

[0143] In an exemplary embodiment, the image processing-based bucket wheel excavator material handling device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the image processing-based bucket wheel excavator material handling method described above.

[0144] Example 4:

[0145] Corresponding to the above method embodiments, this embodiment also provides a medium. The medium described below can be referred to in conjunction with the image processing-based bucket wheel excavator material handling method described above.

[0146] A medium storing a computer program, which, when executed by a processor, implements the steps of the image processing-based bucket wheel excavator material handling method described in the above method embodiments.

[0147] The medium can specifically be any medium capable of storing program code, such as a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0149] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A bucket wheel excavator material handling method based on image processing, characterized in that, include: Acquire material pile image information and the initial position of the bucket wheel excavator, wherein the material pile image information includes complete images of the material pile from at least two angles; The images in the material pile image information are processed by multi-directional convolutional kernels and deep learning networks to obtain image features. The image features and the geometric information in the material pile image information are then used to construct a stereo vision to obtain a three-dimensional model of the material pile. Based on the three-dimensional model of the material pile and the preset dynamic change parameters of the material pile, a dynamic environment model is constructed. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position, thereby obtaining the travel trajectory of the bucket wheel excavator. The specific method for obtaining the travel trajectory of the bucket wheel excavator includes: The actions performed by the bucket wheel excavator are obtained, including the direction of movement, the speed of movement, and the material handling depth. Based on multiple time steps in the three-dimensional model of the stockpile, change data is extracted to obtain dynamic change parameters of the stockpile. A dynamic environment model is constructed based on the height and contour changes in the dynamic parameters of the material pile. The bucket wheel excavator's movement space is obtained by parametric processing based on its moving direction, moving speed, and material handling depth. Based on the real-time state in the dynamic environment model, the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile are input into the reinforcement learning model, and the model is trained in conjunction with the action space of the bucket wheel excavator until the model convergence condition is met, thus obtaining the travel trajectory of the bucket wheel excavator. Based on the walking trajectory of the bucket wheel excavator, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, the motion state is predicted to obtain the swirling trajectory of the bucket wheel excavator around the cantilever. The material handling scheme is constructed based on the bucket wheel's traveling trajectory and the bucket wheel machine's gyratory trajectory around the cantilever, and then evaluated and optimized by combining historical material handling data, material type characteristics, and stockpile stability.

2. The bucket wheel excavator material handling method based on image processing according to claim 1, characterized in that, The images in the material pile image information are processed using multi-directional convolutional kernels and deep learning networks to obtain image features. These image features, along with the geometric information in the material pile image information, are then used to construct a stereoscopic vision model of the material pile, including: The deep learning network is optimized based on the multi-directional convolutional kernels to obtain an optimized deep learning model; Based on the optimized deep learning model, feature extraction is performed on the images in the stockpile image information, and the performance evaluation index is improved by combining an attention mechanism to obtain image features; Feature point matching is performed based on the image features and preset reference features. After removing incorrect matching points, the target matching point is obtained. The reference features are predefined feature points. Depth information is calculated based on the target matching point and the geometric information in the material pile image information. A three-dimensional point cloud model is constructed using the depth information and the target matching point. The three-dimensional point cloud model is then optimized using a filtering algorithm to obtain the three-dimensional model of the material pile.

3. The bucket wheel excavator material handling method based on image processing according to claim 1, characterized in that, Based on the real-time state in the dynamic environment model, the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile are input into the reinforcement learning model, and training is performed in conjunction with the bucket wheel excavator's motion space until the model convergence condition is met, thus obtaining the bucket wheel excavator's trajectory, including: The reinforcement learning model is optimized in multiple dimensions based on the real-time state in the dynamic environment model and the preset actual requirements of the bucket wheel excavator. The reward weight of the reinforcement learning model is dynamically adjusted in combination with the preset adaptive reward function adjustment mechanism to obtain the optimized model. The initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile are input into the optimization model, and the model is trained in combination with the action space of the bucket wheel excavator. The action of the bucket wheel excavator is output according to the real-time state in the dynamic environment model. The output bucket wheel excavator actions are executed to obtain a new state and reward value. The parameters of the optimization model are updated using the new state and reward value until the model convergence condition is met, and the travel trajectory of the bucket wheel excavator is obtained.

4. The bucket wheel excavator material handling method based on image processing according to claim 1, characterized in that, Based on the travel trajectory of the bucket wheel excavator, the segmentation information in the stockpile image information, and the layering height in the three-dimensional model of the stockpile, the motion state is predicted to obtain the bucket wheel excavator's rotation trajectory around the cantilever, including: The travel trajectory of the bucket wheel excavator, the segmentation information in the image information of the material pile, and the layer height in the three-dimensional model of the material pile are input into the deep learning network for error training to obtain the prediction model; Based on the prediction model, the trajectory of the bucket wheel excavator, the segmentation information in the material pile image information, and the layering height in the three-dimensional model of the material pile are used to predict the trajectory, and the initial motion trajectory of the bucket wheel excavator is obtained. The initial motion trajectory of the bucket wheel excavator is input into the reinforcement learning model, and the optimal action is obtained by selecting the action space of the bucket wheel excavator based on the initial motion trajectory. The reward value is calculated for the optimal action according to the reward function. When the convergence condition is met, the trajectory of the bucket wheel excavator rotating around the cantilever is obtained.

5. The bucket wheel excavator material handling method based on image processing according to claim 1, characterized in that, Based on the bucket wheel's travel trajectory and the bucket wheel excavator's gyratory trajectory around the cantilever, and combined with historical material handling data, material type characteristics, and stockpile stability for evaluation and optimization, a material handling scheme is obtained, including: The initial trajectory model is obtained by integrating the bucket wheel's travel trajectory and the bucket wheel machine's gyratory trajectory around the cantilever, and then simulating the integrated result. The initial trajectory model is tested based on preset trajectory constraints to obtain the final trajectory model; Multiple material extraction parameters are obtained by fusing the historical material extraction data, the characteristics of the material type, the stability of the material stack, and the final trajectory model. An initial material handling plan is formulated based on multiple material handling parameters, and a final material handling plan is obtained by evaluating the initial material handling plan.

6. A bucket wheel excavator material handling device based on image processing, characterized in that, include: The acquisition module is used to acquire image information of the material pile and the initial position of the bucket wheel excavator. The image information of the material pile includes complete images of the material pile from at least two angles. The first construction module is used to process the images in the material pile image information according to multi-directional convolutional kernels and deep learning networks to obtain image features, and to construct a stereo vision by combining the image features and the geometric information in the material pile image information to obtain a three-dimensional model of the material pile. The trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the material pile and preset dynamic change parameters of the material pile. Through the real-time state in the dynamic environment model, reinforcement learning technology is used to plan the trajectory of the bucket wheel excavator at its initial position to obtain the walking trajectory of the bucket wheel excavator. The trajectory planning module includes: The acquisition unit is used to acquire the actions performed by the bucket wheel excavator, including the direction of movement, the speed of movement, and the material handling depth. The extraction unit is used to extract change data based on multiple time steps in the three-dimensional model of the material pile to obtain dynamic change parameters of the material pile; A construction unit is used to construct a dynamic environment model based on the height and contour changes in the dynamic change parameters of the stockpile. The processing unit is used to perform parameterization processing based on the moving direction, moving speed and material picking depth of the bucket wheel excavator to obtain the bucket wheel excavator's action space; The training unit is used to input the initial position of the bucket wheel excavator and the target position in the three-dimensional model of the material pile into the reinforcement learning model based on the real-time state in the dynamic environment model, and to train the model in conjunction with the action space of the bucket wheel excavator until the model convergence condition is met, so as to obtain the walking trajectory of the bucket wheel excavator. The prediction module is used to predict the motion state of the bucket wheel excavator based on its travel trajectory, the segmentation information in the image information of the material pile, and the layering height in the three-dimensional model of the material pile, so as to obtain the trajectory of the bucket wheel excavator rotating around the cantilever. The second construction module is used to construct a material handling scheme based on the bucket wheel's travel trajectory and the bucket wheel machine's slewing trajectory around the cantilever, and to evaluate and optimize the scheme by combining historical material handling data, material type characteristics, and stockpile stability.

7. A bucket wheel excavator material handling device based on image processing, characterized in that, include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the steps of the image processing-based bucket wheel excavator material handling method as described in any one of claims 1 to 5.

8. A medium, characterized in that... The medium stores a computer program, which, when executed by a processor, implements the steps of the image processing-based bucket wheel excavator material handling method as described in any one of claims 1 to 5.

Images