Bucket wheel machine material taking method, device and equipment based on image processing and medium

Abstract

The invention provides a bucket wheel machine material taking method and device based on image processing, equipment and a medium, and relates to the technical field of engineering control, and the method comprises the steps: carrying out the stereoscopic vision construction of an image in material pile image information according to a deep learning network, and obtaining a material pile three-dimensional model; a dynamic environment model is constructed based on the material pile three-dimensional model and the material pile dynamic change parameters, track planning is conducted on the initial position of the bucket wheel machine through the reinforcement learning technology, and the walking track of the bucket wheel machine is obtained; according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layering height in the material pile three-dimensional model, motion state prediction is conducted, and the rotation track of the bucket wheel machine around a cantilever is obtained; construction is carried out according to the walking track of the bucket wheel and the rotation track of the bucket wheel machine around the cantilever, and a material taking scheme is obtained by combining evaluation and optimization of historical material taking data, material type characteristics and stacking stability. The problems that the material taking efficiency is low, part of stacked materials are omitted or the materials cannot be effectively taken at all are solved.

Description

Technical Field

[0001] The present invention relates to the field of engineering control technology, and in particular to a bucket wheel excavator material taking method, device, equipment and medium based on image processing. Background Art

[0002] In the field of engineering control technology, bucket wheel machines are widely used as loading and unloading tools for bulk materials in steel raw material yards, power plant coal yards, and port bulk cargo yards. During the operation, a layered material extraction strategy is often applied, that is, the bucket wheel machine extracts materials from the top layer by layer along the height direction of the material pile. However, these operating environments generally have irregular pile shapes, disorderly stacking layouts, and complex and changeable environmental interference factors, making it difficult for bucket wheel machines to move flexibly and accurately extract materials according to the actual conditions of the pile. In the existing technology, the material extraction plan is manually formulated based on the operator's experience, resulting in low material extraction efficiency, partial omission of pile materials, or the inability to effectively extract materials at all.

[0003] Therefore, there is an urgent need for bucket wheel excavator material reclaiming methods, devices, equipment and media based on image processing to solve the problems of low material reclaiming efficiency, partial material pile being missed or simply being unable to effectively reclaim materials. Summary of the invention

[0004] The purpose of the present invention is to provide a bucket wheel excavator material taking method, device, equipment and medium based on image processing to improve the above problems. In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present application provides a bucket wheel excavator material taking method based on image processing, comprising:

[0006] Acquire material pile image information and an initial position of a bucket wheel machine, wherein the material pile image information includes complete images of the material pile at least at two angles;

[0007] Processing the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and performing stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile;

[0008] A dynamic environment model is constructed based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile, and through the real-time state in the dynamic environment model, a reinforcement learning technology is used to perform trajectory planning on the initial position of the bucket wheel machine to obtain a walking trajectory of the bucket wheel machine;

[0009] The motion state is predicted according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation track of the bucket wheel machine around the cantilever;

[0010] The material taking plan is obtained by constructing the bucket wheel walking trajectory and the bucket wheel machine rotating trajectory around the cantilever and evaluating and optimizing the historical material taking data, material type characteristics and stockpile stability.

[0011] In a second aspect, the present application also provides a bucket wheel excavator reclaiming device based on image processing, comprising:

[0012] An acquisition module, used to acquire image information of a material pile and an initial position of a bucket wheel machine, wherein the image information of the material pile includes complete images of the material pile at least at two angles;

[0013] A first construction module is used to process the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and to perform stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile;

[0014] A trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile, and to perform trajectory planning on the initial position of the bucket wheel machine by using the real-time state in the dynamic environment model and the reinforcement learning technology to obtain the walking trajectory of the bucket wheel machine;

[0015] A prediction module, used to predict the motion state according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation track of the bucket wheel machine around the cantilever;

[0016] The second construction module is used to construct according to the bucket wheel walking trajectory and the bucket wheel machine's rotation trajectory around the cantilever, and evaluate and optimize the historical material taking data, material type characteristics and stockpile stability to obtain a material taking plan.

[0017] In a third aspect, the present application also provides a bucket wheel machine reclaiming device based on image processing, comprising:

[0018] Memory for storing computer programs;

[0019] A processor is used to implement the steps of the bucket wheel machine material reclaiming method based on image processing when executing the computer program.

[0020] In a fourth aspect, the present application further provides a medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned bucket wheel excavator material reclaiming method based on image processing are implemented.

[0021] The beneficial effects of the present invention are:

[0022] The present invention uses multi-directional convolution kernels and deep learning network technology to construct stereoscopic vision of the material pile image information, and then generates an accurate three-dimensional model of the material pile. The accurate three-dimensional model of the material pile can truly reproduce the shape and structural characteristics of the material pile; according to the three-dimensional model of the material pile and the dynamic change parameters of the material pile, a dynamic environment model is constructed, and according to the dynamic changes of the material pile, the dynamic changes of the material pile are reflected in real time, thereby realizing adaptive optimization of the walking trajectory of the bucket wheel machine. The walking trajectory of the bucket wheel machine can be flexibly moved and accurately taken according to the actual situation of the pile; a rotating trajectory of the bucket wheel machine around the cantilever is also constructed, and the walking trajectory of the bucket wheel machine is dynamically adjusted by the rotating trajectory to ensure that the bucket wheel machine can accurately locate the specific position of the pile when taking materials, thereby realizing efficient and uniform material taking operations. The construction of the rotating trajectory of the bucket wheel machine around the cantilever solves the problem of uneven material taking in the traditional bucket wheel machine taking method. The present invention solves the problems of low material taking efficiency, partial omission of piled materials or inability to effectively take materials at all.

[0023] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or be understood by implementing the embodiments of the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 It is a schematic flow chart of a bucket wheel machine material reclaiming method based on image processing described in an embodiment of the present invention;

[0026] Figure 2 It is a schematic diagram of the structure of the bucket wheel excavator reclaiming equipment based on image processing described in an embodiment of the present invention.

[0027] Markings in the figure: 800, bucket wheel excavator material taking equipment based on image processing; 801, processor; 802, memory; 803, multimedia component; 804, I / O interface; 805, communication component. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0029] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. At the same time, in the description of the present invention, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0030] Embodiment 1:

[0031] This embodiment provides a bucket wheel excavator material reclaiming method based on image processing.

[0032] See also Figure 1 , the figure shows that the method includes steps S1 to S5, including:

[0033] S1: Acquire image information of a pile of materials and an initial position of a bucket wheel machine, wherein the image information of the pile of materials includes complete images of the pile of materials at least at two angles;

[0034] In this step, the initial position of the bucket wheel excavator includes the initial position of the trolley, the initial pitch angle of the boom and the initial rotation angle. The complete image of the stockpile at at least two angles ensures multi-angle viewing angles and complete image coverage of the stockpile.

[0035] S2: Processing the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and performing stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile;

[0036] In order to clarify the specific method of obtaining the three-dimensional model of the stockpile, step S2 includes S21 to S24, which are specifically:

[0037] S21: Optimizing the deep learning network according to the multi-directional convolution kernel to obtain an optimized deep learning model;

[0038] In order to clarify the specific method of obtaining the optimized deep learning model, step S21 includes S211 to S214, which are:

[0039] S211: Acquire feature dynamic information;

[0040] In this step, the characteristic dynamic information is the characteristics of the input data, and the characteristic dynamic information is obtained through time series analysis, differential operation or other dynamic feature extraction methods.

[0041] S212: extracting features in multiple directions from the characteristic dynamic information according to the multi-directional convolution kernel to obtain multiple information features;

[0042] In this step, the feature extraction in multiple directions includes features in horizontal, vertical, diagonal and other directions to capture multi-dimensional information in the data.

[0043] The information characteristic expression is:

[0044] F=Conv(I,K) (1)

[0045] In the above formula (1), F represents information features, Conv represents convolution operation, I represents input image, and K represents multi-directional convolution kernel;

[0046] Wherein, the input image is feature dynamic information;

[0047] Through multi-directional convolution operations, features in different directions are extracted from the feature dynamic information, and using the multi-directional convolution kernel to extract features from the feature dynamic information can more comprehensively capture the structural information in the data.

[0048] S213: Integrate the plurality of information features according to a feature fusion method to obtain multi-directional local features;

[0049] S214: Perform weighted summation on the multi-directional local features, and perform weight assignment based on the feature strength and position information of the feature dynamic information in the network to obtain an optimized deep learning model.

[0050] In this step, the optimized deep learning model is:

[0051]

[0052] In the above formula (2), F 优化 represents the optimized deep learning model, Weight represents the weight distribution function, and F 融合 represents multi-directional local features, and I represents feature dynamic information;

[0053] Among them, the weighted summation and weight distribution dynamically adjust the contribution of the features according to the importance and global effect of the features, so that the optimized deep learning model pays more attention to important features.

[0054] S22: extracting features from the image in the stockpile image information based on the optimized deep learning model, and improving performance evaluation indicators in combination with an attention mechanism to obtain image features;

[0055] In this step, the image feature expression is:

[0056] F′=Attention(F) (3)

[0057] In the above formula (3), F′ represents the image feature, Attention represents the attention mechanism, and F represents the information feature;

[0058] Among them, by combining the optimized deep learning model with the attention mechanism, the key features of the pile image can be accurately extracted, the efficiency of feature extraction can be improved, redundant information can be reduced, and the model's adaptability to complex scenes can be enhanced; the performance evaluation indicators include accuracy and recall rate.

[0059] S23: performing feature point matching based on the image feature and a preset reference feature, and obtaining a target matching point by removing an erroneous matching point, wherein the reference feature is a predefined feature point;

[0060] In this step, the reference features are used as a benchmark for matching, and the removal of erroneous matching points is performed by using the RANSAC algorithm;

[0061] The target matching point is:

[0062] M=Filter(Match(F′,R)) (4)

[0063] In the above formula (4), M represents the target matching point, Filter represents the function of removing the wrong matching point, Match represents the matching function, F′ represents the image feature, and R represents the reference feature.

[0064] S24: Calculate depth information according to the target matching points and the geometric information in the material pile image information, construct a three-dimensional point cloud model through the depth information and the target matching points, and optimize the three-dimensional point cloud model through a filtering algorithm to obtain a three-dimensional model of the material pile.

[0065] In this step, the three-dimensional model of the stockpile is:

[0066] D=Depth(M,A 几何 ) (5)

[0067] In the above formula (5), D represents the depth information, Depth represents the depth information calculation function, M represents the target matching point, and A 几何 Represents geometric information;

[0068] P 点云 =PointCloud(D,M) (6)

[0069] In the above formula (6), P 点云 represents the point cloud model, PointCloud represents the point cloud construction function, D represents the depth information, and M represents the target matching point;

[0070] P=Filter(P 点云 ) (7)

[0071] In the above formula (7), P represents the three-dimensional model of the pile, Filter represents the filter function, and P 点云 Represents a point cloud model;

[0072] Wherein, the geometric information represents the geometric information in the stockpile image information;

[0073] The filtering algorithm is used to remove noise and redundant points from the three-dimensional point cloud model, making the three-dimensional model of the material pile smoother and more accurate; the three-dimensional point cloud model constructed based on depth information and target matching points can truly reflect the shape and structure of the material pile, and the three-dimensional model of the material pile solves the problem of inaccurate feature extraction in the prior art.

[0074] S3: constructing a dynamic environment model based on the three-dimensional model of the material pile and preset dynamic change parameters of the material pile, and performing trajectory planning for the initial position of the bucket wheel machine by using the reinforcement learning technology through the real-time state in the dynamic environment model to obtain the walking trajectory of the bucket wheel machine;

[0075] In order to clarify the specific method of obtaining the walking trajectory of the bucket wheel machine, step S3 includes S31 to S35, which are specifically:

[0076] S31: Acquire the bucket wheel machine execution action, where the bucket wheel machine execution action includes moving direction, moving speed and material taking depth;

[0077] S32: extracting change data based on multiple time steps in the three-dimensional model of the pile to obtain dynamic change parameters of the pile;

[0078] In this step, the multiple time steps include a first time step and a second time step. Based on the three-dimensional model of the stockpile, change data extraction is performed at multiple time steps to obtain dynamic change parameters of the stockpile.

[0079] In order to clarify the specific method of obtaining the dynamic change parameters of the stockpile, step S32 includes S321 to S323, which are:

[0080] S321: Based on the first time step in the three-dimensional model of the material pile, feature extraction is performed on geometric information in the three-dimensional model of the material pile to obtain first time step features;

[0081] In this step, the geometric information in the three-dimensional model of the stockpile is converted into quantifiable feature data.

[0082] S322: Based on the second time step in the three-dimensional model of the material pile, feature extraction is performed on geometric information in the three-dimensional model of the material pile to obtain second time step features;

[0083] S323: Calculate parameters according to the first time step characteristics and the second time step characteristics to obtain dynamic change parameters of the pile, wherein the dynamic change parameters of the pile include volume change rate, position change, height change, contour change and shape change parameters.

[0084] In this step, by calculating the first time step characteristics and the second time step characteristics and comparing the characteristic data of the two time steps, the changes of the pile during the operation of the bucket wheel excavator are dynamically reflected, thereby solving the problem that the dynamic changes of the pile are difficult to monitor and quantify in real time.

[0085] S33: constructing a dynamic environment model according to the height change and contour change in the dynamic change parameters of the pile;

[0086] In this step, the height change and the contour change are used to construct a real-time reflection of the dynamic change of the stockpile.

[0087] S34: performing parameter processing according to the moving direction, moving speed and material taking depth of the bucket wheel machine to obtain the action space of the bucket wheel machine;

[0088] S35: According to the real-time status in the dynamic environment model, the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile are input into the reinforcement learning model, and training is performed in combination with the bucket wheel machine motion space until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

[0089] In order to clarify the specific training process of the bucket wheel machine's walking trajectory, step S35 includes S351 to S353, specifically:

[0090] S351: Optimizing the reinforcement learning model in multiple dimensions according to the real-time status in the dynamic environment model and the preset actual requirements of the bucket wheel machine, and dynamically adjusting the reward weight of the reinforcement learning model in combination with a preset adaptive reward function adjustment mechanism to obtain an optimized model;

[0091] In this step, the dynamic adjustment mechanism is expressed as:

[0092] R(s,a)=α·R 原始 (s,a)+β·R 原始 (s,a) (8)

[0093] In the above formula (8), R(s,a) represents the reward obtained by taking action a in state s, α and β represent the dynamically adjusted weight coefficients, and R 原始 represents the original reward function;

[0094] Among them, the actual requirements of the bucket wheel machine include motion accuracy and energy efficiency. The reinforcement learning model can better adapt to the dynamic environment through multi-dimensional optimization. By reflecting the dynamic changes of the material pile in real time, it realizes adaptive optimization of the walking trajectory of the bucket wheel machine to solve the problem that the reinforcement learning model is difficult to take into account multi-objective optimization in a complex dynamic environment.

[0095] S352: inputting the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile into the optimization model, and training is performed in combination with the bucket wheel machine motion space, and outputting the bucket wheel machine motion according to the real-time state in the dynamic environment model;

[0096] In this step, through the training of the optimization model, the optimal action sequence that adapts to the dynamic environment can be generated. These sequences effectively guide the walking trajectory planning of the bucket wheel machine, solve the problem of efficiently generating the optimal action sequence in a complex dynamic environment, and improve the path planning efficiency and accuracy of the bucket wheel machine.

[0097] S353: Execute the output bucket wheel machine action to obtain a new state and a reward value, and update the parameters of the optimization model according to the new state and the reward value until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

[0098] In this step, by continuously updating and optimizing the optimization model, it is possible to gradually converge to the optimal solution and generate a stable bucket wheel excavator travel trajectory.

[0099] The dynamic environment model and the bucket wheel machine motion space are used to adaptively optimize the walking trajectory of the bucket wheel machine, and the walking trajectory of the bucket wheel machine can be flexibly moved and accurately taken according to the actual situation of the pile.

[0100] S4: Predicting the motion state according to the walking trajectory of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation trajectory of the bucket wheel machine around the cantilever;

[0101] In order to clarify the specific method of obtaining the trajectory of the bucket wheel machine rotating around the cantilever, step S4 includes S41 to S44, which are specifically:

[0102] S41: inputting the walking track of the bucket wheel machine, the segmentation information in the stockpile image information, and the layer height in the three-dimensional model of the stockpile into the deep learning network for error training to obtain a prediction model;

[0103] In this step, the deep learning network learns the intrinsic relationship between the walking trajectory of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, and improves the prediction accuracy of the model by minimizing the error between the predicted value and the actual value.

[0104] S42: performing trajectory prediction on the walking trajectory of the bucket wheel machine, the segmentation information in the material pile image information, and the layer height in the three-dimensional model of the material pile according to the prediction model to obtain an initial motion trajectory of the bucket wheel machine;

[0105] S43: inputting the initial motion trajectory of the bucket wheel machine into the reinforcement learning model, selecting an action in the action space of the bucket wheel machine according to the initial motion trajectory of the bucket wheel machine, and obtaining an optimal action;

[0106] In this step, the optimal action is selected in the complex bucket wheel machine action space through the reinforcement learning model, thereby optimizing the motion trajectory of the bucket wheel machine.

[0107] S44: Calculate the reward value for the optimal action according to the reward function, and when the convergence condition is reached, obtain the trajectory of the bucket wheel machine rotating around the cantilever.

[0108] In this step, the convergence condition is:

[0109] ∣R t +1-R t ∣<∈ (9)

[0110] In the above formula (9), R t represents the reward value of the tth iteration, ∈ represents the threshold.

[0111] Among them, when the absolute value of the difference between the reward values ​​of two consecutive iterations is less than ∈, the convergence state is reached, and the bucket wheel machine's swing trajectory around the cantilever is the optimal trajectory for the bucket wheel machine to efficiently and safely complete material grabbing and unloading during the cantilever rotation process; the reward function is calculated based on factors such as trajectory smoothness, energy consumption and operation time;

[0112] The bucket wheel machine's swing trajectory around the cantilever is dynamically adjusted according to the bucket wheel machine's travel trajectory, ensuring that the bucket wheel machine can accurately locate a specific position of the material pile when taking materials, thereby achieving efficient and uniform material taking operations.

[0113] S5: constructing according to the bucket wheel walking trajectory and the bucket wheel machine rotating trajectory around the cantilever, combining with the evaluation and optimization of historical material taking data, material type characteristics and stockpile stability, to obtain a material taking plan.

[0114] In order to clarify the specific method of obtaining the material extraction plan, step S5 includes S51 to S54, which are specifically:

[0115] S51: integrating the bucket wheel walking trajectory and the bucket wheel machine swinging trajectory around the boom, and simulating and constructing the integration result to obtain an initial trajectory model;

[0116] S52: testing the initial trajectory model based on preset trajectory constraints to obtain a final trajectory model;

[0117] In this step, the verification is to ensure that the initial trajectory model meets various constraints in actual operation and obtain the final trajectory model that meets the requirements;

[0118] The trajectory constraints include safety distance, speed limit, equipment capability and / or motion range, etc.

[0119] S53: A plurality of material taking parameters are obtained by fusing the historical material taking data, the material type characteristics, the stockpile stability and the final trajectory model;

[0120] In this step, more accurate material taking parameters are generated by integrating multiple factors to avoid the problem of unreasonable material taking scheme caused by a single factor;

[0121] The material type characteristics include density and friction coefficient, the pile stability includes the shape and stability of the material pile, and the material fetching parameters include the material fetching speed, material fetching angle and / or material fetching depth.

[0122] S54: formulating an initial material taking plan according to the plurality of material taking parameters, and obtaining a final material taking plan by evaluating the initial material taking plan.

[0123] In this step, the evaluation operation is performed to ensure that the final material extraction plan has high efficiency and reliability in actual operation, while reducing equipment wear and material waste;

[0124] The evaluation includes indicators such as efficiency, safety and / or energy consumption.

[0125] Embodiment 2:

[0126] This embodiment provides a bucket wheel machine reclaiming device based on image processing, the device comprising:

[0127] An acquisition module, used to acquire image information of a material pile and an initial position of a bucket wheel machine, wherein the image information of the material pile includes complete images of the material pile at least at two angles;

[0128] A first construction module is used to process the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and to perform stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile;

[0129] A trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile, and to perform trajectory planning on the initial position of the bucket wheel machine by using the real-time state in the dynamic environment model and the reinforcement learning technology to obtain the walking trajectory of the bucket wheel machine;

[0130] To clarify the specific ways to obtain the trajectory planning module, there are:

[0131] An acquisition unit, used for acquiring an execution action of a bucket wheel machine, wherein the execution action of the bucket wheel machine includes a moving direction, a moving speed and a material taking depth;

[0132] An extraction unit, used for extracting change data based on multiple time steps in the three-dimensional model of the pile to obtain dynamic change parameters of the pile;

[0133] A construction unit, used to construct according to the height change and contour change in the dynamic change parameters of the pile to obtain a dynamic environment model;

[0134] A processing unit, used for performing parameterized processing according to the moving direction, moving speed and material taking depth of the bucket wheel machine to obtain the action space of the bucket wheel machine;

[0135] A training unit is used to input the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile into the reinforcement learning model according to the real-time status in the dynamic environment model, and to perform training in combination with the bucket wheel machine motion space until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

[0136] A prediction module, used to predict the motion state according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation track of the bucket wheel machine around the cantilever;

[0137] The second construction module is used to construct according to the bucket wheel walking trajectory and the bucket wheel machine's rotation trajectory around the cantilever, and evaluate and optimize the historical material taking data, material type characteristics and stockpile stability to obtain a material taking plan.

[0138] It should be noted that, regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0139] Embodiment 3:

[0140] Corresponding to the above method embodiment, a bucket wheel excavator material reclaiming device based on image processing is also provided in this embodiment. The bucket wheel excavator material reclaiming device based on image processing described below and the bucket wheel excavator material reclaiming method based on image processing described above can be referenced to each other.

[0141] Figure 2 FIG. 8 is a block diagram of a bucket wheel machine reclaiming device 800 based on image processing according to an exemplary embodiment. Figure 2 As shown, the bucket wheel machine reclaiming device 800 based on image processing may include: a processor 801, a memory 802. The bucket wheel machine reclaiming device 800 based on image processing may also include one or more of a multimedia component 803, an I / O interface 804, and a communication component 805.

[0142] The processor 801 is used to control the overall operation of the bucket wheel machine reclaiming device 800 based on image processing to complete all or part of the steps in the bucket wheel machine reclaiming method based on image processing. The memory 802 is used to store various types of data to support the operation of the bucket wheel machine reclaiming device 800 based on image processing, and these data may include, for example, instructions for any application or method operated on the bucket wheel machine reclaiming device 800 based on image processing, and application-related data, such as contact data, sent and received messages, pictures, audio, video, etc. The memory 802 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, disk or optical disk. The multimedia component 803 may include a screen and an audio component. The screen may be, for example, a touch screen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may be further stored in the memory 802 or sent via the communication component 805. The audio component also includes at least one speaker for outputting audio signals. The I / O interface 804 provides an interface between the processor 801 and other interface modules, and the above-mentioned other interface modules can be keyboards, mice, buttons, etc. These buttons can be virtual buttons or physical buttons. The communication component 805 is used for wired or wireless communication between the bucket wheel machine reclaiming device 800 based on image processing and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component 805 can include: Wi-Fi module, Bluetooth module, NFC module.

[0143] In an exemplary embodiment, the bucket wheel excavator material reclaiming equipment 800 based on image processing can be implemented by one or more application specific integrated circuits (Application Specific Integrated Circuit, referred to as ASIC), digital signal processors (Digital Signal Processor, referred to as DSP), digital signal processing devices (Digital Signal Processing Device, referred to as DSPD), programmable logic devices (Programmable Logic Device, referred to as PLD), field programmable gate arrays (Field Programmable Gate Array, referred to as FPGA), controllers, microcontrollers, microprocessors or other electronic components to execute the above-mentioned bucket wheel excavator material reclaiming method based on image processing.

[0144] Embodiment 4:

[0145] Corresponding to the above method embodiment, a medium is also provided in this embodiment. The medium described below and the bucket wheel excavator material reclaiming method based on image processing described above can be referenced to each other.

[0146] A medium stores a computer program, and when the computer program is executed by a processor, the steps of the bucket wheel machine material taking method based on image processing in the above method embodiment are implemented.

[0147] The medium may specifically be a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or other medium that can store program codes.

[0148] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0149] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A bucket wheel excavator material taking method based on image processing, characterized in that: include: Acquire material pile image information and an initial position of a bucket wheel machine, wherein the material pile image information includes complete images of the material pile at least at two angles; Processing the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and performing stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile; A dynamic environment model is constructed based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile, and through the real-time state in the dynamic environment model, a reinforcement learning technology is used to perform trajectory planning on the initial position of the bucket wheel machine to obtain a walking trajectory of the bucket wheel machine; The motion state is predicted according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation track of the bucket wheel machine around the cantilever; The material taking plan is obtained by constructing the bucket wheel walking trajectory and the bucket wheel machine rotating trajectory around the cantilever and evaluating and optimizing the historical material taking data, material type characteristics and stockpile stability.

2. The bucket wheel excavator material taking method based on image processing according to claim 1 is characterized in that: The image in the material pile image information is processed according to a multi-directional convolution kernel and a deep learning network to obtain image features, and the image features and geometric information in the material pile image information are subjected to stereoscopic visual construction to obtain a three-dimensional model of the material pile, including: Optimizing the deep learning network according to the multi-directional convolution kernel to obtain an optimized deep learning model; Extracting features of images in the stockpile image information based on the optimized deep learning model, and improving performance evaluation indicators in combination with an attention mechanism to obtain image features; Performing feature point matching based on the image features and preset reference features, and obtaining target matching points after removing erroneous matching points, wherein the reference features are predefined feature points; Depth information is calculated according to the target matching points and geometric information in the material pile image information, a three-dimensional point cloud model is constructed through the depth information and the target matching points, and the three-dimensional point cloud model is optimized through a filtering algorithm to obtain a three-dimensional model of the material pile.

3. The bucket wheel excavator material reclaiming method based on image processing according to claim 1, characterized in that: A dynamic environment model is constructed based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile. Through the real-time state in the dynamic environment model, the reinforcement learning technology is used to perform trajectory planning on the initial position of the bucket wheel machine to obtain the walking trajectory of the bucket wheel machine, including: Acquire the bucket wheel machine execution action, wherein the bucket wheel machine execution action includes moving direction, moving speed and material taking depth; Extracting change data based on multiple time steps in the three-dimensional model of the pile to obtain dynamic change parameters of the pile; A dynamic environment model is constructed based on the height change and contour change in the dynamic change parameters of the stockpile; Parameterized processing is performed according to the moving direction, moving speed and material taking depth of the bucket wheel machine to obtain the action space of the bucket wheel machine; According to the real-time status in the dynamic environment model, the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile are input into the reinforcement learning model, and training is performed in combination with the bucket wheel machine motion space until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

4. The bucket wheel excavator material taking method based on image processing according to claim 3 is characterized in that: According to the real-time state in the dynamic environment model, the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile are input into the reinforcement learning model, and training is performed in combination with the bucket wheel machine action space until the model convergence condition is reached, and the walking trajectory of the bucket wheel machine is obtained, including: The reinforcement learning model is optimized in multiple dimensions according to the real-time status in the dynamic environment model and the preset actual requirements of the bucket wheel machine, and the reward weight of the reinforcement learning model is dynamically adjusted in combination with the preset adaptive reward function adjustment mechanism to obtain an optimized model; The initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile are input into the optimization model, and training is performed in combination with the bucket wheel machine motion space, and the bucket wheel machine motion is output according to the real-time state in the dynamic environment model; The output bucket wheel machine action is executed to obtain a new state and a reward value, and the parameters of the optimization model are updated according to the new state and the reward value until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

5. The bucket wheel excavator material taking method based on image processing according to claim 3 is characterized in that: The motion state is predicted according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile to obtain the rotation track of the bucket wheel machine around the cantilever, including: Inputting the walking track of the bucket wheel machine, the segmentation information in the stockpile image information, and the layer height in the three-dimensional model of the stockpile into the deep learning network for error training to obtain a prediction model; According to the prediction model, the walking trajectory of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile are predicted to obtain the initial motion trajectory of the bucket wheel machine; Inputting the initial motion trajectory of the bucket wheel machine into the reinforcement learning model, selecting an action in the action space of the bucket wheel machine according to the initial motion trajectory of the bucket wheel machine, and obtaining an optimal action; The reward value of the optimal action is calculated according to the reward function, and when the convergence condition is reached, the trajectory of the bucket wheel machine rotating around the cantilever is obtained.

6. The bucket wheel excavator material taking method based on image processing according to claim 1, characterized in that: According to the bucket wheel walking trajectory and the bucket wheel machine swinging trajectory around the cantilever, the material reclaiming scheme is obtained by evaluating and optimizing the historical reclaiming data, material type characteristics and stockpile stability, including: According to the bucket wheel walking trajectory and the bucket wheel machine swinging trajectory around the cantilever, an initial trajectory model is obtained by simulating and constructing the integration result; Testing the initial trajectory model based on preset trajectory constraints to obtain a final trajectory model; A plurality of material taking parameters are obtained by fusing the historical material taking data, the material type characteristics, the stockpile stability and the final trajectory model; An initial material taking plan is formulated according to the plurality of material taking parameters, and a final material taking plan is obtained by evaluating the initial material taking plan.

7. Bucket wheel excavator reclaiming device based on image processing, characterized in that: include: An acquisition module, used to acquire image information of a material pile and an initial position of a bucket wheel machine, wherein the image information of the material pile includes complete images of the material pile at least at two angles; A first construction module is used to process the image in the material pile image information according to a multi-directional convolution kernel and a deep learning network to obtain image features, and to perform stereoscopic vision construction on the image features and geometric information in the material pile image information to obtain a three-dimensional model of the material pile; A trajectory planning module is used to construct a dynamic environment model based on the three-dimensional model of the stockpile and preset dynamic change parameters of the stockpile, and to perform trajectory planning on the initial position of the bucket wheel machine by using the real-time state in the dynamic environment model and the reinforcement learning technology to obtain the walking trajectory of the bucket wheel machine; A prediction module, used to predict the motion state according to the walking track of the bucket wheel machine, the segmentation information in the material pile image information and the layer height in the three-dimensional model of the material pile, so as to obtain the rotation track of the bucket wheel machine around the cantilever; The second construction module is used to construct according to the bucket wheel walking trajectory and the bucket wheel machine's rotation trajectory around the cantilever, and evaluate and optimize the historical material taking data, material type characteristics and stockpile stability to obtain a material taking plan.

8. The bucket wheel excavator reclaiming device based on image processing according to claim 7 is characterized in that: The trajectory planning module comprises: An acquisition unit, used for acquiring an execution action of a bucket wheel machine, wherein the execution action of the bucket wheel machine includes a moving direction, a moving speed and a material taking depth; An extraction unit, used for extracting change data based on multiple time steps in the three-dimensional model of the pile to obtain dynamic change parameters of the pile; A construction unit, used to construct according to the height change and contour change in the dynamic change parameters of the pile to obtain a dynamic environment model; A processing unit, used for performing parameterized processing according to the moving direction, moving speed and material taking depth of the bucket wheel machine to obtain the action space of the bucket wheel machine; A training unit is used to input the initial position of the bucket wheel machine and the target position in the three-dimensional model of the stockpile into the reinforcement learning model according to the real-time status in the dynamic environment model, and to perform training in combination with the bucket wheel machine motion space until the model convergence condition is reached to obtain the walking trajectory of the bucket wheel machine.

9. Bucket wheel excavator reclaiming equipment based on image processing, characterized in that: include: Memory for storing computer programs; A processor is used to implement the steps of the bucket wheel machine material reclaiming method based on image processing as described in any one of claims 1 to 6 when executing the computer program.

10. A medium, characterized in that : A computer program is stored on the medium, and when the computer program is executed by the processor, the steps of the bucket wheel machine material reclaiming method based on image processing as described in any one of claims 1 to 6 are implemented.

Images