Excavator material processing system based on self-adaptive control and excavator

Abstract

The embodiment of the invention provides an excavator material processing system based on self-adaptive control and an excavator. The system comprises a material identification module, a control module and a self-adaptive material processing module, wherein the material identification module is used for collecting three-dimensional point cloud data, image data and temperature data of a target material; obtaining a fusion feature vector of the target material according to the three-dimensional point cloud data, the image data and the temperature data; the control module is used for determining the physical characteristics of the target material according to the fusion feature vector and a preset material feature library; and the self-adaptive material processing module is used for generating a control instruction based on the physical characteristics and a preset operation rule base, and processing the target material according to the control instruction. By means of the system, the material processing speed and accuracy are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of excavating equipment and multi-sensor fusion, and particularly relates to an excavator material processing system based on adaptive control and an excavator. BACKGROUND

[0002] In the field of modern engineering construction, excavators, as a kind of key construction equipment, are widely used in earth excavation, material handling and other operation scenarios. With the continuous expansion of the scale of engineering projects and the increasing requirements of construction, the operation performance, operation precision and intelligent level of excavators have become important indicators for judging the construction efficiency and safety level. Efficient, accurate and stable excavator operation can not only significantly reduce construction costs and shorten the construction period, but also effectively reduce resource waste and environmental impact, helping to complete construction projects with high quality, high standards and high efficiency. Therefore, research on the optimization of excavator operation performance and intelligent control technology has become a focus of the engineering machinery industry.

[0003] In the prior art, the control method of the excavator mostly uses a single sensor to realize scene judgment or material recognition. In the subsequent operation process, the operation system controls according to preset working condition parameters (such as fixed digging depth, force, bucket angle, etc.), adjusts the feedback signals (such as pressure, position, speed) in real time through classical algorithms such as PID, to ensure the stability and precision of equipment operation, avoid overload or instability, and ensure the completion of the intended operation task.

[0004] However, the control method of the excavator in the prior art has the problems of insufficient material recognition capability, poor material processing stability and low efficiency. SUMMARY

[0005] The excavator material processing system based on adaptive control and the excavator provided by the embodiments of the present application are used to solve the problems of insufficient material recognition capability, poor material processing stability and low efficiency in the prior art.

[0006] In a first aspect, the embodiments of the present application provide an excavator material processing system based on adaptive control, comprising:

[0007] a material recognition module, a control module and an adaptive material processing module;

[0008] The material recognition module is configured to collect three-dimensional point cloud data, image data and temperature data of a target material, and obtain a fusion feature vector of the target material according to the three-dimensional point cloud data, the image data and the temperature data.

[0009] The control module is configured to determine the physical properties of the target material according to the fusion feature vector and a preset material feature library.

[0010] The adaptive material processing module is configured to generate a control instruction based on the physical characteristics and a preset operation rule library, and process the target material according to the control instruction.

[0011] The control module is specifically configured to:

[0012] determine the material characteristics corresponding to the fusion feature vector through a pre-trained material judgment model;

[0013] compare the material characteristics with material characteristic rules in the preset material characteristic library, and determine the physical characteristics corresponding to the fusion feature vector when the comparison result is that there is no rule conflict; wherein the preset material characteristic library includes physical attribute constraint rules, type characteristic mapping rules and abnormal value filtering rules.

[0014] In one possible implementation, the material recognition module includes a data acquisition unit and a data fusion unit.

[0015] The data acquisition unit is configured to acquire three-dimensional point cloud data, image data and temperature data of the target material; and the data fusion unit is configured to pre-process the three-dimensional point cloud data, the image data and the temperature data respectively, and perform feature extraction and feature weighted fusion processing on the pre-processed three-dimensional point cloud data, the pre-processed image data and the pre-processed temperature data to obtain a fusion feature vector of the target material.

[0016] In one possible implementation, the data acquisition unit includes a laser radar sensor, a camera and an infrared sensor.

[0017] The laser radar sensor is arranged at one end of a boom of an excavator connected to a dipper stick, and is configured to acquire the three-dimensional point cloud data of the target material.

[0018] The number of cameras is at least three, and each camera is arranged at a different position of the excavator to acquire the image data of the target material from different angles.

[0019] The infrared sensor is arranged at a dipper link bracket position of the excavator to acquire temperature data of a surface of the target material.

[0020] In one possible implementation, the data fusion unit is specifically configured to:

[0021] The three-dimensional point cloud data is denoised and processed and outliers are removed to obtain the preprocessed three-dimensional point cloud data; the image data is processed for image enhancement and distortion correction to obtain the preprocessed image data; the temperature data is normalized to obtain the preprocessed temperature data;

[0022] The geometric features corresponding to the preprocessed three-dimensional point cloud data, the visual features corresponding to the preprocessed image data, and the temperature features corresponding to the preprocessed temperature data are extracted through a feature fusion model, the weights corresponding to the geometric features, the visual features, and the temperature features are adaptively adjusted, and the geometric features, the visual features, and the temperature features are processed for feature fusion based on the weights, to obtain the fusion feature vector of the target material.

[0023] In a possible implementation, the material judgment model is obtained by training an initial model based on a plurality of batches of training samples generated by using a random sampling strategy on the collected multi-modal feature data, standard sample data, construction site measured data, and physical simulation data.

[0024] In a possible implementation, the adaptive material processing module includes an instruction generation unit and an instruction execution unit.

[0025] The instruction generation unit is configured to match the physical characteristics with the preset operation rule library, and generate a control instruction according to a matching result; the preset operation rule library includes an operation rule physical characteristic correspondence relationship;

[0026] The instruction execution unit is configured to determine a target implement, a digging intensity, a digging speed, a bucket opening and closing angle, a grabbing claw action, and a material processing mode according to the control instruction, and process the target material using the target implement, the digging intensity, the digging speed, the bucket opening and closing angle, the grabbing claw action, and the material processing mode.

[0027] In a possible implementation, when the instruction generation unit generates a control instruction according to a matching result, the instruction generation unit is specifically configured to:

[0028] When the matching result indicates that one of the operation rules in the preset operation rule library matches the physical characteristics successfully, the control instruction is generated according to the operation rule;

[0029] When the matching result indicates that multiple operation rules in the preset operation rule library are successfully matched with the physical characteristics, the multiple operation rules are sorted according to preset weights corresponding to the multiple operation rules respectively, and the corresponding control instructions are generated in sequence according to the order of the multiple operation rules.

[0030] In a possible implementation, the instruction execution unit includes a tool library and an electrically-controlled hydraulic wrist arranged at the end of the excavator arm;

[0031] The electrically-controlled hydraulic wrist is configured to select the target tool from the tool library according to the control instruction, and replace the tool of the excavator with the target tool; and after the target tool is replaced, the target material is processed by the digging intensity, the digging speed, the bucket opening and closing angle, the grabbing claw action, and the material processing mode.

[0032] In a second aspect, the embodiments of the present application provide an excavator, which is provided with the excavator material processing system based on adaptive control as described in the first aspect and / or various possible implementations of the first aspect.

[0033] The embodiments of the present application provide an excavator material processing system based on adaptive control and an excavator. The system includes a material identification module, a control module, and an adaptive material processing module, which realizes intelligent identification and efficient processing of different types of materials. First, the material identification module uses sensors to collect three-dimensional point cloud data, image data, and temperature data of the target material, extracts information representing the material type, shape, texture, and thermal characteristics through a multi-modal fusion algorithm (such as an attention mechanism or a feature concatenation network), and generates a fusion feature vector. Next, the control module performs similarity matching between the material characteristics corresponding to the fusion feature vector and standard material samples in a preset material feature library, accurately determines the physical characteristics of the material, including hardness, humidity, flammability, size, and other key parameters, and provides a basis for subsequent operations. Subsequently, the adaptive material processing module automatically matches the optimal processing strategy according to the identification result and a preset operation rule library, generates control instructions, including selecting appropriate tools (such as crushing pincers, grab buckets, or shovels), setting processing parameters (such as intensity, angle, and frequency), and completing corresponding processing operations through a control actuator. At the same time, the module has self-feedback adjustment capability, dynamically adjusts operation parameters by monitoring force feedback or displacement information in the processing process in real time, and ensures the stability and safety of the processing effect. The entire system realizes closed-loop control from material perception, intelligent identification to fine processing, significantly improving the adaptive ability, processing efficiency, and resource utilization rate of the excavator in complex field operations. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0035] Figure 1 A structural schematic diagram of an excavator provided for an embodiment of the present application;

[0036] Figure 2 A structural schematic diagram of an excavator material handling system based on adaptive control provided for an embodiment of the present application;

[0037] Figure 3 A signaling diagram of an excavator material handling system based on adaptive control provided for an embodiment of the present application.

[0038] Through the above drawings, the specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the inventive concept in any way, but to illustrate the inventive concept by reference to specific embodiments. DETAILED DESCRIPTION

[0039] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, like reference numerals refer to like elements, unless the context clearly dictates otherwise. The following description of exemplary embodiments is not intended to represent all embodiments in accordance with the present application. Rather, they are merely examples in accordance with some aspects of the present application as detailed in the appended claims.

[0040] In the field of modern engineering construction, excavators, as a kind of key construction equipment, are widely used in earth excavation, material handling and other operation scenarios. With the continuous expansion of the scale of engineering projects and the increasing requirements of construction, the operation performance, operation precision and intelligent level of excavators have become important indicators to judge the construction efficiency and safety level. Efficient, accurate and stable excavator operation not only can significantly reduce construction cost and shorten construction period, but also can effectively reduce resource waste and environmental impact, and help to complete construction projects with high quality, high standard and high efficiency. Therefore, the research on the optimization of excavator operation performance and intelligent control technology has become the focus of the engineering machinery industry.

[0041] In the prior art, the control method of the excavator mostly uses a single sensor to realize scene judgment or material recognition. In the subsequent operation process, the operation system controls according to the preset working condition parameters (such as fixed digging depth, force, bucket angle, etc.), adjusts the feedback signals (such as pressure, position, speed) in real time through classical algorithms such as PID, so as to ensure the stability and precision of equipment action, avoid overload or instability, and ensure the completion of the established operation task.

[0042] However, the control method of the excavator in the prior art cannot accurately identify different types, hardness and shapes of materials by a single sensor, resulting in the need for frequent manual adjustment by the operator during operation, reducing work efficiency, and easily causing equipment damage due to judgment errors; and for complex application scenarios, irregularly shaped materials, the traditional single tool cannot achieve stable and efficient processing, and the material may fall off during processing, affecting construction safety and efficiency.

[0043] Based on this, the present application proposes an excavator material processing system based on adaptive control, which aims at the problems of the excavator material processing method in the prior art relying on manual experience, the processing parameters being unable to be dynamically adjusted, and poor adaptability to unknown materials. The present application starts from the complex and frequently changing material types in the actual operation site, and proposes an excavator material processing system based on adaptive control. The technical concept is derived from the behavior characteristics of human operators who flexibly adjust the operation mode according to visual, tactile and other sensory information when processing different materials. By fusing multi-source sensor (such as three-dimensional point cloud, image, temperature) data, extracting multi-dimensional fusion features of the target material, and combining a pre-set material feature library, the system realizes accurate identification of the physical characteristics of the material. Then, through the control module, the operation rule library is matched according to the identification result, and adaptive control strategies and parameter instructions are intelligently generated to drive the processing module to select the processing mode as needed and dynamically adjust. The system not only improves the identification ability and processing flexibility of the excavator when facing different types of materials, but also can perceive feedback in real time during processing, further optimizing the operation strategy, thereby achieving the technical effects of improving material processing efficiency, reducing the misoperation rate and prolonging the service life of the equipment.

[0044] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0045] Figure 1 The excavator structure schematic diagram provided for the embodiments of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the excavator includes: an excavator main body 101; a control system 102; a camera 103; a laser radar sensor 104; an infrared sensor 105; a multifunctional universal quick-change electric hydraulic wrist 106; and a tool library 107.

[0046] When the excavator body 101 approaches the material, the sensors in the material identification module start working, the laser radar 104 emits a laser beam to scan the material, and the three-dimensional point cloud data of the material is obtained; the camera 103 takes an image of the material; the infrared sensor 105 detects the temperature of the material, and then these data are transmitted to the intelligent control module in real time, so that the control system 102 can select the appropriate tool in the tool library 107 according to different scenes and materials, and use the multifunctional universal quick-change electric hydraulic wrist 106 to adaptively process the material.

[0047] The laser radar sensor 104 is arranged at one end of the boom connected with the dipper arm of the excavator, and is used to obtain the three-dimensional point cloud data of the target material; the number of cameras 103 is at least three, and each camera is arranged at a different position of the excavator, and is used to obtain image data of the target material from different angles; the infrared sensor 105 is arranged at the position of the dipper link bracket of the excavator, and is used to collect temperature data of the surface of the target material.

[0048] It should be noted that high-precision and high-resolution laser radar sensors are selected in this application to ensure that the three-dimensional point cloud data of the material can be quickly and accurately obtained under complex working conditions. The laser radar is installed at the front end of the boom of the excavator, so that it can clearly scan the material in the working area. At the same time, multiple high-definition cameras are provided and distributed at different positions of the excavator to obtain multi-angle image information of the material. The camera needs to have good dustproof and waterproof performance to adapt to the harsh environment of the construction site. The infrared sensor is installed at the position of the dipper link bracket to detect the temperature distribution of the surface of the material at close range and high precision.

[0049] It should also be noted that in order to realize the rapid and stable transmission of sensor data, a high-speed data transmission bus is used to connect the laser radar, camera and infrared sensor to the control module. In the control module, a high-performance data processing platform is built, and a multi-core processor, a large-capacity memory and a high-speed solid state disk are configured to ensure that a large amount of sensor data can be processed in real time. At the same time, a special data processing software is developed to realize real-time acquisition, preprocessing, fusion and analysis of sensor data, and to provide accurate and reliable material feature data for the control module.

[0050] Figure 2 A structural schematic diagram of the excavator material processing system based on adaptive control provided by the embodiment of the application is shown in Figure 2 As shown in the figure, the system comprises a material identification module, a control module and an adaptive material processing module; wherein the material identification module comprises a data acquisition unit and a data fusion unit; the data acquisition unit comprises a laser radar sensor, a camera and an infrared sensor; the adaptive material processing module comprises an instruction generation unit and an instruction execution unit.

[0051] The material recognition module is responsible for multi-modal perception and feature extraction of the material to be processed, which is the basis for the system to realize intelligent judgment. The module is composed of a data acquisition unit and a data fusion unit. The data acquisition unit is equipped with a laser radar sensor, multiple cameras, and an infrared sensor, which are used to collect three-dimensional point cloud data, image data, and temperature data of the target material, respectively. The sensor layout has been optimized, for example, the laser radar is installed at the connection between the excavator arm and the bucket rod, the cameras are arranged at multiple angles on the machine body to cover the side view angles of the material, and the infrared sensor is set at the bucket link support position for accurate collection of surface temperature. These raw data first undergo a preprocessing process in the data fusion unit: the point cloud data is improved in spatial structure accuracy through denoising and outlier removal, the image data is optimized in image quality through enhancement and distortion correction, and the temperature data is normalized to unify the dimension. Then, the geometric features, visual features, and temperature features corresponding to the three data sources are extracted through a feature fusion model, and the features are weighted and fused according to the weights learned by the model, finally generating a high-dimensional fusion feature vector representing the overall appearance of the target material.

[0052] The control module is responsible for converting the fusion feature vector into physical characteristics, and is the bridge connecting perception and decision-making. The core logic of the control module is to first use the material judgment model to judge the fusion feature vector and obtain the material characteristics, and then match the material characteristics with the preset material feature library. The material feature library is a structured knowledge base that covers the typical physical characteristics of various materials (such as hardness, density, moisture content, etc.) and the associated rule system, including three types of rules: physical attribute constraint rules (describing the logical relationship between physical variables), type feature mapping rules (defining the feature combination that a certain type of material needs to meet), and outlier filtering rules (used to eliminate unreasonable identification results). The system verifies the rationality of the fusion features through rule comparison to ensure that the results have no logical conflicts and avoid misidentification caused by perception errors. After verification, the physical characteristic parameters of the material corresponding to the fusion feature vector are output, providing a basis for the selection of subsequent operation rules and the generation of action instructions.

[0053] The adaptive material processing module is the core of the system, responsible for selecting the optimal processing strategy according to the material characteristics and converting it into specific operation actions. The module consists of an instruction generation unit and an instruction execution unit. The instruction generation unit matches the physical characteristics output by the control module with the pre-set operation rule library to find the matching operation rules. If only one rule matches, the control instruction corresponding to the rule is generated directly; if multiple rules match, the rules are sorted according to their pre-set weights, and the weighted average or constraint optimization method is used to fuse the influence of each rule on the same operation parameter, and finally the fine control instruction containing the digging force, speed, angle, tool type, etc. is generated. Then, the instruction execution unit calls the target tool in the tool library according to the control instruction, and realizes automatic tool replacement through the electrically controlled hydraulic wrist installed at the end of the bucket rod. After the replacement is completed, the system executes the processing action according to the set parameters, such as selecting a crusher for hard ore and automatically adjusting the crushing force and frequency, selecting a grabber for metal or wood recyclables for precise grabbing and storage, and using a standard shovel for ordinary earthwork or construction waste transportation. In addition, the processing module has real-time feedback capability, continuously monitors key parameters (such as crushing particle size, processing force feedback) during processing, and adjusts dynamically according to pre-set thresholds to ensure that the processing action is efficient, stable, and safe.

[0054] Through the synergistic effect of the above three modules, the system can quickly and accurately identify materials in unknown or complex operating environments, and autonomously select and accurately execute appropriate processing methods according to their physical characteristics and industry rules, greatly improving the automation level, operating efficiency and material utilization rate of excavation operations.

[0055] Embodiments of the present application provide an adaptively controlled excavator material handling system and excavator. The system includes a material identification module, a control module, and an adaptive material handling module, enabling intelligent identification and efficient processing of different types of materials. First, the material identification module uses sensors to collect 3D point cloud data, image data, and temperature data of the target material. Using a multimodal fusion algorithm (such as an attention mechanism or a feature splicing network), it extracts information characterizing the material's type, shape, texture, and thermal properties, generating a fused feature vector. Next, the control module performs a similarity match between the material characteristics corresponding to the fused feature vector and standard material samples in a pre-set material feature library, accurately determining the material's physical properties, including key parameters such as hardness, moisture, flammability, and size, to inform subsequent operations. Subsequently, the adaptive material handling module automatically matches the identification results with a pre-set operational rule library to the optimal handling strategy, generating control instructions that include selecting appropriate attachments (such as crushing jaws, grabs, or buckets), setting handling parameters (such as force, angle, and frequency), and controlling actuators to complete the corresponding handling operations. The module also features self-feedback regulation, dynamically adjusting operating parameters through real-time monitoring of force feedback or displacement information during the processing process to ensure stable and safe processing results. The entire system implements closed-loop control from material sensing and intelligent identification to refined processing, significantly improving the excavator's adaptability, processing efficiency, and resource utilization in complex on-site operations.

[0056] Figure 3 The signaling of the excavator material handling system based on adaptive control provided in the embodiment of the present application Figure 1 ;like Figure 3 As shown, Figure 3 is Figure 1 、 Figure 2 Based on this, the adaptive control process of excavator material handling is described in detail, which includes:

[0057] S301 , a data acquisition unit acquires three-dimensional point cloud data, image data, and temperature data of a target material.

[0058] In one feasible method, a lidar sensor is arranged at one end of the excavator's boom connected to the dipper arm, for obtaining the three-dimensional point cloud data of the target material; the number of cameras is at least three, which are respectively arranged at different positions of the excavator, for obtaining image data of the target material from different angles; and an infrared sensor is arranged at the bucket position of the excavator, for collecting temperature data on the surface of the target material.

[0059] It should be understood that in the present system, by setting multiple types of sensors to collect data on the target material, the laser radar is used to obtain the spatial structure information of the material (three-dimensional point cloud data), the camera obtains image information from multiple angles to provide the visual features of the material, and the infrared sensor is used to collect surface temperature information to reflect the thermal characteristics of the material. The principle of this multi-modal perception design is to supplement each other's information blind spots through different types of data to achieve comprehensive and accurate perception of the shape, material and thermal characteristics of the material.

[0060] It can be understood that through the above sensor fusion scheme, multi-dimensional, multi-angle and multi-physical quantity comprehensive perception of the target material is achieved, and recognition errors caused by single sensor due to angle obstruction, data loss or signal noise are avoided. The design improves the accuracy and robustness of material recognition, provides data guarantee for subsequent intelligent operation strategies based on material characteristics, and thus improves the precision, efficiency and self-adaptive ability of the excavator operation.

[0061] S302, the data acquisition unit sends the three-dimensional point cloud data, image data and temperature data of the target material to the data fusion unit.

[0062] It can be understood that after completing the perception task, the data acquisition unit uniformly sends all the perception data to the data fusion unit to provide rich and reliable raw input data for subsequent material type recognition and operation strategy generation.

[0063] S303, the data fusion unit pre-processes the three-dimensional point cloud data, image data and temperature data, and performs feature extraction and feature weighted fusion processing on the pre-processed three-dimensional point cloud data, pre-processed image data and pre-processed temperature data to obtain a fusion feature vector of the target material.

[0064] In an implementable manner, the data fusion unit performs noise reduction processing and outlier removal processing on the three-dimensional point cloud data to obtain pre-processed three-dimensional point cloud data, performs image enhancement processing and distortion correction processing on the image data to obtain pre-processed image data, and performs normalization processing on the temperature data to obtain pre-processed temperature data.

[0065] Through the feature fusion model, the geometric features corresponding to the pre-processed three-dimensional point cloud data, the visual features corresponding to the pre-processed image data, and the temperature features corresponding to the pre-processed temperature data are extracted, the weights corresponding to the geometric features, the visual features and the temperature features are adaptively adjusted, and the geometric features, the visual features and the temperature features are processed based on the weights to obtain a fusion feature vector of the target material.

[0066] It should be understood that in this embodiment, the data fusion unit is intended to extract effective information about the material from the multimodal sensor data to achieve more accurate material identification and characterization. First, the three-dimensional point cloud data is subjected to noise reduction and outlier removal processing in order to remove noise introduced by sensor accuracy or environmental interference and improve the geometric structure quality of the point cloud. Image enhancement and distortion correction processing are performed on the image data in order to enhance the texture and morphological features of the target material in the image and eliminate geometric distortion during the imaging process. The temperature data is normalized in order to eliminate the deviation caused by the difference in sensor dimensions and make it comparable and fusible.

[0067] Next, a feature fusion model extracts geometric features from the 3D point cloud, visual features from the image, and thermal features from the temperature data. These heterogeneous features are then weighted and fused based on a weight distribution mechanism. This model adaptively adjusts the contribution of each feature based on the needs of the specific task, thereby constructing a more discriminative fused feature vector.

[0068] For example, the feature matrix of multi-source data fusion can be expressed as:

[0069]

[0070] Among them, the feature vector F obtained by the lidar sensor li It can be expressed as:

[0071]

[0072] Where, is the coordinate of the center of mass; is the length, width and height of the bounding box; is the point cloud density; is the point cloud standard deviation; is the eigenvalue of the point cloud covariance matrix; C1, C2, C3 are shape descriptors; is the main direction angle of the point cloud.

[0073] Feature vector obtained using the camera for:

[0074]

[0075] Where, is the scale-invariant feature vector after dimensionality reduction by principal component analysis; c is the semantic segmentation category probability vector; is the mean and standard deviation of RGB channels; is the image entropy; for texture energy; is the contour complexity; The main direction angle.

[0076] Feature vector obtained by using the infrared sensor is:

[0077]

[0078] wherein, is the mean and standard deviation of temperature; is the highest / lowest temperature; is the hotspot region center coordinate; is the temperature centroid coordinate; is the maximum temperature gradient; is the hotspot region proportion; is the temperature gradient direction entropy; is the thermal contrast.

[0079] S304, the data fusion unit sends the fused feature vector to the control module.

[0080] S305, the control module obtains the material characteristics corresponding to the fused feature vector by using the material judgment model, and compares the material characteristics with the material characteristic rules in the preset material characteristic library. When there is no rule conflict in the comparison result, the physical characteristics corresponding to the fused feature vector are determined.

[0081] Optionally, the material judgment model is obtained by using a random sampling strategy to collect multi-modal feature data, standard sample data, construction site measured data and physical simulation data, generating multiple batches of training samples, and training the initial model based on the training samples of each batch.

[0082] It should be noted that, in order to improve the generalization performance and practicability of the material judgment model, the model adopts a random sampling strategy in the training stage to construct multiple batches of training samples from multiple sources of data (such as actually collected multi-modal data, standard material sample data, construction site measured data and physically simulated generated data). This strategy ensures that the model can learn the feature fusion rules with universality and robustness in a diversified, noisy and complex environment with strong scene uncertainty. Through iterative training of multiple batches of data, the model gradually optimizes the feature weighting and fusion parameters, thereby achieving the final goal of accurate identification and judgment of the target material.

[0083] Specifically, the training data acquisition approach of the material judgment model includes standard material sample data (standard sample collection), construction site measured data, and physical simulation data. The standard sample collection process is to collect standard samples of typical materials (such as granite, clay, construction waste, etc.); the hardness is measured using a pressure testing machine to obtain standard hardness data, and the density is measured using the drainage method to calculate the standard density; in a controllable environment, 10,000 sets of material data are collected simultaneously using a laser radar, an industrial camera, and an infrared thermal imager for model learning of the corresponding relationship between the standard characteristics and physical properties of different typical materials.

[0084] The construction site measured data is collected under different geological conditions (such as sandy soil, rock stratum), and different operation scenarios (such as excavation, loading); real-time calibration: the material excavation resistance is measured in real time through the mechanical arm force sensor to deduce the material hardness. (In actual construction, the excavation resistance data can be obtained in real time through the force sensor of the mechanical arm or the bucket, which can accurately deduce the hardness of the material on site, form a more reliable label, and avoid subjective misjudgment.

[0085] The physical simulation data is used to make up for the extreme working condition data (such as super-hard ore) that is difficult to collect, high in cost, or dangerous in reality, enrich the data diversity, and improve the robustness; a virtual excavation scene is constructed using PyBullet; based on the Hertz contact theory (describing the contact deformation relationship of objects) and the Mohr-Coulomb failure criterion (describing the material failure characteristics), a physical model of materials with different hardness / density is established; simulation samples are generated by randomly perturbing material parameters.

[0086] It can be understood that the training samples generated by the above method can effectively enhance the generalization ability of the model.

[0087] After obtaining the training data, the data is aligned to a unified space-time coordinate system for data preprocessing, and is standardized / encoded to form a unified input format, then a random sampling strategy is used to generate samples in batches, each batch contains 64 samples, then the model is trained, the Adam optimizer is used, the initial learning rate is 0.001, the cosine annealing learning rate scheduling is used, the training is stopped when the validation set loss does not decrease for 10 consecutive rounds, finally 5-fold cross-validation is used to fuse multiple optimal models to improve stability, and a trained material judgment model is obtained.

[0088] It should be understood that after a large amount of actual excavator operation data is trained, the material judgment model can automatically learn the multi-sensor fusion feature mode of various materials under different working conditions, and then adaptively adjust the fusion strategy and parameters according to the real-time collected data. For example, in the fusion stage, the advantages of weighted average method, Kalman filtering algorithm, and D-S evidence theory are comprehensively used, and the algorithm weight is dynamically adjusted according to the material characteristics and operation scene.

[0089] For example, when identifying materials with obvious surface temperature differences, the fusion weight of the D-S evidence theory based on infrared sensor data is enhanced; in high-precision material position operations, the role of Kalman filtering algorithm in processing lidar data is increased, thereby realizing optimal fusion of material information.

[0090] Therefore, when encountering new materials or complex working conditions, the neural network can quickly analyze data characteristics, dynamically change the contribution proportion of each sensor data in the fusion result, and improve the accuracy and reliability of material identification. The deep learning optimization process is closely combined with the entire multi-sensor fusion architecture and other algorithms to form an organic and adaptive intelligent data fusion system.

[0091] In an implementable manner, the control module compares the material characteristics with the material characteristic rules in the preset material characteristic library, and determines the physical characteristics corresponding to the fusion characteristic vector when there is no rule conflict in the comparison result.

[0092] The preset material characteristic library includes physical property constraint rules, type characteristic mapping rules, and abnormal value filtering rules.

[0093] It can be understood that after the control module receives the characteristic fusion vector from the material identification module, the material characteristics are quickly and preliminarily judged through the material judgment model, and finally the judgment result is verified and corrected according to the material characteristic rules (i.e., the established rules of the expert system), to ensure the accuracy and reliability of the judgment.

[0094] It should be noted that the essence of the established rules of the expert system is the structured expression of domain knowledge, and the core is to establish the causal relationship or constraint condition of "material characteristics, material properties / type". These rules need to cover the internal logic, physical laws or industry consensus among material characteristics, including physical property constraint rules, inherent rules between material physical characteristics, defining constraint relationships that must be true or must not be true; type characteristic mapping rules, based on the correspondence between material types and core characteristics, defining the feature combination that a certain type of material must satisfy; abnormal value filtering rules, aiming at unreasonable results that may be output by deep learning (such as feature conflicts caused by data noise), defining absolute exclusion rules that cannot be established. The rules are mainly set by artificial setting and supplemented by data driving. Artificial setting is based on the experience and literature of domain experts, which is converted into structured rules. The rule library is continuously optimized with the system running, and is updated according to the error cases in actual operation. The verification and correction core is a closed-loop process of inputting the results to be verified, rule matching, conflict detection, result optimization, and feedback iteration.

[0095] S306, the control module sends the physical characteristics to the instruction generation unit.

[0096] S307, the instruction generation unit matches the physical characteristics with the preset operation rule library, and generates a control instruction according to the matching result.

[0097] It should be noted that the process of generating the control instruction is essentially a process of mapping the material characteristic space to the actuator action space, and the core is to establish the mapping relationship between the physical characteristics and the optimal operation parameters. The preset rule is constructed by combining domain knowledge and data driven, and a theoretical calculation model is established according to the material mechanics characteristics and the kinematics principle of the machine, the operation experience of the domain expert is converted into executable rules, and the parameters of the preset rule are optimized through historical data mining and machine learning algorithm.

[0098] In an implementable manner, when the matching result indicates that there is one operation rule in the preset operation rule library that matches the physical characteristics successfully, the control instruction is generated according to the operation rule.

[0099] It should be understood that according to the accurate judgment of the physical characteristics, the instruction generation unit generates a series of dynamic and accurate operation instructions in real time according to the preset operation rule and algorithm. These instructions comprehensively cover the selection of the excavator's tool, the digging intensity, the speed, the bucket opening angle, the grabbing claw action, and the material processing method, etc. For example, in the face of high hardness ore, the instruction will automatically increase the digging intensity, and at the same time optimize the cutting angle and digging speed of the bucket, so as to improve the digging efficiency and reduce the equipment loss; for loose material, the instruction will correspondingly reduce the digging intensity and increase the digging speed to ensure efficient and safe operation.

[0100] In another implementable manner, when the matching result indicates that there are multiple operation rules in the preset operation rule library that match the physical characteristics successfully, the multiple operation rules are sorted according to the preset weights corresponding to the multiple operation rules respectively, and the corresponding control instructions are generated in turn according to the order of the multiple operation rules.

[0101] For example, the multiple operation rules of the preset operation rule library are shown in Table 1:

[0102] Table 1: Preset operation rule library

[0103]

[0104] The physical characteristics of a certain material are: high humidity (0.82), medium particle size (avg: 12mm), high adhesion coefficient (0.91), and high density (1.65g / cm³). It can be seen that the input physical characteristics trigger three rules R1, R2 and R3 at the same time, and the order from high to low according to the weight is:

[0105] ;

[0106] The propulsion force F is weightedly averaged, and the bucket angle θ is weightedly averaged:

[0107]

[0108] Alternatively, the parameters can be optimized by constraint optimization, and the corresponding control parameters are generated by the established optimization target, the optimal action path is calculated by dynamic programming, and the energy consumption and time are minimized. The specific generation method can be determined according to the actual situation, and the embodiments of the present application are not limited herein.

[0109] S308, the instruction generation unit sends the control instruction to the instruction execution unit.

[0110] S309, the instruction execution unit determines the target tool, the digging intensity, the digging speed, the bucket opening and closing angle, the grabbing claw action and the material processing method according to the control instruction; and processes the target material using the target tool with the digging intensity, the digging speed, the bucket opening and closing angle, the grabbing claw action and the material processing method.

[0111] The instruction execution unit includes a tool library and an electrically controlled hydraulic wrist arranged at the end of the excavator boom. The electrically controlled hydraulic wrist is a multifunctional universal quick-change electrically controlled hydraulic wrist, which can realize the rotation of the tool around the X axis swing, realize the rotation of the tool around the Z axis turn, can quickly change the working tool, is equipped with an auxiliary claw and a standby oil circuit to realize auxiliary grabbing and special tool operation; at the same time, a multifunctional tool library is provided on the vehicle, the appropriate tool is selected according to the system material identification and application scene result, and the target material is processed with the digging intensity, the digging speed, the bucket opening and closing angle, the grabbing claw action and the material processing method after the target tool is changed.

[0112] The tool library is personalized combined according to common application scenes, and can include a bucket, a flat ground bucket, a cable bucket, a breaking hammer, a soil loosener, a material grabber, a cutter, a land leveler, etc. In actual use, the appropriate tool can be selected according to the system material identification and application scene result.

[0113] In addition, the multifunctional universal quick-change electrically controlled hydraulic wrist uses the closed-loop control or proportional valve technology of the hydraulic system to accurately adjust the rotation angle, clamping force and other parameters. The pressure oil is provided by the hydraulic pump, and the flow direction, pressure and flow of the oil are adjusted by the electrically controlled valve group. The intelligent control module sends instructions, the electrically controlled valve group acts, and the hydraulic oil enters the corresponding executing mechanism (rotary motor, clamping cylinder, swing cylinder) to act, which can realize the rotation of the tool around the X axis swing, realize the rotation of the tool around the Z axis turn, can quickly change the working tool, realize intelligent multifunctional operation of the excavator, and is equipped with an auxiliary claw and a standby oil circuit to realize auxiliary grabbing and special tool operation.

[0114] It should be noted that after execution according to the generated control instructions, the actual effect is monitored in real time by the force sensor and the position sensor of the actuator, and compared with the expected parameters. If the deviation exceeds the threshold, the correction is triggered, and when an abnormal scene is detected, the emergency plan is immediately called.

[0115] Then, according to the example given in the above embodiment, the excavator executes according to the above parameters, and the force sensor and the position sensor are read in real time and compared with the expected value: if the actual F = 42 kN, which exceeds the target 34.6 kN ± 10%, the correction module is triggered to re-optimize; if the ground adhesion is abnormal (such as detecting slip), the emergency rule (such as pressure reduction, arm contraction, angle adjustment) is automatically called.

[0116] It should also be noted that the control instructions generated by the instruction generation unit are dependent on the results of sensor recognition and rule judgment, which belong to the decision-making stage of the planning stage. However, after actually entering the processing stage, the material processing module faces the "current real material state" and the "current available state of the device", and needs to make a new decision.

[0117] For example, the control module recognizes that the front is "high-hardness ore", and then the instruction generation unit issues a "crushing treatment" instruction, specifying the particle size as 2-5 cm. But the instruction execution unit finds that: the current crushing accessory is "light hammer type", which cannot meet the strength of crushing the ore; the outer layer of the ore is mixed with wet mud, which is easy to adhere and affect the processing efficiency; the current processing equipment is vibrating too much (with overheating warning); (or the current accessory is being used); at this time, the instruction execution unit must "rejudge" and "reselect" according to the real-time state and the preset operation rule library: start other crushers in the accessory library; add a dust spray device to avoid dust; and dynamically adjust the crushing frequency to reduce the load of the equipment.

[0118] It can be understood that, although the instruction generation unit has generated the preliminary operation instruction, the instruction execution unit still needs to dynamically select the most suitable processing mode according to the diversity of the material type, characteristics and processing target in the current working environment, in order to realize more efficient and accurate working effect. This is because the control instruction usually focuses on the basic action parameters (such as force, angle, path) of the actuator, and the specific material processing mode (such as whether to crush, whether to classify and recycle, whether to need to replace the tool, etc.) needs higher level decision judgment. Through comprehensive analysis of the hardness, density, recyclability and other characteristics of the material, the instruction execution unit can automatically call the corresponding processing strategy and tool resources, such as automatically switching to the crushing device, the grabbing device or the ordinary shovel, etc., and dynamically optimize the parameters according to the real-time monitoring results (such as crushing granularity, conveying state), so as to ensure the high efficiency, high precision and intelligent cooperation of the processing process. This way effectively realizes the decoupling and cooperation of the "decision layer-control layer-execution layer", so that the system has the self-adaptive ability to cope with complex and changeable construction environment, and improves the intelligent and automatic level of the whole operation.

[0119] It should be noted that the devices in the embodiments provided in the present application are common market devices, which can be selected according to needs during specific use, and the circuit connection relationship of each device is a simple series and parallel connection circuit, which can be easily realized by those skilled in the art, and belongs to the prior art, which will not be described in detail.

[0120] Each embodiment or implementation in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same and similar parts between each embodiment can be referred to.

[0121] It should be noted that the "one embodiment", "embodiment", "exemplary embodiment", "some embodiments" and the like mentioned in the specification mean that the described embodiments can include a specific feature, structure or characteristic, but not necessarily every embodiment includes the specific feature, structure or characteristic. In addition, such phrases do not necessarily refer to the same embodiment. In addition, when a specific feature, structure or characteristic is described in combination with an embodiment, it is within the knowledge of those skilled in the art to realize such feature, structure or characteristic in combination with other explicitly or implicitly described embodiments.

[0122] It should be noted that, in the present document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0123] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be capable of developing other similar embodiments or variations without departing from the scope of the application. The scope of the application is defined by the appended claims.

[0124] Finally, it should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that those skilled in the art will be capable of developing other similar embodiments or variations without departing from the scope of the application. The scope of the application is defined by the appended claims.

Claims

1. An excavator material handling system based on adaptive control, characterized in that: include: Material identification module, control module and adaptive material handling module; The material identification module is configured to collect three-dimensional point cloud data, image data, and temperature data of a target material; and obtain a fused feature vector of the target material based on the three-dimensional point cloud data, the image data, and the temperature data. The control module is used to determine the physical properties of the target material based on the fused feature vector and a preset material feature library; The adaptive material processing module is used to generate control instructions based on the physical characteristics and a preset operation rule library, and process the target material according to the control instructions; The control module is specifically used for: Using a pre-trained material judgment model, determine the material characteristics corresponding to the fused feature vector; The material characteristics are compared with the material characteristic rules in the preset material characteristic library. When the comparison result shows that there is no rule conflict, the physical characteristics corresponding to the fused feature vector are determined; wherein the preset material characteristic library includes physical attribute constraint rules, type feature mapping rules and outlier filtering rules.

2. The system according to claim 1, wherein: The material identification module includes a data acquisition unit and a data fusion unit; The data acquisition unit is used to collect the three-dimensional point cloud data, image data and temperature data of the target material; the data fusion unit is used to preprocess the three-dimensional point cloud data, the image data and the temperature data respectively, and perform feature extraction and feature weighted fusion processing on the preprocessed three-dimensional point cloud data, the preprocessed image data and the preprocessed temperature data to obtain a fused feature vector of the target material.

3. The system according to claim 2, characterized in that The data acquisition unit includes: a lidar sensor, a camera and an infrared sensor; The laser radar sensor is arranged at one end of the excavator's boom connected to the stick, and is used to obtain the three-dimensional point cloud data of the target material; There are at least three cameras, which are respectively arranged at different positions of the excavator and used to obtain the image data of the target material from different angles; The infrared sensor is arranged at the bucket connecting rod bracket position of the excavator and is used to collect temperature data of the surface of the target material.

4. The system according to claim 2, wherein: The data fusion unit is specifically used for: Performing noise reduction and outlier removal processing on the three-dimensional point cloud data to obtain the pre-processed three-dimensional point cloud data; performing image enhancement processing and distortion correction processing on the image data to obtain the pre-processed image data; performing normalization processing on the temperature data to obtain the pre-processed temperature data; Through the feature fusion model, the geometric features corresponding to the preprocessed three-dimensional point cloud data, the visual features corresponding to the preprocessed image data, and the temperature features corresponding to the preprocessed temperature data are extracted, the weights corresponding to the geometric features, the visual features, and the temperature features are adaptively adjusted, and feature fusion processing is performed on the geometric features, the visual features, and the temperature features based on the weights to obtain the fused feature vector of the target material.

5. The system according to claim 1, wherein: The material judgment model is obtained by using a random sampling strategy to collect multimodal feature data, standard sample data, construction site measured data and physical simulation data to generate multiple batches of training samples, and then training the initial model based on the training samples of each batch.

6. The system according to claim 1, wherein: The adaptive material processing module includes an instruction generation unit and an instruction execution unit; The instruction generation unit is used to match the physical characteristics with the preset operation rule library and generate a control instruction according to the matching result; the preset operation rule library includes the correspondence between the operation rules and the physical characteristics; The instruction execution unit is used to determine the target attachment, digging force, digging speed, bucket opening and closing angle, grab claw action and material handling method according to the control instruction; and use the target attachment to process the target material with the digging force, the digging speed, the bucket opening and closing angle, the grab claw action and the material handling method.

7. The system according to claim 6, characterized in that When the instruction generation unit generates a control instruction according to the matching result, it is specifically used to: When the matching result indicates that one of the operation rules in the preset operation rule library successfully matches the physical characteristic, generating the control instruction according to the operation rule; When the matching result indicates that multiple operation rules in the preset operation rule library successfully match the physical characteristics, the multiple operation rules are sorted according to the preset weights corresponding to each of the multiple operation rules, and the corresponding control instructions are generated in sequence according to the order of the multiple operation rules.

8. The system according to claim 6, wherein: The command execution unit includes an attachment library and an electronically controlled hydraulic wrist arranged at the end of the excavator boom; The electronically controlled hydraulic wrist is used to select the target attachment from the attachment library according to the control instruction and replace the attachment of the excavator with the target attachment; after the replacement of the target attachment is completed, the target material is processed with the digging force, the digging speed, the bucket opening and closing angle, the grab claw action and the material processing method.

9. An excavator, characterized in that: The excavator is provided with an excavator material handling system based on adaptive control according to any one of claims 1 to 8.

Images