Control system, control method, device, storage medium and program product

Abstract

Provided in the present disclosure are a control system, a control method, a device, a storage medium and a program product. The control system comprises: an environment sensing module, which is configured to output sensing result information on the basis of environment information and / or control result information, and adjust the state thereof on the basis of a system adjustment instruction; a decision-making and planning module, which is configured to generate a motion trajectory on the basis of an operation task and the sensing result information, output the motion trajectory, perform predetermination on the motion trajectory to obtain predetermination result information, receive feedback result information corresponding to the motion trajectory, and output the system adjustment instruction to the environment sensing module on the basis of the predetermination result information and the feedback result information; and a control execution module, which is configured to execute the operation task on the basis of the motion trajectory, obtain the feedback result information and control result information during the execution of the operation task, output the feedback result information to the decision-making and planning module, and output the control result information to the environment sensing module.

Description

Control system, control method, device, storage medium and program product

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the application with CN application number 202410415004.X and application date April 7, 2024, and claims its priority. The disclosed content of the CN application is hereby introduced as a whole into this application. Technical Field

[0003] The present disclosure relates to the technical field of intelligent engineering machinery, and in particular to a control system, a control method, a device, a storage medium, and a program product. Background Art

[0004] Currently, the intelligentization of construction machinery is gaining increasing attention and has made considerable progress. Currently, intelligent construction machinery generally uses a sequential architecture of perception-planning-control.

[0005] Summary of the Invention

[0006] According to one aspect of the present disclosure, a control system for engineering machinery equipment is provided, including: an environmental perception module, which is used to receive environmental information and / or control result information, and output perception result information based on the environmental information and / or the control result information, and receive system adjustment instructions, and adjust its own state based on the system adjustment instructions; a decision planning module, which is used to receive an operation task and the perception result information, generate a motion trajectory based on the operation task and the perception result information, output the motion trajectory, pre-judge the motion trajectory to obtain pre-judgement result information, receive feedback result information corresponding to the motion trajectory, and output the system adjustment instruction to the environmental perception module based on the pre-judgement result information and the feedback result information; and a control execution module, which is used to receive the motion trajectory, execute the operation task based on the motion trajectory, obtain the feedback result information and the control result information in the process of executing the operation task, output the feedback result information to the decision planning module, and output the control result information to the environmental perception module.

[0007] In some embodiments, the environmental perception module is further configured to output the perception result information directly to the control execution module; the control execution module is further configured to receive the perception result information and perform corresponding operations based on the perception result information.

[0008] In some embodiments, the decision planning module is used to calculate the adjustment parameters based on the prediction result information and the feedback result information, and output the system adjustment instruction based on the adjustment parameters, wherein the system adjustment instruction includes the adjustment parameters, and the adjustment parameters are used to adjust the state of the environmental perception module; the environmental perception module is used to extract the adjustment parameters from the system adjustment instruction, and adjust its own state based on the adjustment parameters.

[0009] In some embodiments, the feedback result information includes a reaction result of the working object of the engineering machinery equipment.

[0010] In some embodiments, the perception result information includes: the environmental information; or the perception result information includes: a logical judgment result generated based on the environmental information and / or the control result information.

[0011] In some embodiments, the environmental perception module includes: an environmental perception element and a multi-degree-of-freedom platform, wherein the environmental perception element is installed on the multi-degree-of-freedom platform; the adjustment parameters include: the intrinsic parameter value of the environmental perception element and / or the parameters of the multi-degree-of-freedom platform.

[0012] In some embodiments, the environmental perception module is also used to transmit the environmental information to the decision-making and planning module, wherein the environmental information is first data information with the element coordinate system of the environmental perception module itself as the coordinate system; the decision-making and planning module is also used to convert the first data information into second data information with the reference coordinate system as the coordinate system based on the transformation matrix between the element coordinate system and the reference coordinate system of the engineering machinery equipment, and generate a motion trajectory based on the work task and the second data information.

[0013] In some embodiments, the decision-making planning module is also used to adjust the motion trajectory based on the perception result information of the environmental perception module, and output the adjusted motion trajectory to the control execution module; the control execution module is also used to control the movement of the engineering machinery equipment based on the adjusted motion trajectory.

[0014] In some embodiments, the decision-making planning module is also used to adjust the motion trajectory based on the feedback result information of the control execution module, and output the adjusted motion trajectory to the control execution module; the control execution module is also used to control the movement of the engineering machinery equipment based on the adjusted motion trajectory.

[0015] In some embodiments, the decision planning module is also used to send control instructions to the control execution module to control the operating state or operating speed of the control execution module, so that the operating state or operating speed of the control execution module matches the state adjusted by the environmental perception module.

[0016] According to another aspect of the present disclosure, a control method for engineering machinery equipment is provided, including: using an environmental perception module to receive environmental information and / or control result information, and outputting perception result information based on the environmental information and / or the control result information; using a decision-making and planning module to receive an operation task and the perception result information, generating a motion trajectory based on the operation task and the perception result information, outputting the motion trajectory, and pre-judging the motion trajectory to obtain pre-judgment result information; using a control execution module to receive the motion trajectory, executing the operation task based on the motion trajectory, and obtaining the feedback result information and the control result information in the process of executing the operation task, outputting the feedback result information to the decision-making and planning module, and outputting the control result information to the environmental perception module; using the decision-making and planning module to receive feedback result information corresponding to the motion trajectory, and outputting a system adjustment instruction to the environmental perception module based on the pre-judgment result information and the feedback result information; and using the environmental perception module to receive the system adjustment instruction, and adjusting the state of the environmental perception module itself based on the system adjustment instruction.

[0017] In some embodiments, the control method further includes: using the environmental perception module to directly output the perception result information to the control execution module; and using the control execution module to receive the perception result information and perform corresponding operations based on the perception result information.

[0018] In some embodiments, the decision-making planning module is used to output a system adjustment instruction to the environmental perception module based on the prediction result information and the feedback result information, including: using the decision-making planning module to calculate an adjustment parameter based on the prediction result information and the feedback result information, wherein the adjustment parameter is used to adjust the state of the environmental perception module; and using the decision-making planning module to output the system adjustment instruction based on the adjustment parameter, wherein the system adjustment instruction includes the adjustment parameter; using the environmental perception module to adjust its own state based on the system adjustment instruction, including: using the environmental perception module to extract the adjustment parameter from the system adjustment instruction; and using the environmental perception module to adjust its own state based on the adjustment parameter.

[0019] In some embodiments, the feedback result information includes a reaction result of the working object of the engineering machinery equipment.

[0020] In some embodiments, the perception result information includes: the environmental information; or the perception result information includes: a logical judgment result generated based on the environmental information and / or the control result information.

[0021] In some embodiments, the control method also includes: using the environmental perception module to transmit the environmental information to the decision-making and planning module, wherein the environmental information is first data information with the element coordinate system of the environmental perception module itself as the coordinate system; and using the decision-making and planning module based on the transformation matrix between the element coordinate system and the reference coordinate system of the engineering machinery equipment, converting the first data information into second data information with the reference coordinate system as the coordinate system, and generating a motion trajectory based on the work task and the second data information.

[0022] In some embodiments, the control method also includes: using the decision-making planning module to adjust the motion trajectory based on the perception result information of the environmental perception module, and outputting the adjusted motion trajectory to the control execution module; and using the control execution module to control the movement of the engineering machinery equipment based on the adjusted motion trajectory.

[0023] In some embodiments, the control method also includes: using the decision planning module to adjust the motion trajectory based on the feedback result information of the control execution module, and outputting the adjusted motion trajectory to the control execution module; and using the control execution module to control the movement of the engineering machinery equipment based on the adjusted motion trajectory.

[0024] In some embodiments, the control method also includes: using the decision-making planning module to send control instructions to the control execution module to control the operating state or operating speed of the control execution module, so that the operating state or operating speed of the control execution module matches the adjusted state of the environmental perception module.

[0025] According to another aspect of the present disclosure, a control system for engineering machinery equipment is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the aforementioned control method based on instructions stored in the memory.

[0026] According to another aspect of the present disclosure, there is provided an engineering machinery device, comprising: the control system as described above.

[0027] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the control method as described above is implemented.

[0028] According to another aspect of the present disclosure, a computer program product is provided, comprising instructions, which, when executed by a processor, enable the processor to perform the control method as described above.

[0029] According to another aspect of the present disclosure, a computer program is provided, comprising instructions, which, when executed by a processor, cause the processor to perform the control method as described above.

[0030] Other features and advantages of the present disclosure will become apparent from the following detailed description of exemplary embodiments of the present disclosure with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0032] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0033] FIG1 is a block diagram schematically illustrating a control system for engineering machinery equipment according to some embodiments of the present disclosure;

[0034] FIG2 is a flow chart illustrating a control method for engineering machinery equipment according to some embodiments of the present disclosure;

[0035] FIG3 is a schematic diagram illustrating an architecture system of an intelligent unmanned excavator according to some embodiments of the present disclosure;

[0036] FIG4 is a schematic diagram illustrating a joint coordinate model of a working device of an intelligent unmanned excavator according to some embodiments of the present disclosure;

[0037] FIG5 is a flow chart illustrating a method for controlling an intelligent unmanned excavator according to some embodiments of the present disclosure;

[0038] FIG6 is a block diagram schematically illustrating a control system for engineering machinery equipment according to other embodiments of the present disclosure;

[0039] FIG7 is a block diagram schematically illustrating a structure of a control system for engineering machinery equipment according to other embodiments of the present disclosure. DETAILED DESCRIPTION

[0040] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0041] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0042] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0043] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0044] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0045] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0046] In related technologies, the focus of intelligent construction machinery is intelligent operation. However, the tasks, objects, and environments of construction machinery operations are more complex and variable than the driving tasks and road environments of autonomous vehicles. In some cases, the sequential perception-planning-control architecture may no longer meet the operational requirements. For example, when a loader is operating, the traveling mechanism and bucket move simultaneously to complete the combined action. If the planning is not thorough, the bucket will block the view of the sensor pointing in the direction of travel. When an excavator is operating, the slewing mechanism, boom, dipper arm, and bucket move simultaneously to complete the complex operation. Regardless of where the onboard sensors are installed, they will move with the machine. In some cases, the sensors will no longer point in the direction of travel or the direction of operation, and so on. Therefore, the sequential perception-planning-control architecture of construction machinery in related technologies may result in poor coordination during the execution of the construction machinery.

[0047] In view of this, embodiments of the present disclosure provide a control system or a control method for engineering machinery equipment to improve the coordination of the engineering machinery during operation.

[0048] Figure 1 is a block diagram schematically illustrating a control system for construction machinery according to some embodiments of the present disclosure. As shown in Figure 1 , the control system includes an environmental perception module 110, a decision-making and planning module 120, and a control execution module 130. For example, the control system can be used in intelligent and unmanned systems for construction machinery such as excavators and loaders.

[0049] The environment perception module 110 is used to receive environment information and / or control result information, and output perception result information based on the environment information and / or control result information, as well as receive system adjustment instructions and adjust its own state based on the system adjustment instructions.

[0050] That is, the environmental perception module 110 can receive environmental information and output perception result information based on the environmental information; or the environmental perception module 110 can receive control result information and output perception result information based on the control result information; or the environmental perception module 110 can receive environmental information and control result information and output perception result information based on the environmental information and control result information.

[0051] For example, when the engineering machinery equipment just starts working, the environmental perception module can receive environmental information and output perception result information based on the environmental information. The decision-making planning module generates and outputs the motion trajectory based on the work task and the perception result information. The control execution module executes the work task based on the motion trajectory and obtains the control result information, and outputs the control result information to the environmental perception module, so that the environmental perception module can receive the control result information. Since the environmental perception module is still receiving environmental information at this time, the environmental perception module receives environmental information and / or control result information.

[0052] In some embodiments, the environmental perception module includes: an environmental perception element and a multi-degree-of-freedom platform. The environmental perception element is mounted on the multi-degree-of-freedom platform. For example, the environmental perception element includes, but is not limited to, a camera, a lidar, or a millimeter-wave radar, etc. (also referred to as a sensor). The multi-degree-of-freedom platform can be a controllable platform or a multi-degree-of-freedom structure. For example, the multi-degree-of-freedom platform includes, but is not limited to, a partial structure on engineering machinery or a separately installed device.

[0053] For example, the installation location and installation method of environmental sensing components can be determined according to the work tasks.

[0054] If the internal parameters of the environment sensing element can be changed (for example, the focal length or frequency can be changed), a program interface for controlling the change of its internal parameters can be provided to the decision-making planning module.

[0055] For example, the environmental perception element is installed on a multi-degree-of-freedom platform, so that the decision-making planning module can change the base point position and viewing angle of the environmental perception element within a certain range according to the task command execution mechanism.

[0056] In some embodiments, environmental information includes: environmental image information captured by a camera and / or three-dimensional information detected by a radar (e.g., three-dimensional information including distance information, direction information, etc.). Control result information is the changes in the posture and state of the engineering machinery during operation, output by the control execution module, and includes, for example, boom posture information (e.g., boom angle).

[0057] In an embodiment of the present disclosure, the environment perception module 110 is configured to receive at least one of the above-mentioned environment information and the above-mentioned control result information, and output perception result information to the decision planning module 120 based on at least one of the environment information and the control result information.

[0058] For example, the perception result information includes: environmental information. That is, the environmental perception module can directly output the environmental information as the perception result information to the decision-making planning module.

[0059] For another example, the perception result information includes: a logical judgment result generated based on the environmental information and / or the control result information. In other words, the environmental perception module can generate a logical judgment result based on the environmental information and / or the control result information. For example, if the environmental perception module determines that there is an obstacle in front of the construction machinery (e.g., an excavator) based on the environmental information and / or the control result information, it will generate a logical judgment result regarding the presence of the obstacle in front. The environmental perception module then outputs this logical judgment result as the perception result information to the decision-making and planning module.

[0060] For another example, the environmental perception module may also output the environmental information and the above-mentioned logical judgment results as perception result information to the decision-making planning module.

[0061] In an embodiment of the present disclosure, the environmental perception module 110 is also used to receive system adjustment instructions from the decision-making and planning module 120, and adjust its own state based on the system adjustment instructions. This makes it easier for the environmental perception module to achieve a better perception function. For example, the state of the environmental perception module includes: the intrinsic parameter values ​​of the environmental perception element and / or the parameters of the multi-degree-of-freedom platform. That is, the environmental perception module 110 can adjust the intrinsic parameter values ​​of the environmental perception element (for example, focal length, etc.) and / or the parameters of the multi-degree-of-freedom platform (for example, the azimuth angle of the multi-degree-of-freedom platform, for example, the azimuth angle includes the horizontal angle and / or the pitch angle, etc.) based on the system adjustment instructions. For another example, the state of the environmental perception module may also include: the position or posture of the environmental perception element, etc.

[0062] In some embodiments, the environment perception module may be an environment perception circuit. That is, the environment perception module may be implemented using circuits. In other embodiments, the environment perception module may include software components or be implemented using a combination of software and hardware.

[0063] The decision-making planning module 120 is used to receive work tasks and perception result information, generate a motion trajectory based on the work tasks and perception result information, output the motion trajectory, predict the motion trajectory to obtain prediction result information, receive feedback result information corresponding to the motion trajectory, and output system adjustment instructions to the environmental perception module based on the prediction result information and the feedback result information.

[0064] For example, an operation task (for example, excavation, loading, etc.) can be input into the decision-making and planning module, so that the decision-making and planning module receives the operation task. The decision-making and planning module also receives the perception result information output by the environmental perception module, generates a motion trajectory based on the operation task and the perception result information, and outputs the motion trajectory to the control execution module, so that the control execution module executes the operation task based on the motion trajectory.

[0065] The decision-making and planning module can also predict the motion trajectory to obtain prediction results. For example, if the task is to raise the boom, the decision-making and planning module can predict the motion trajectory of the boom and predict whether the sensing range (for example, the camera range) of the environmental sensing element installed on the boom will be reduced. The decision-making and planning module can also receive feedback results corresponding to the motion trajectory. This feedback result information is provided by the control execution module.

[0066] In some embodiments, the feedback information includes the reaction results of the working object of the engineering machinery. For example, if the working task is excavating and loading materials, and the working object is the material, the reaction result of the working object on the bucket is manifested as a pressure change in the boom cylinder. The pressure change of the cylinder is fed back to the decision-making and planning module, and the decision-making and planning module can determine whether the current boom has encountered foreign objects during the excavation process based on the pressure change of the cylinder.

[0067] The decision-making and planning module can output system adjustment instructions to the environmental perception module based on the prediction result information and the feedback result information. For example, if the decision-making and planning module predicts that the perception range of the environmental perception element installed on the boom (such as the excavator boom) will decrease, and determines the current state of the boom (such as the height of the boom, or whether it has encountered an obstacle, etc.) based on the reaction results of the working object of the construction machinery equipment, the system adjustment instruction can be output to the environmental perception module, causing the environmental perception module to adjust its own state, such as changing the focal length or angle, to increase the perception range.

[0068] In some embodiments, the decision-making planning module may be a decision-making planning circuit. That is, the decision-making planning module may be implemented in a circuit manner. In other embodiments, the decision-making planning module may include a software component, or be implemented in a combination of software and hardware.

[0069] The control execution module 130 is used to receive a motion trajectory (as mentioned above, the motion trajectory is a trajectory generated after comprehensively considering the work task and perception results), execute the work task based on the motion trajectory, and obtain feedback result information and control result information during the execution of the work task, output the feedback result information to the decision planning module, and output the control result information to the environmental perception module.

[0070] In some embodiments, the control result information is part of the feedback result information. For example, taking an excavator as an example, the control result information includes: working device posture information (eg, boom angle, etc.).

[0071] In some embodiments, the feedback result information includes the reaction results of the working object of the engineering machinery. For example, in addition to the aforementioned control result information (e.g., boom posture information), the feedback result information may also include the reaction results of the working object (e.g., the pressure of the working device's cylinder detected by a pressure sensor). The control execution module outputs this feedback result information to the decision planning module.

[0072] In other embodiments, the control execution module can control the machine to complete the work task based on the motion trajectory and the reaction of the work object.

[0073] In some embodiments, the control execution module may be a control execution circuit. That is, the control execution module may be implemented using circuitry. For example, the control execution module may include a control unit and an actuator. In other embodiments, the control execution module may include software components or be implemented using a combination of software and hardware.

[0074] Thus, a control system for engineering machinery equipment according to some embodiments of the present disclosure is provided. The control system includes: an environmental perception module for receiving environmental information and / or control result information, and outputting perception result information based on the environmental information and / or control result information, as well as receiving system adjustment instructions and adjusting its own state based on the system adjustment instructions; a decision-making and planning module for receiving work tasks and perception result information, generating a motion trajectory based on the work tasks and perception result information, outputting the motion trajectory, pre-judging the motion trajectory to obtain pre-judgment result information, receiving feedback result information corresponding to the motion trajectory, and outputting system adjustment instructions to the environmental perception module based on the pre-judgment result information and the feedback result information; and a control execution module for receiving the motion trajectory, executing the work tasks based on the motion trajectory, and obtaining feedback result information and control result information during the execution of the work tasks, outputting the feedback result information to the decision-making and planning module, and outputting the control result information to the environmental perception module. The above-mentioned control system can improve the coordination of engineering machinery equipment during the execution of work tasks.

[0075] In addition, the above-mentioned control system can enable engineering machinery equipment to consider the corresponding movement of the environmental perception module and its potential impact on the perception function when planning and executing machine operation behaviors during intelligent operation, and take appropriate compensation measures to improve the coordination of the system.

[0076] In some embodiments, the decision-making and planning module can also be used to adjust the motion trajectory based on the prediction and feedback information to facilitate environmental perception. For example, if the decision-making and planning module determines, based on the prediction and feedback information, that an obstacle has been encountered and that the obstacle is blocking the environmental perception module's perception of the surroundings, the movement trajectory of the construction machinery may be adjusted to facilitate the environmental perception module's better perception of the surroundings.

[0077] In some embodiments, the environmental perception module 110 is further configured to output perception result information directly to the control execution module. The control execution module 130 is further configured to receive the perception result information and perform corresponding operations based on the perception result information. In other words, some perception results can be directly used by the control execution module without passing through the decision-making and planning module. This enables timely response to the environment and improves the responsiveness of the control system.

[0078] For example, during an emergency avoidance operation, if the environmental perception module determines that an obstacle is ahead of the construction machinery based on environmental information, it will directly output this obstacle perception information to the control execution module, which then executes an avoidance action based on this information, such as stopping the vehicle. This improves the control system's response speed and reduces the likelihood of collision with an obstacle.

[0079] In some embodiments, the decision-making planning module 120 is configured to calculate adjustment parameters based on the pre-judgment result information and the feedback result information, and output system adjustment instructions based on the adjustment parameters, wherein the system adjustment instructions include the adjustment parameters, and the adjustment parameters are used to adjust the state of the environmental perception module. The environmental perception module 110 is configured to extract the adjustment parameters from the system adjustment instructions and adjust its own state based on the adjustment parameters. This enables the decision-making planning module to adjust the state of the environmental perception module, thereby enabling appropriate compensation measures to be taken for the corresponding movement of the environmental perception module and its potential impact on the perception function, thereby improving the coordination of the system.

[0080] For example, the adjustment parameters include: intrinsic parameters of the environment sensing element (e.g., focal length) and / or parameters of the multi-degree-of-freedom platform. For example, the intrinsic parameters of the environment sensing element include focal length. For another example, the parameters of the multi-degree-of-freedom platform include the pan angle and / or pitch angle of the multi-degree-of-freedom platform.

[0081] In some embodiments, the environmental perception module 110 is further configured to transmit environmental information to the decision-making and planning module, where the environmental information is first data information with the component coordinate system of the environmental perception module itself as the coordinate system. The decision-making and planning module 120 is further configured to convert the first data information into second data information with the reference coordinate system as the coordinate system based on a transformation matrix between the component coordinate system and the reference coordinate system of the construction machinery, and to generate a motion trajectory based on the work task and the second data information.

[0082] That is to say, the environmental information detected by the environmental perception module is data with its own component coordinate system as the coordinate system, while the data processed by the decision-making and planning module is data with the reference coordinate system as the coordinate system. Therefore, after receiving the environmental information from the environmental perception module, the decision-making and planning module needs to convert the first data information with the component coordinate system as the coordinate system into the second data information with the reference coordinate system as the coordinate system based on the above-mentioned transformation matrix of the coordinate system, and then generate a motion trajectory based on the work task and the second data information.

[0083] In some embodiments, by measuring and calculating the transformation matrix between the coordinate system of each environmental sensing element and the reference coordinate system of the machine in real time, the detection results of each environmental sensing element can be expressed to the machine reference coordinate system in a timely manner through appropriate spatial transformation and time alignment.

[0084] In some embodiments, the coordinate parameters (for example, offset and angle) between the element coordinate system and the reference coordinate system of the environmental perception module can be obtained through physical measurement, multi-sensor detection fusion, etc., so as to obtain the transformation matrix and calibrate the environmental perception module.

[0085] In some embodiments, the decision-making and planning module 120 is further configured to adjust the motion trajectory based on the perception results of the environmental perception module and output the adjusted motion trajectory to the control execution module 130. The control execution module 130 is further configured to control the movement of the construction machinery based on the adjusted motion trajectory. This can improve the adaptability and safety of the construction machinery.

[0086] For example, if the perception result information includes information about pedestrians or obstacles appearing in front of the driving direction of the construction machinery equipment, the decision-making planning module adjusts the motion trajectory (for example, the driving trajectory of the construction machinery equipment) based on the perception result information, and outputs the adjusted motion trajectory to the control execution module. The control execution module then adjusts the motion trajectory of the construction machinery equipment based on the adjusted motion trajectory, so as to avoid colliding with pedestrians or obstacles as much as possible.

[0087] For another example, if, while the construction machinery is executing along its trajectory, the environmental perception module senses the presence of pedestrians or obstacles ahead of the construction machinery (such a situation may affect the environmental perception module's perception range), the module sends this perception result information to the decision-making and planning module, which adjusts the motion trajectory based on the perception result information and outputs the adjusted motion trajectory to the control execution module. The control execution module controls the movement of the construction machinery based on the adjusted motion trajectory, thereby changing the motion trajectory. By changing the motion trajectory, the environmental perception of the environmental perception module can be improved.

[0088] In some embodiments, the decision-making and planning module 120 is further configured to adjust the motion trajectory based on the feedback from the control execution module and output the adjusted motion trajectory to the control execution module. The control execution module is further configured to control the movement of the construction machinery based on the adjusted motion trajectory. This can improve the adaptability and safety of the construction machinery.

[0089] For example, the feedback result information includes information that the pressure of the bucket cylinder of the excavator exceeds a threshold value. The decision-making planning module determines that the bucket has hit a hard object based on the feedback result information, and thus adjusts the motion trajectory and outputs the adjusted motion trajectory to the control execution module. The control execution module adjusts the motion trajectory of the engineering machinery equipment based on the adjusted motion trajectory, for example, adjusts the excavation trajectory of the bucket to bypass the hard object.

[0090] In some embodiments, the decision-making and planning module 120 is further configured to send control instructions to the control execution module 130 to control the operating state or operating speed of the control execution module 130, so that the operating state or operating speed of the control execution module 130 matches the state adjusted by the environment perception module 110. This can improve the coordination of the engineering machinery equipment.

[0091] For example, the decision-making planning unit can sacrifice some operational efficiency or other optimization metrics to ensure that the planned motion trajectory can be coordinated with the normal operation of the environmental perception module, such as planning the construction machinery to maintain a specific posture or stop during operation. For example, if the environmental perception module's pitch angle changes slowly, the decision-making planning module can reduce the operating speed of the control execution module's actuator, thereby ensuring that the control execution module matches the adjusted state of the environmental perception module.

[0092] In some embodiments, the environmental perception module 110, the decision-making and planning module 120, and the control execution module 130 respectively include software and / or hardware. The specific hardware and software implementations can be different. For example, the perception algorithm, decision-making and planning algorithm, and advanced control algorithm can be deployed on a vehicle-mounted computing unit or distributed across multiple vehicle-mounted computing units. Under the conditions of meeting bandwidth, latency, reliability, and security, they can also be partially deployed in computing units outside the vehicle, such as control rooms in mines, construction sites, and workshops (i.e., edge-end deployment), or in dedicated computing centers (i.e., cloud-edge-end deployment).

[0093] FIG2 is a flow chart illustrating a control method for engineering machinery according to some embodiments of the present disclosure. As shown in FIG2 , the control method includes steps S202 to S210 .

[0094] In step S202, the environment perception module is used to receive environment information and / or control result information, and output perception result information based on the environment information and / or control result information.

[0095] In other embodiments, the environment perception module may transmit its working status in addition to the perception results to the decision-making and planning module during its operation.

[0096] In step S204, the decision-making planning module is used to receive the work task and the perception result information, generate a motion trajectory based on the work task and the perception result information, output the motion trajectory, and predict the motion trajectory to obtain prediction result information.

[0097] In step S206, the control execution module receives the motion trajectory, executes the work task based on the motion trajectory, and obtains feedback result information and control result information during the execution of the work task, outputs the feedback result information to the decision planning module, and outputs the control result information to the environmental perception module.

[0098] In step S208, the decision-making planning module receives feedback result information corresponding to the motion trajectory, and outputs a system adjustment instruction to the environment perception module based on the prediction result information and the feedback result information.

[0099] In step S210, the environment perception module is used to receive a system adjustment instruction, and the state of the environment perception module itself is adjusted based on the system adjustment instruction.

[0100] Thus, a control method for engineering machinery equipment according to some embodiments of the present disclosure is provided. In this control method, an environmental perception module is used to output perception result information based on environmental information and / or control result information; a decision-making and planning module is used to generate and output a motion trajectory based on the work task and the perception result information, and the motion trajectory is prejudged to obtain prejudgment result information; a control execution module is used to execute the work task based on the motion trajectory, and obtain feedback result information and control result information, which are then output to the decision-making and planning module, and the control result information is output to the environmental perception module; the decision-making and planning module is used to output system adjustment instructions to the environmental perception module based on the prejudgment result information and the feedback result information; and the environmental perception module is used to adjust the state of the environmental perception module based on the system adjustment instructions. This control method can improve the coordination of engineering machinery equipment during work execution.

[0101] In some embodiments, the control method further includes: a decision-making planning module adjusting the motion trajectory based on the prediction result information and the feedback result information to facilitate environmental perception.

[0102] In some embodiments, the control method further includes: utilizing the environment perception module to directly output the perception result information to the control execution module; and utilizing the control execution module to receive the perception result information and perform corresponding operations based on the perception result information. This can improve the response speed of the control system.

[0103] In some embodiments, a decision-making planning module is used to output a system adjustment instruction to the environmental perception module based on the prediction result information and the feedback result information, including: using the decision-making planning module to calculate the adjustment parameters based on the prediction result information and the feedback result information, wherein the adjustment parameters are used to adjust the state of the environmental perception module; and using the decision-making planning module to output the system adjustment instruction based on the adjustment parameters, wherein the system adjustment instruction includes the adjustment parameters.

[0104] In some embodiments, using the environment perception module to adjust its own state based on the system adjustment instruction includes: using the environment perception module to extract adjustment parameters from the system adjustment instruction; and using the environment perception module to adjust its own state based on the adjustment parameters.

[0105] In this way, the decision-making planning module can adjust the state of the environmental perception module, so that appropriate compensation measures can be taken for the corresponding movement of the environmental perception module and its potential impact on the perception function, thereby improving the coordination of the system.

[0106] In some embodiments, the feedback result information includes a reaction result of an operation object of the engineering machinery equipment.

[0107] In some embodiments, the perception result information includes: environmental information.

[0108] In other embodiments, the perception result information includes: a logical judgment result generated based on the environmental information and / or the control result information.

[0109] In some embodiments, the control method further includes: utilizing the environmental perception module to transmit environmental information to the decision-making and planning module, wherein the environmental information is first data information having a coordinate system defined by the element coordinate system of the environmental perception module itself; and utilizing the decision-making and planning module to convert the first data information into second data information having a coordinate system defined by the reference coordinate system based on a transformation matrix between the element coordinate system and a reference coordinate system of the engineering machinery, and generating a motion trajectory based on the work task and the second data information. In some embodiments, by measuring and calculating the transformation matrix between the coordinate system of each environmental perception element and the reference coordinate system of the machine in real time, the detection results of each environmental perception element can be promptly expressed in the machine reference coordinate system through appropriate spatial transformation and temporal alignment.

[0110] In some embodiments, the control method further includes: utilizing the decision-making and planning module to adjust the motion trajectory based on the perception results of the environmental perception module, and outputting the adjusted motion trajectory to the control execution module; and utilizing the control execution module to control the movement of the construction machinery based on the adjusted motion trajectory. Specifically, during the execution of the motion trajectory, the decision-making and planning module promptly adjusts the machine's motion trajectory based on the operating status of the environmental perception module. This can improve the resilience and safety of the construction machinery.

[0111] In some embodiments, the control method further includes: utilizing the decision-making and planning module to adjust the motion trajectory based on feedback from the control execution module, and outputting the adjusted motion trajectory to the control execution module; and utilizing the control execution module to control the movement of the construction machinery based on the adjusted motion trajectory. Specifically, during the execution of the operation trajectory, the decision-making and planning module promptly adjusts the operation plan based on the operating status of the control execution module. This can improve the adaptability and safety of the construction machinery.

[0112] In some embodiments, the control method further includes: using the decision-making planning module to send a control instruction to the control execution module to control the operating state or operating speed of the control execution module so that the operating state or operating speed of the control execution module matches the state adjusted by the environment perception module. This can improve the coordination of the engineering machinery equipment.

[0113] In some embodiments, when planning the operation behavior of engineering machinery equipment, the decision-making planning module calculates the motion trajectory of each environmental perception element corresponding to the operation behavior and its impact on the perception function, so as to ensure as much as possible that the environmental perception element can provide sufficient quantity and quality of perception information in the planned movement.

[0114] In other embodiments, during the execution of the operation trajectory, the decision-making planning module can dynamically adjust the intrinsic parameter value, position or posture of the environmental perception element according to actual conditions.

[0115] In the embodiments of the present disclosure, engineering machinery and equipment include but are not limited to excavators, cranes, bulldozers, loaders, etc. The following describes a control system for engineering machinery and equipment, taking an intelligent unmanned excavator as an example.

[0116] FIG3 is a schematic diagram illustrating an architecture system of an intelligent unmanned excavator according to some embodiments of the present disclosure.

[0117] As shown in Figure 3, the environmental perception module 110 (for example, an environmental perception element) is installed on the upper structure of the body and / or the arm (for example, the boom) of the excavator 31. For example, the environmental perception element can be fixed to a movable device on the body or the boom, and the partial structure on the body or the movable device can serve as a multi-degree-of-freedom platform (for example, a controllable platform or a multi-degree-of-freedom structural device). As mentioned above, the multi-degree-of-freedom platform includes but is not limited to: a partial structure on engineering machinery equipment or a separately set device, etc. For example, the multi-degree-of-freedom platform can be the boom 32 or the upper structure of the body 33. That is, the environmental perception element can be installed on the boom 32, or on the top of the upper structure of the body 33. This allows the environmental perception module to adjust its position and viewing angle when the excavator is operating, so as to better perceive the target working surface. The environmental perception module collects environmental information of the target working surface and sends the data to the decision-making and planning module 120.

[0118] For example, the decision-making and planning module 120 can be installed on one or more on-board computing units 34 on the excavator. For another example, under the conditions of meeting bandwidth, latency, reliability, and security, the decision-making and planning module 120 can be partially deployed in a computing unit outside the vehicle, such as a control room 37 of a mine, construction site, or workshop (i.e., edge-to-end deployment), or a dedicated cloud computing center 38 (i.e., cloud-edge-to-end deployment). The decision-making and planning module receives the input of the work task information and outputs the motion trajectory to the control execution module 130 based on the perception results of the environmental perception module. When the work task of the excavator changes, such as the rotation or digging point changes, the decision-making and planning module predicts the impact of the motion trajectory on environmental perception, calculates more appropriate perception parameters, and dynamically adjusts the position and perspective of the environmental perception element of the environmental perception module so that the environmental perception module changes with the movement of the excavator.

[0119] For example, the hardware of the decision-making planning module includes but is not limited to computing units such as industrial computers, boards, and central computers.

[0120] The control execution module 130, serving as the interface for excavator command control and execution data, can control the excavator to complete operational actions such as walking, digging, and working device movement, based on the motion trajectory decisions made by the decision-making and planning module. These operational results serve as input to the environmental perception module. Furthermore, in specific operating conditions, such as emergency avoidance, the control execution module 130 can directly receive some perception results to enable timely environmental response. Furthermore, the control execution module 130 provides feedback to the decision-making and planning module 120 regarding the impact of the work object and the environment on the machine during the control execution process.

[0121] FIG4 is a schematic diagram illustrating a joint coordinate model of a working device of an intelligent unmanned excavator according to some embodiments of the present disclosure.

[0122] Figure 4 is a symmetrical plan view of the working device. The coordinate system involved in Figure 4 adopts a right-hand coordinate system. Therefore, Figure 4 shows two of the three coordinate axes of each coordinate system, and the other coordinate axis perpendicular to these two coordinate axes is not shown. The coordinate system X0Y0Z0 is the reference coordinate system, and O0 is the coordinate origin. Point O is the connection point between the boom and the excavator body. The coordinate system X1Y1Z1 is the coordinate system at point O. m1 is the horizontal offset of the position of the connection point between the boom and the excavator from the reference coordinate origin, and n1 is the vertical offset of the position of the connection point between the boom and the excavator from the reference coordinate origin.

[0123] Point P is the connection point between the boom and the dipper arm. The coordinate system X2Y2Z2 is the coordinate system at point P, a2 is the boom length, and θ2 is the boom joint angle. a3 is the boom length, and θ3 is the boom joint angle. Point Q is the connection point between the boom and the bucket. The coordinate system X3Y3Z3 is the coordinate system at point Q, a4 is the bucket length, and θ4 is the bucket joint angle. is the pitch angle of the shovel tooth. The coordinate system X4Y4Z4 is the coordinate system at the shovel tooth.

[0124] F is the location of the environment perception module installed on the boom, l0 is the distance from the environment perception module's installation location F to point O, β is the pitch angle of the perception module's multi-degree-of-freedom platform, and the coordinate system XgYgZg is the coordinate system at point F.

[0125] Figure 5 is a flow chart illustrating a control method for an intelligent unmanned excavator according to some embodiments of the present disclosure. As shown in Figure 5 , the control method includes steps S501 to S517.

[0126] In step S501, excavator posture information is received, for example, the excavator posture information includes the driving state of the excavator, and the posture information of the working device of the excavator is obtained by an angle measuring device on the working device.

[0127] In step S502, the perception result information is received. For example, the decision planning module may receive relevant parameters from the environment perception module.

[0128] In step S503, geometric relationships are obtained (e.g., the corresponding geometric relationships between the excavation point, the pitch angle of the shovel tooth orientation, and the relative position and posture of the working device relative to the excavator itself). For example, an excavator posture detection fusion system is established. Based on the upper structure of the excavator body including four rigid rotation joints such as the slewing device, the boom, the dipper arm, and the bucket, a three-dimensional coordinate system model of the slewing device, the boom, the environmental perception module (also referred to as the perception system), the dipper arm, and the bucket is established. Within a certain working range, the corresponding geometric relationships between the excavation point, the pitch angle of the shovel tooth orientation, the bucket joint angle, the boom joint angle, and the dipper arm joint angle relative to the body posture are obtained, and the coordinates (Xg, Yg, Zg) of the installation position of the environmental perception module and the values ​​of the horizontal angle α and the pitch angle β of the multi-degree-of-freedom platform of the environmental perception module are solved, as shown in Figure 4.

[0129] In step S504, the excavator's state is planned or determined. For example, the decision-making planning module plans the excavator's operating state. For example, the excavator's operating states include driving, rotating, and digging. The following describes each of the different operating states.

[0130] In step S505, the planning environment perception module is at the highest point. For example, when the excavator is driving, the decision-making planning module formulates a planning control strategy, such as raising the working device so that the camera, lidar and other sensing elements can maintain a sufficient forward field of view.

[0131] In step S506, the excavator boom posture is output. For example, given the coordinate values ​​(Xg, Yg, Zg) of the environmental perception module mounted on the boom, the boom posture is obtained by solving the angle θ2 in Figure 4 and output to the control execution module.

[0132] In step S507, the posture is adjusted. For example, the control execution module adjusts the posture of the working device and controls the working device to move to the planned point position.

[0133] In step S508, parameters are defined. For example, the decision-making and planning module defines parameters for the environmental perception module. For example, if the movement of the working device affects the horizontal angle or pitch angle of the environmental perception module relative to the vehicle body coordinate system, the horizontal angle and pitch angle of the environmental perception module relative to the vehicle body coordinate system in this working state are defined. The horizontal angle α and pitch angle β of the multi-degree-of-freedom platform of the environmental perception module are calculated, and the environmental perception module is adjusted in real time. In step S509, parameters are calculated. For example, parameters such as the focal length of the environmental perception module are calculated.

[0134] For example, when an excavator is rotating, the environmental perception module rotates with the excavator, reducing the amount of information about specific areas or objects within the module's range of perception, or even out of range. The decision-making and planning module calculates and adjusts parameters such as the environmental perception module's focal length to expand its field of view and thereby acquire more perception information.

[0135] In step S510, the sensing range is obtained, that is, the sensing range of the environment sensing module on the target working surface is obtained.

[0136] In step S511, the difference between the sensing range and the target area operation range is compared.

[0137] For example, the 3D information of a specific target perception object or area can be compared with the environmental perception module's field of view information and the target area's operating range information, combined with the module's coordinates (Xg, Yg, Zg) and field of view. During the rotation process, the deviation value is calculated in real time, and the horizontal angle α of the environmental perception module is dynamically adjusted. When the rotation condition is achieved, the environmental perception module has maximum coverage time and space on the target operating surface, thereby improving perception quality.

[0138] In step S512, the excavation point is planned. For example, when the excavator is excavating, the decision-making planning module plans the coordinates of the next excavation point within the excavator's working range based on the perception results at a certain moment.

[0139] In step S513, the coordinates (Xg, Yg, Zg) of the location of the environment perception module as shown in FIG4 are calculated.

[0140] In step S514, it is determined whether the excavation point is within the sensing range. For example, the internal parameter values ​​of the environment sensing element and the pitch angle and horizontal angle of the multi-degree-of-freedom platform are combined to determine whether the excavation point is within the sensing range (i.e., the field of view) of the environment sensing module.

[0141] In step S515, the deviation is calculated. For example, if the excavation point is not within the field of view, the deviation between the field of view center of the environment perception module and the excavation point is calculated, and the intrinsic parameter values ​​of the environment perception element and the parameters (α, β) of the multi-degree-of-freedom platform are adjusted.

[0142] In step S516, the intrinsic parameters of the environment sensing element and / or the parameters of the multi-degree-of-freedom platform are calculated and output. For example, the decision planning module calculates and outputs the intrinsic parameters of the environment sensing element and / or the parameters of the multi-degree-of-freedom platform, thereby outputting corresponding system adjustment instructions.

[0143] In step S517, the internal parameter values ​​of the environment perception element and / or the parameters of the multi-degree-of-freedom platform are adjusted. For example, the environment perception module receives the system adjustment instruction from the decision-making and planning module and adjusts the environment perception posture in real time to better perceive the working environment.

[0144] In addition, the control execution module receives decision-making planning information, performs corresponding actions, and feeds back the execution results to the decision-making planning module for more accurate execution control.

[0145] Thus, a control method for an intelligent unmanned excavator according to some embodiments of the present disclosure has been provided. This method can improve the coordination of the intelligent unmanned excavator under complex and changing operating tasks, objects, and environments. During operation, the intelligent unmanned excavator considers the corresponding movement of the sensing elements and their potential impact on the sensing function while planning and executing the machine's operating behavior, and takes appropriate compensation measures, thereby improving the coordination of the system.

[0146] FIG6 is a block diagram schematically illustrating a control system for engineering machinery equipment according to some other embodiments of the present disclosure. The control system includes a memory 610 and a processor 620.

[0147] The memory 610 may be a disk, a flash memory, or any other non-volatile storage medium. The memory is used to store the instructions in the embodiments corresponding to FIG. 2 and / or FIG. 5 .

[0148] The processor 620 is coupled to the memory 610 and can be implemented as one or more integrated circuits, such as a microprocessor or a microcontroller. The processor 620 is used to execute instructions stored in the memory, which can improve the coordination of the engineering machinery during the operation.

[0149] In one embodiment, as shown in FIG7 , the control system 700 includes a memory 710 and a processor 720. The processor 720 is coupled to the memory 710 via a BUS 730. The control system 700 can also be connected to an external storage device 750 via a storage interface 740 to access external data, and can also be connected to a network or another computer system (not shown) via a network interface 760, which will not be described in detail here.

[0150] In this embodiment, the memory stores data instructions, and the processor processes the instructions, thereby improving the coordination of the engineering machinery during the execution of the operation.

[0151] In some embodiments of the present disclosure, there is further provided an engineering machinery device, including: a control system as described above (e.g., the control system shown in FIG. 1 , FIG. 6 , or FIG. 7 ). For example, the engineering machinery device includes but is not limited to an excavator, a crane, a bulldozer, a loader, and the like.

[0152] In another embodiment, the present disclosure further provides a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) having computer program instructions stored thereon, which, when executed by a processor, implement the steps of the method in the embodiment corresponding to Figure 2 and / or Figure 5. It should be understood by those skilled in the art that the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable non-transitory storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram and the combination of the processes and / or boxes in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0154] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0156] In some embodiments of the present disclosure, a computer program product is further provided, comprising instructions, which, when executed by a processor, enable the processor to perform the control method as described above.

[0157] In some embodiments of the present disclosure, a computer program is further provided, comprising instructions, which, when executed by a processor, cause the processor to perform the control method as described above.

[0158] The present disclosure has been described in detail so far. To avoid obscuring the concept of the present disclosure, some details known in the art have not been described. Based on the above description, those skilled in the art can fully understand how to implement the technical solutions disclosed herein.

[0159] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art will appreciate that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art will appreciate that modifications may be made to the above embodiments without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A control system for engineering machinery equipment, comprising: An environment perception module, used to receive environment information and / or control result information, and output perception result information based on the environment information and / or the control result information, and receive system adjustment instructions, and adjust its own state based on the system adjustment instructions; a decision-making and planning module, configured to receive a work task and the perception result information, generate a motion trajectory based on the work task and the perception result information, output the motion trajectory, predict the motion trajectory to obtain prediction result information, receive feedback result information corresponding to the motion trajectory, and output the system adjustment instruction to the environment perception module based on the prediction result information and the feedback result information; and A control execution module is used to receive the motion trajectory, execute the work task based on the motion trajectory, and obtain the feedback result information and the control result information in the process of executing the work task, output the feedback result information to the decision planning module, and output the control result information to the environmental perception module.

2. The control system according to claim 1, wherein: The environment perception module is also used to directly output the perception result information to the control execution module; The control execution module is also used to receive the perception result information and perform corresponding operations based on the perception result information.

3. A control system according to claim 1 or 2, wherein: The decision-making planning module is used to calculate the adjustment parameters based on the prediction result information and the feedback result information, and output the system adjustment instruction based on the adjustment parameters, wherein the system adjustment instruction includes the adjustment parameters, and the adjustment parameters are used to adjust the state of the environment perception module; The environment perception module is used to extract the adjustment parameters from the system adjustment instructions and adjust its own state based on the adjustment parameters.

4. The control system according to any one of claims 1 to 3, wherein: The feedback result information includes a reaction result of the working object of the engineering machinery equipment.

5. The control system according to any one of claims 1 to 4, wherein: The perception result information includes: the environmental information; or The perception result information includes: a logic judgment result generated based on the environmental information and / or the control result information.

6. The control system according to claim 3, wherein: The environment sensing module comprises: an environment sensing element and a multi-degree-of-freedom platform, wherein the environment sensing element is mounted on the multi-degree-of-freedom platform; The adjustment parameters include: the intrinsic parameter value of the environment sensing element and / or the parameters of the multi-degree-of-freedom platform.

7. The control system according to any one of claims 1 to 6, wherein: The environment perception module is also used to transmit the environment information to the decision-making planning module, wherein the environment information is first data information with the element coordinate system of the environment perception module itself as the coordinate system; The decision planning module is also used to convert the first data information into second data information with the reference coordinate system as the coordinate system based on the transformation matrix between the component coordinate system and the reference coordinate system of the engineering machinery equipment, and generate a motion trajectory based on the work task and the second data information.

8. The control system according to any one of claims 1 to 7, wherein: The decision-making planning module is further used to adjust the motion trajectory based on the perception result information of the environment perception module, and output the adjusted motion trajectory to the control execution module; The control execution module is also used to control the movement of the engineering machinery equipment based on the adjusted movement trajectory.

9. The control system according to any one of claims 1 to 8, wherein: The decision-making planning module is further used to adjust the motion trajectory based on the feedback result information of the control execution module, and output the adjusted motion trajectory to the control execution module; The control execution module is also used to control the movement of the engineering machinery equipment based on the adjusted movement trajectory.

10. The control system according to any one of claims 1 to 9, wherein: The decision-making planning module is also used to send a control instruction to the control execution module to control the operating state or operating speed of the control execution module so that the operating state or operating speed of the control execution module is consistent with the control execution module. The environment perception module adjusts the state to match.

11. A control method for engineering machinery equipment, comprising: Utilizing an environment perception module to receive environment information and / or control result information, and outputting perception result information based on the environment information and / or the control result information; Using a decision-making planning module to receive the operation task and the perception result information, generate a motion trajectory based on the operation task and the perception result information, output the motion trajectory, and predict the motion trajectory to obtain prediction result information; Utilize a control execution module to receive the motion trajectory, execute the operation task based on the motion trajectory, obtain the feedback result information and the control result information in the process of executing the operation task, output the feedback result information to the decision planning module, and output the control result information to the environment perception module; Utilizing the decision-making and planning module to receive feedback result information corresponding to the motion trajectory, and outputting a system adjustment instruction to the environment perception module based on the prediction result information and the feedback result information; and The environment perception module is utilized to receive the system adjustment instruction, and the state of the environment perception module itself is adjusted based on the system adjustment instruction.

12. The control method according to claim 11, further comprising: Utilizing the environment perception module to directly output the perception result information to the control execution module; and The control execution module is used to receive the perception result information, and to perform corresponding operations based on the perception result information.

13. The control method according to claim 11 or 12, wherein: Outputting a system adjustment instruction to the environment perception module based on the prediction result information and the feedback result information by using the decision planning module includes: Utilizing the decision-making planning module to calculate an adjustment parameter based on the prediction result information and the feedback result information, wherein the adjustment parameter is used to adjust the state of the environment perception module; and Outputting the system adjustment instruction based on the adjustment parameter using the decision planning module, wherein the system adjustment instruction includes the adjustment parameter; Utilizing the environment perception module to adjust its own state based on the system adjustment instruction includes: extracting the adjustment parameter from the system adjustment instruction using the environment perception module; and The environment perception module is used to adjust its own state based on the adjustment parameters.

14. The control method according to any one of claims 11 to 13, wherein: The feedback result information includes a reaction result of the working object of the engineering machinery equipment.

15. The control method according to any one of claims 11 to 14, wherein: The perception result information includes: the environmental information; or The perception result information includes: a logic judgment result generated based on the environmental information and / or the control result information.

16. The control method according to any one of claims 11 to 15, further comprising: Using the environment perception module to transmit the environment information to the decision-making and planning module, wherein the environment information is first data information with the element coordinate system of the environment perception module itself as the coordinate system; and The decision planning module is used to convert the first data information into second data information with the reference coordinate system as the coordinate system based on the transformation matrix between the component coordinate system and the reference coordinate system of the engineering machinery equipment, and generate a motion trajectory based on the work task and the second data information.

17. The control method according to any one of claims 11 to 16, further comprising: Using the decision-making planning module to adjust the motion trajectory based on the perception result information of the environment perception module, and output the adjusted motion trajectory to the control execution module; and The control execution module is used to control the movement of the engineering machinery equipment based on the adjusted movement trajectory.

18. The control method according to any one of claims 11 to 17, further comprising: Using the decision-making planning module to adjust the motion trajectory based on the feedback result information of the control execution module, and output the adjusted motion trajectory to the control execution module; and The control execution module is used to control the movement of the engineering machinery equipment based on the adjusted movement trajectory.

19. The control method according to any one of claims 11 to 18, further comprising: The decision-making planning module sends a control instruction to the control execution module to control the control execution module. The operating state or operating speed of the module makes the operating state or operating speed of the control execution module match the state after adjustment of the environment perception module.

20. A control system for engineering machinery equipment, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the control method according to any one of claims 11 to 19 based on instructions stored in the memory.

21. An engineering machinery equipment, comprising: A control system as claimed in any one of claims 1 to 10 or a control system as claimed in claim 20.

22. A computer-readable storage medium having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, implement the control method according to any one of claims 11 to 19.

23. A computer program product, comprising instructions, which, when executed by a processor, cause the processor to perform the control method as described in any one of 11 to 19.

24. A computer program comprising instructions, which, when executed by a processor, cause the processor to perform the control method described in any one of 11 to 19.

Images