Auxiliary operation method and system for excavator

Abstract

The invention discloses an excavator auxiliary operation method and system, belongs to the technical field of excavator equipment, and aims to solve the control problem of excavator auxiliary operation. Comprising the following steps: acquiring a handle key signal for finding out an action instruction sequence stored in a handle key, current data recorded by continuous actions and a movement track of the recorded continuous actions; therefore, control instructions of all execution mechanisms are generated in sequence, and reproduction action image data of the excavator are obtained; guiding the current adjustment of each electromagnetic valve and the main pump according to the current data recorded by the continuous action; and analyzing the image data of the reproduction action of the excavator to obtain a motion trail of the reproduction action of the excavator, comparing the motion trail with the motion trail of the recorded continuous action, judging whether the reproduction action of the excavator is standard or not, and maintaining auxiliary operation according to a standard judgment result. According to the invention, mechanical accumulative errors can be avoided, and safety risks caused by mechanical operation of a driver can be reduced.

Description

Technical Field

[0001] The present invention relates to an excavator auxiliary operation and system, belonging to the field of excavator equipment. Background Art

[0002] In some construction scenarios, excavators or other machinery are required to perform repetitive similar tasks. For example, the excavator's material throwing and the feeding in steel mills both require drivers to perform repetitive operations. After long-term repetition of similar actions, the driver may enter a mechanical operation state, resulting in a decline in attention, which may affect work efficiency and quality and pose safety risks. Especially at night or in bad weather, relying on the driver's visual judgment cannot calibrate repetitive actions.

[0003] Although existing construction machinery has proposed an auxiliary operation mode to enable excavators to perform automatic repetitive operations, when existing excavators perform repetitive operations, they mostly rely on resistive elements or Hall sensors to judge the position of the working device of the excavating machinery. Although the sensors can record the real-time motion data of the actuator, there are other devices in the excavator operation environment, and the sensors are vulnerable to environmental interferences such as vibration and electromagnetic interference, resulting in distorted position data. Secondly, it is difficult for the sensors to comprehensively identify external obstacles, and the safety of automatic operation cannot be guaranteed.

[0004] Therefore, the accuracy of the excavator in performing repetitive actions in the prior art is not high, and it cannot sense the surrounding environment to identify potential risks. The automation performance of the excavator's reproduction action is unstable and unreliable. Summary of the Invention

[0005] The purpose of the present invention is to provide an excavator auxiliary operation and system, which can couple electronic control, monitoring, and image recognition technologies, comprehensively record the continuous action sequence of the excavator, the current change during the action, the real-time posture, and the external environment, and accurately control the action accuracy through current data when the excavator reproduces the action, and combine the image recognition technology to comprehensively judge whether the trajectory of the reproduced action is consistent with the recorded action, so as to avoid mechanical cumulative errors and reduce the safety risks caused by the driver's mechanical operation.

[0006] To solve the above technical problems, the present invention is implemented by the following technical solutions.

[0007] In the first aspect, the present invention provides an excavator auxiliary operation method, including: Obtaining a handle key signal; Searching for the mapping relationship between the action instruction sequence and the handle key based on the handle key signal to obtain the action instruction sequence stored by the handle key, the current data of the continuous action record, and the motion trajectory of the recorded continuous action; Sequentially generate control instructions for each actuator according to the action instruction sequence stored in the handle buttons, trigger each actuator to reproduce the recorded continuous actions, and obtain the image data of the excavator's reproduced actions; Guide the current adjustment of each solenoid valve and main pump according to the current data recorded in the continuous actions, so that the speed, position, and angle of the reproduced actions of each actuator are close to the recorded continuous actions; Analyze the image data of the excavator's reproduced actions to obtain the motion trajectory of the excavator's reproduced actions; According to the comparison result between the motion trajectory of the recorded continuous actions and the motion trajectory of the excavator's reproduced actions, judge whether the excavator's reproduced actions are standard, and based on the standard judgment result, maintain the auxiliary operation.

[0008] Optionally, before obtaining the handle button signal, it also includes defining the mapping relationship between the action instruction sequence and the corresponding handle buttons; among them, a handle button mapping table, a current data storage table, and a motion trajectory storage table are predefined in the memory.

[0009] Optionally, based on the handle button signal, search for the mapping relationship between the action instruction sequence and the handle buttons to obtain the action instruction sequence stored in the handle buttons, the current data recorded in the continuous actions, and the motion trajectory of the recorded continuous actions, including: The driver triggers the action record start and end instructions; After the control terminal receives the action record start instruction, it continuously obtains the excavator operation signals in real time; Sequentially generate control instructions for the relevant actuators according to the continuous excavator operation signals and sequentially generate an action instruction sequence; Based on the control instructions of the relevant actuators, drive each actuator of the excavator to perform sequential continuous actions; and record the current data of the corresponding solenoid valve and main pump changes and obtain the image data of the excavator's actions; After the control terminal receives the action record end instruction, it analyzes the image data of the excavator's actions to obtain the motion trajectory of the recorded actions of the excavator; Store the action instruction sequence, the current data of the corresponding solenoid valve and main pump changes, and the motion trajectory of the recorded actions of the excavator into the handle button mapping table, the current data storage table, and the motion trajectory storage table respectively; among them, if the mapping table corresponding to the handle button stores the action instruction sequence recorded last time, clear the mapping table, the current data storage table, and the motion trajectory storage table to overwrite the action instruction sequence, current data, and motion trajectory recorded last time; The action instruction sequence, the current data of the corresponding solenoid valve and main pump changes, and the motion trajectory of the recorded actions of the excavator are used to assist the excavator in reproducing the recorded actions.

[0010] Optionally, the driver triggers the start and end commands for action recording, including: inputting the start and end commands for action recording by any one or a combination of methods such as keys, voice, and somatosensory control.

[0011] Optionally, recording the current data of the corresponding solenoid valves and main pump changes and obtaining the image data of the excavator actions, including: Obtaining the current values of the main pump drive motor and each solenoid valve at fixed time intervals and storing them as the initial current data set; Processing the initial current data set using second-order filtering to obtain the current data with spikes and jitters removed; After each actuator of the excavator starts to move, continuously capturing frame images of the excavator actions at fixed time intervals.

[0012] Optionally, analyzing the image data of the excavator actions to obtain the motion trajectory of the recorded actions of the excavator, including: Using the edge detection algorithm to extract features from each frame image to obtain the outline of the excavator actuators in each frame image; Based on shape matching or the excavator pose model, identifying the key points of the outline of the excavator actuators in each frame image to obtain the key point coordinates of each actuator of the excavator in each frame image; where the key points include the boom, arm, bucket, and slewing mechanism; Calculating the displacement of the key point coordinates of each actuator in adjacent frame images respectively to obtain the motion trajectory of the continuous actions recorded by the excavator.

[0013] Optionally, using the edge detection algorithm to extract features from each frame image to obtain the outline of the excavator actuators in each frame image, further including: Performing repeated edge detection and contour extraction on the outer edge area of the outline of the excavator actuators to obtain the outlines of other devices and obstacles similar to the excavator actuators; Calculating the relative distance between the outlines of other devices and obstacles and the outline of the excavator actuators; Generating an excavator stop command based on the comparison result between the preset safety threshold and the relative distance.

[0014] Optionally, based on the comparison result between the motion trajectory of the recorded continuous actions and the motion trajectory of the excavator's reproduced actions, determining whether the excavator's reproduced actions are standard and maintaining the auxiliary operation according to the standard judgment result, including: Respectively extracting the changes in the cylinder strokes of the boom, bucket, and arm in the motion trajectory of the recorded continuous actions and the motion trajectory of the excavator's reproduced actions; Calculate the position of the change in the cylinder stroke of the boom, bucket, and arm in the motion trajectory of the recorded continuous action and the motion trajectory of the reproduction action of the excavator to obtain the position deviation value between the recorded action and the reproduction action; Calculate the compensation current values of each solenoid valve and the main pump according to the position deviation value, and use them to compensate and control the current of each solenoid valve and the main pump, so that the change in the cylinder of each actuator is consistent with the cylinder stroke of the recorded action.

[0015] In a second aspect, the present invention provides an excavator auxiliary operation system, including: a control terminal, a multi-functional handle, a variety of actuators, and a camera device; The control terminal is built-in with a controller, a memory, and an image processing chip, and is used to execute the steps of the method according to any one of claims 1 to 8; The multi-functional handle is electrically connected to the control device, and more than one button is provided on the multi-functional handle, and the mapping relationship between each button and the recorded action is defined through the control device; The variety of actuators include a boom, an arm, a bucket, and a slewing mechanism; The camera device communicates with the image processing chip and is used to upload the image data of the actions of each actuator of the excavator; The image processing chip extracts the motion trajectory of the recorded action of the excavator from the image data and stores and extracts the motion trajectory of the reproduction action; calculates whether the reproduction action of the excavator meets the position accuracy standard according to the motion trajectories of the recorded action and the reproduction action; the image processing chip also extracts the information of the obstacles around the excavator from the image data; and sends the position accuracy of the reproduction action of the excavator and the information of the obstacles around the excavator to the controller.

[0016] Optionally, it further includes an output device, and the output device includes an instrument and a display; it is used to visually output the image data of the real-time operation of the excavator, the deviation of the motion trajectory of the reproduction action, and the obstacles around the excavator.

[0017] Compared with the prior art, the beneficial effects achieved by the present invention are: 1. Simplify the difficulty of interaction between the driver and the control terminal. The driver only needs to trigger the start and end commands of action recording and operate the multi-functional handle, and the control terminal can automatically record the action and take pictures of the excavator state, and regulate the current of each solenoid valve and the main pump in combination with the current data of all solenoid valves and the main pump recorded. The reference provided for each repeated action can be consistent with the recorded composite action. Finally, combined with the image data, accurately judge the accuracy of the reproduction action and the external environment of the excavator to ensure the operation accuracy and operation safety.

[0018] 2. By integrating a control device, a multi-functional handle, multiple actuators, and a camera device, the operation efficiency, accuracy, and safety of the excavator are improved. This system can process and analyze the data at the operation site in real time, improve the accuracy of the excavator's repetitive operations, provide auxiliary decision-making support for the operator, and simplify the operation process of the excavator, thereby reducing repetitive work, reducing the safety risks caused by the mechanical operations of the driver, and improving work efficiency and quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 The figure shows a flowchart of the excavator action recording method of the present invention; Figure 2 The figure shows a flowchart of the excavator auxiliary operation of the present invention; Figure 3 The figure shows a structural diagram of the excavator action recording system of the present invention; Figure 4 The figure shows an installation embodiment diagram of the excavator action recording system of the present invention.

[0020] In the figure: 1. VCU controller; 2. Instrument; 3. Multi-functional handle; 4. Boom; 5. Image processing chip; 6. Slewing mechanism; 7. Camera device. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and cannot be used to limit the protection scope of the present invention. Embodiment

[0022] This embodiment provides an excavator auxiliary operation method, as Figure 1 shown including: Step 1: Obtain the handle key signal; <� Step 2: Based on the handle key signal, search for the mapping relationship between the action instruction sequence and the handle key, and obtain the action instruction sequence stored by the handle key, the current data of the continuous action record, and the movement trajectory of the recorded continuous action; Step 3: Sequentially generate control instructions for each actuator according to the action instruction sequence stored by the handle key, trigger each actuator to reproduce the recorded continuous action, and obtain the excavator reproduction action image data; Step 4: Guide the current adjustment of each solenoid valve and the main pump according to the current data of the continuous action record, so that the speed, position, and angle of the reproduced action of each actuator are close to the recorded continuous action; Step 5: Analyze the excavator reproduction action image data to obtain the movement trajectory of the excavator reproduction action; Step 6: According to the comparison result between the motion trajectory of the recorded continuous actions and the motion trajectory of the excavator's reproduced actions, determine whether the excavator's reproduced actions are standard, and based on the standard judgment result, maintain the auxiliary operation. In this embodiment, after starting to record the actions, the current change data of each actuator and the posture of the continuous actions of the excavator are recorded simultaneously. The key operation is associated with the specific actions through the action instruction sequence, providing detailed parameters for action reproduction and enabling the excavator to operate according to the preset action sequence. At the same time, the reproduction process is recorded for subsequent analysis. The recorded current data is used as the ideal value for guiding the repeated actions, accurately controlling the current, improving the accuracy of action reproduction, reducing errors, and finally, through image processing technology, directly obtaining the actual motion path of the excavator to provide a data basis for subsequent comparison.

[0023] Optionally, before obtaining the handle key signal in Step 1, the mapping relationship between the action instruction sequence and the corresponding handle key needs to be defined; among them, the handle key mapping table, current data storage table, and motion trajectory storage table are predefined in the memory. In this embodiment, before recording the actions, the memory data structure is preconfigured in advance to ensure the efficient operation of the system.

[0024] Optionally, as Figure 2 shown, in Step 2, based on the handle key signal, searching for the mapping relationship between the action instruction sequence and the handle key to obtain the action instruction sequence stored by the handle key, the current data of the continuous action record, and the motion trajectory of the recorded continuous action, including: Step 2.1: The driver triggers the start and end instructions for action recording; Step 2.2: After the control terminal receives the action recording start instruction, it continuously obtains the operation signals of the excavator in real time; Step 2.3: According to the continuous operation signals of the excavator, sequentially generate control instructions for the relevant actuators and sequentially generate an action instruction sequence; Step 2.4: Based on the control instructions of the relevant actuators, drive each actuator of the excavator to perform continuous actions in sequence; and record the current data of the corresponding solenoid valve and main pump changes and obtain the image data of the excavator's actions; Step 2.5: After the control terminal receives the action recording end instruction, it analyzes the image data of the excavator's actions to obtain the motion trajectory of the recorded actions of the excavator; Step 2.6: Store the action instruction sequence, the current data of the corresponding solenoid valve and main pump changes, and the motion trajectory of the recorded actions of the excavator into the handle key mapping table, current data storage table, and motion trajectory storage table respectively; among them, if the handle key mapping table stores the action instruction sequence recorded last time, clear the mapping table, current data storage table, and motion trajectory storage table to overwrite the action instruction sequence, current data, and motion trajectory recorded last time; The action instruction sequence, the corresponding solenoid valves, the changing current data of the main pump, and the movement trajectory of the actions recorded by the excavator are used to assist the excavator in reproducing the recorded actions.

[0025] In order to meet the various complex working conditions requirements of excavator operations and reduce repetitive operations in this embodiment, for example, when loading materials in a steel mill, a series of actions such as bucket retraction, boom lifting, rotation, boom lowering, and bucket release need to be continuously repeated. After long-term repetition of similar actions, the driver may enter a mechanical operation state, resulting in a decline in attention, which may affect work efficiency and quality and pose safety risks. The method of this embodiment can record action parameters, current data, and image data of continuous actions at the same time. Through precise drive of electrical signals, real-time verification of visual images, and risk monitoring of movement trajectories and postures, a closed-loop control of excavator action recording, reproduction, and adjustment is realized. It can effectively simplify the operation process and avoid safety risks caused by mechanical operations of the driver under various complex environments such as meeting accuracy, environment, and excavator load conditions.

[0026] Optionally, the driver triggers the start and end instructions for action recording, including: inputting the start and end instructions for action recording through any one or a combination of methods such as keys, voice, and somatosensory control; in this embodiment, after the vehicle is started, the user can select the action recording option on the instrument panel.

[0027] Optionally, in step 3, recording the changing current data of the corresponding solenoid valves and the main pump and obtaining the image data of the excavator actions includes: Step 3.1: Obtain the current values of the main pump drive motor and each solenoid valve at fixed time intervals and store them as the initial current data set. Step 3.2: Process the initial current data set using second-order filtering to obtain current data with spikes and jitters removed. Step 3.3: After each actuator of the excavator starts to move, collect consecutive frame images of the excavator actions at fixed time intervals.

[0028] During the action recording of this embodiment, the VCU will record the changes of all solenoid valves and the main pump current, and sort them according to the operation sequence. Sample all the current data output to the solenoid valves and the main pump at the 10ms operation cycle of the controller. After the current data is collected, adjust the spikes and jitters in the data, use second-order limited wave to eliminate unstable current signals, and only store the relatively stable current data in the controller.

[0029] Optionally, in step 5, parsing the image data of the excavator actions to obtain the movement trajectory of the actions recorded by the excavator includes: Step 5.1: Use the edge detection algorithm to extract features from each frame of image to obtain the contour of the excavator actuator in each frame of image. Step 5.2: Based on shape matching or the excavator posture model, key point recognition is performed on the contour of the excavator actuator in each frame image to obtain the key point coordinates of each actuator of the excavator in each frame image; the key points include the boom, dipper arm, bucket, and slewing mechanism; Step 5.3: Perform displacement calculation on the key point coordinates of each actuator in adjacent frame images to obtain the motion trajectory of the continuous action recorded by the excavator.

[0030] The key point identification in this embodiment also includes the identification of cylinder stroke, and real-time acquisition of timing data of each cylinder piston to avoid sudden changes in cylinder length caused by single-frame false detection. In particular, the boom cylinder stroke is used as a comparison feature to confirm whether the subsequent reproduced action is exactly the same as the previously recorded picture group. The instrument will also display the camera image. Since a group of images of the recorded action have been collected before, the position of each cylinder in the next 0.1 second can also be calculated through the above analysis, thereby obtaining the position of the working device. The activity range of the working device for the recorded action can be marked on the camera image displayed on the instrument, making the comparison effect more intuitive.

[0031] Optionally, in step 5.1, the step of extracting features from each frame of image using an edge detection algorithm to obtain the outline of the excavator actuator in each frame of image may further include: Repeated edge detection and contour extraction are performed on the outer edge area of the excavator actuator to obtain the contours of other equipment and obstacles similar to the excavator actuator; Calculate the relative distances of other equipment and obstacle outlines to the outline of the excavator's actuators; Based on the comparison result of the preset safety threshold and the relative distance, an excavator stop instruction is generated.

[0032] This embodiment also provides real-time obstacle information feedback via an output device. If the relative distance between the obstacle outline and the excavator's actuators falls below a preset safety threshold, the display will show the movement paths of the excavator and its actuators, as well as the real-time distances between the actuators and the obstacle. The vehicle will automatically exit assisted driving mode and stop. If the relative distance between the obstacle outline and the excavator's actuators falls below the preset safety threshold, only the movement paths of the excavator and its actuators will be displayed.

[0033] Optionally, in step 6, judging whether the excavator's reproduced action is standard based on the comparison result of the motion trajectory of the recorded continuous action with the motion trajectory of the excavator's reproduced action and maintaining the auxiliary operation based on the standard judgment result includes: Step 6.1: Extract the cylinder stroke changes of the boom, bucket, and dipper arm from the motion trajectory of the recorded continuous action and the motion trajectory of the excavator's reproduced action respectively; Step 6.2: Calculate the change in the cylinder stroke of the boom, bucket, and arm in the recorded continuous motion trajectory and the motion trajectory of the excavator's reproduction motion to obtain the position deviation value between the recorded motion and the reproduction motion; Step 6.3: Calculate the compensation current values for each solenoid valve and main pump according to the position deviation value, which are used to compensate and control the current of each solenoid valve and main pump, so that the change in the cylinder of each actuator is consistent with the cylinder stroke of the recorded motion.

[0034] This embodiment is not limited to using the SURF algorithm or machine learning for feature extraction, calculation, and comparison, effectively improving the cylinder accuracy of the boom, bucket, and arm in the motion trajectory of the excavator's reproduction motion, making its position at different times infinitely close to the motion trajectory of the recorded continuous motion, reducing the position deviation caused by time and speed differences. In addition, the present invention pre-establishes a table or database containing solenoid valve models, working parameters, position deviation values, and corresponding compensation current values. Query the corresponding compensation current value according to the solenoid valve and working parameters associated with the position deviation value. Of course, the compensation current value is not limited to being obtained by looking up the table, and can also establish a mapping relationship between the position deviation value and the compensation current value through a big data training model. Regression models (such as linear regression, support vector regression) or deep learning models (such as neural networks) can be used for training, and the database or model can be updated according to the changes in the actual working conditions to improve the adaptability and robustness of the system. Embodiment

[0035] This embodiment provides an excavator auxiliary operation system, as Figure 3 and Figure 4 shown, including: a control terminal, a multi-functional handle 3, multiple actuators, and a camera device 7; The control terminal is built-in with a VCU controller 1, a memory, and an image processing chip 5, which is used to execute the steps of the method described in Embodiment 1; The multi-functional handle 3 is electrically connected to the control device, and more than one button is provided on the multi-functional handle 3, and the mapping relationship between each button and the recorded motion is defined through the control device; The multiple actuators include a boom 4, an arm, a bucket, and a slewing mechanism 6; The camera device 7 communicates with the image processing chip 5 and is used to upload the image data of the actions of each actuator of the excavator; The image processing chip 5 extracts the motion trajectory of the recorded motion of the excavator from the image data and stores and extracts the motion trajectory of the reproduction motion; calculates whether the reproduction motion of the excavator meets the position accuracy standard according to the motion trajectories of the recorded motion and the reproduction motion; the image processing chip 5 also extracts the information of the obstacles around the excavator from the image data; and sends the position accuracy of the excavator's reproduction motion and the information of the obstacles around the excavator to the VCU controller 1.

[0036] In this embodiment, a VCU controller 1, a memory, and an image processing chip 5 are integrated to achieve vehicle control and detect image processing. The multifunctional handle 3 is electrically connected to the control port of the VCU controller 1, and the mapping relationship between the editable operation instructions and the buttons can be associated. The camera device 7 is used to collect the excavator and the working environment in real time and upload it to the image processing chip 5 to extract the trajectory of the actuator in real time, calculate and reproduce the position accuracy of the action and the safety distance of the obstacle. Finally, the image processing chip 5 sends the position accuracy data and the obstacle information to the VCU controller 1 through the CAN bus to support the VCU controller 1 to dynamically adjust the operation strategy. Compared with the existing auxiliary operation systems and methods, it can significantly improve the automatic operation ability and safety of the excavator, and is suitable for precise construction scenarios under complex working conditions. When in use, the user can edit the names of all recorded actions to facilitate memorizing the corresponding button actions. The user can select the actions preset on the multifunctional handle 3 according to different construction scenarios and adapt to more construction scenarios through different permutations and combinations. It should be noted that this solution is only for auxiliary driving, and the driver still needs to pay attention to construction safety at all times and dynamically adjust the recorded actions according to the changes in the environment to cover the actions that have been invalidated and are no longer used. In addition, some preset actions are also provided at the factory, such as rotating 90 degrees, turning about 90 degrees left and right, fully retracting and extending the bucket, etc. The user can customize and place these actions on the buttons they want to preset.

[0037] Optionally, an output device is further included, and the output device includes an instrument 2 and a display; it is used for visually outputting the image data of the excavator's real-time operation, the deviation of the reproduced action movement trajectory, and the obstacles around the excavator. In this embodiment, the display device intuitively displays the deviation of the reproduced action movement trajectory, such as the position, speed, and angle deviation of the specific key points of the boom 4, the stick, the bucket, and the slewing mechanism 6, etc., which can effectively reduce mechanical errors and, through the combination of the graphical interface and the numerical instrument 2, reduce the difficulty of the operator in interpreting complex data. In addition, the camera screen is directly displayed so that the driver can observe the on-site safety panoramically.

[0038] In summary, the difficulty of the interaction between the driver and the control terminal is simplified. The driver only needs to trigger the start and end commands of the action recording and operate the multi-functional handle, then the control terminal can automatically record the actions and capture the status of the excavator, and regulate the current of each solenoid valve and the main pump by combining the current data of all solenoid valves and the main pump recorded. This provides a reference for each repeated action, which can be consistent with the recorded composite action. Finally, the accuracy of the reproduced action and the external environment of the excavator are accurately judged by combining the image data to ensure the operation accuracy and safety. By integrating the control device, multi-functional handle, various actuators and camera device, the operation efficiency, accuracy and safety of the excavator are improved. This system can process and analyze the data on the operation site in real time, improve the accuracy of the repeated operation of the excavator, provide auxiliary decision-making support for the operator, and simplify the operation process of the excavator, thereby reducing repetitive work, reducing the safety risks caused by the mechanical operation of the driver, and improving the work efficiency and quality.

[0039] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0040] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0041] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0042] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 and / or steps for implementing the functions specified in one block or a plurality of blocks.

[0043] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these fall within the protection scope of the present invention.

Claims

1. An auxiliary operation method for an excavator, characterized in that Including: Obtaining the handle key signal; Searching for the mapping relationship between the action instruction sequence and the handle key based on the handle key signal to obtain the action instruction sequence stored by the handle key, the current data of the continuous action record, and the motion trajectory of the recorded continuous action; Sequentially generating control instructions for each actuator according to the action instruction sequence stored by the handle key, triggering each actuator to reproduce the recorded continuous action and obtaining the excavator reproduction action image data; Guiding the current adjustment of each solenoid valve and the main pump according to the current data of the continuous action record, so that the speed, position, and angle of the reproduced action of each actuator are close to the recorded continuous action; Analyzing the excavator reproduction action image data to obtain the motion trajectory of the excavator reproduction action; Judging whether the excavator reproduction action is standard according to the comparison result between the motion trajectory of the recorded continuous action and the motion trajectory of the excavator reproduction action, and maintaining the auxiliary operation according to the standard judgment result.

2. The auxiliary operation method of the excavator according to claim 1 further includes defining a mapping relationship between the action instruction sequence and the corresponding handle buttons before obtaining the handle button signals; wherein, Pre-defining a handle key mapping table, a current data storage table, and a motion trajectory storage table in the memory.

3. The excavator auxiliary operation method according to claim 2, searching for the mapping relationship between the action instruction sequence and the handle key based on the handle key signal to obtain the action instruction sequence stored by the handle key, the current data of the continuous action record, and the motion trajectory of the recorded continuous action, including: The driver triggers the action record start and end instructions; After the control terminal receives the action record start instruction, it real-time obtains continuous excavator operation signals; Sequentially generating control instructions for relevant actuators according to the continuous excavator operation signals and sequentially generating an action instruction sequence; Based on the control instructions of the relevant actuators, driving each actuator of the excavator to perform continuous actions in sequence; and recording the current data of the corresponding solenoid valve and the main pump changes and obtaining the image data of the excavator action; After the control terminal receives the action record end instruction, it analyzes the image data of the excavator action to obtain the motion trajectory of the recorded action of the excavator; Storing the action instruction sequence, the current data of the corresponding solenoid valve and the main pump changes, and the motion trajectory of the recorded action of the excavator into the handle key mapping table, the current data storage table, and the motion trajectory storage table respectively; wherein, if the mapping table corresponding to the handle key stores the action instruction sequence recorded last time, clear the mapping table, the current data storage table, and the motion trajectory storage table to overwrite the action instruction sequence, the current data, and the motion trajectory recorded last time; The action instruction sequence, the current data of the corresponding solenoid valve and the main pump changes, and the motion trajectory of the recorded action of the excavator are used to assist the excavator in reproducing the recorded action.

4. The auxiliary operation method of the excavator according to claim 3, characterized in that, The driver triggers the action record start and end instructions, including: inputting the action record start and end instructions in any one way or a combination of multiple ways of button, voice, and somatosensory control.

5. The auxiliary operation method of the excavator according to claim 3, characterized in that Recording the current data of the corresponding solenoid valve and the main pump changes and obtaining the image data of the excavator action, including: Obtaining the current values of the main pump drive motor and each solenoid valve at fixed time positions and storing them as the initial current data set; The initial current data set is processed using second-order filtering to obtain current data with spikes and jitter eliminated. After each actuator of the excavator starts to move, continuous frame images of the excavator's movement are collected at fixed time intervals.

6. The auxiliary operation method of an excavator according to claim 5, characterized in that, The image data of the excavator's movements are analyzed to obtain the motion trajectory of the excavator's recorded movements, including: The edge detection algorithm is used to extract features from each frame of image to obtain the outline of the excavator's actuator in each frame of image; Based on shape matching or the excavator posture model, the key points of the excavator actuator contour in each frame image are identified to obtain the key point coordinates of each actuator in each frame image; the key points include the boom, dipper arm, bucket, and slewing mechanism; The displacement of the key point coordinates of each actuator in adjacent frame images is calculated respectively to obtain the motion trajectory of the continuous action recorded by the excavator.

7. The method for recording the actions of an excavator according to claim 6, characterized in that, The edge detection algorithm is used to extract features from each frame of the image to obtain the outline of the excavator actuator in each frame of the image, which also includes: Repeated edge detection and contour extraction are performed on the outer edge area of the excavator actuator to obtain the contours of other equipment and obstacles similar to the excavator actuator; Calculate the relative distances of other equipment and obstacle outlines to the outline of the excavator's actuators; Based on the comparison result of the preset safety threshold and the relative distance, an excavator stop instruction is generated.

8. The auxiliary operation method of an excavator according to claim 6, wherein Based on the comparison results of the motion trajectory of the recorded continuous action and the motion trajectory of the excavator's reproduced action, it is judged whether the excavator's reproduced action is standard and based on the standard judgment result, the auxiliary operation is maintained, including: Extract the cylinder stroke changes of the boom, bucket, and dipper arm from the motion trajectory of the recorded continuous action and the motion trajectory of the excavator's reproduced action respectively; The position calculation is performed on the cylinder stroke changes of the boom, bucket, and dipper arm in the motion trajectory of the recorded continuous action and the motion trajectory of the excavator's reproduced action to obtain the position deviation value of the recorded action and the reproduced action; The compensation current value of each solenoid valve and main pump is calculated based on the position deviation value, which is used to compensate and control the current of each solenoid valve and main pump so that the change of each actuator cylinder is consistent with the cylinder stroke of the recorded action.

9. An excavator auxiliary operation system, characterized in that, include: Control terminal, multi-function handle, various actuators and camera devices; The control terminal has a built-in controller, a memory, and an image processing chip, and is used to execute the steps of the method according to any one of claims 1 to 8; The multifunctional handle is electrically connected to the control device. The multifunctional handle is provided with one or more buttons. The mapping relationship between each button and the recorded action is defined by the control device. The various actuators include a boom, an arm, a bucket and a slewing mechanism; The camera device communicates with the image processing chip to upload image data of the actions of the various actuators of the excavator; The image processing chip extracts the motion trajectory of the actions recorded by the excavator based on the image data, stores and extracts the motion trajectory of the reproduced actions; calculates whether the reproduced actions of the excavator meet the position accuracy standard according to the motion trajectories of the recorded actions and the reproduced actions; the image processing chip also extracts the information of the obstacles around the excavator based on the image data; and sends the position accuracy of the reproduced actions of the excavator and the information of the obstacles around the excavator to the controller.

10. The auxiliary operation system of an excavator according to claim 9, characterized in that, It further includes an output device, and the output device includes an instrument and a display; it is used to visually output the image data of the real-time operation of the excavator, the deviation of the motion trajectory of the reproduced actions, and the obstacles around the excavator.

Images