Operating end, working condition distinguishing method and system thereof and engineering machinery

Abstract

The invention discloses a working end, a working condition distinguishing method and system thereof and engineering machinery, and relates to the technical field of engineering machinery. The method is applied to the engineering machinery which drives a mechanical arm to act through an oil cylinder so as to adjust the attitude of a working end, and comprises the following steps: acquiring stroke data of the oil cylinder of the engineering machinery and a working state image of the working end, and determining standard attitude data of the working end corresponding to the stroke data based on the stroke data; and determining actual attitude data of the operation end based on the working state image, and finally determining whether the operation end is replaced based on the relationship between the standard attitude data and the actual attitude data. Therefore, the problems of maintenance resource mismatching, low after-sales service efficiency and even secondary faults caused by the fact that potential risks are not thoroughly solved due to the fact that design and manufacturing defects, illegal modification of clients or improper operation cannot be distinguished by means of the current method when the engineering machinery breaks down are solved, and the service quality of the engineering machinery is improved. And the public praise and the market competitiveness of enterprise products are seriously influenced.

Description

Technical Field

[0001] This application relates to the field of engineering machinery technology, and more specifically, to a working end and its working condition determination method, system, and engineering machinery. Background Technology

[0002] In the field of construction machinery, such as excavators, bulldozers, and loaders, fatigue cracking has always been a key focus of after-sales quality analysis because the robotic arm and working end are critical structural components that bear the load.

[0003] However, significant technical challenges exist in actual after-sales fault diagnosis. For example, when customers replace the original standard working end with a non-original standard working end without authorization, the stress conditions of the working device will change. Factors such as differences in the weight of the working end and mismatch in structural stiffness will introduce abnormal load distribution. In harsh working conditions such as mining and tunnel excavation, rough operation will exacerbate fatigue damage to structural components.

[0004] Currently, due to a lack of effective traceability methods and operational monitoring data, after-sales technicians struggle to accurately determine the root cause of cracking failures in the working device. They cannot distinguish whether the cause is a design or manufacturing defect, unauthorized customer modifications, or improper operation. This diagnostic ambiguity not only leads to misallocation of maintenance resources and inefficient after-sales service, but also risks secondary failures due to the failure to thoroughly address potential risks, severely impacting the company's product reputation and market competitiveness. Summary of the Invention

[0005] In view of this, the embodiments of this application are committed to providing a working end and its working condition discrimination method, system and construction machinery, to solve the problem that when construction machinery fails, current means cannot distinguish whether the failure is caused by design and manufacturing defects, customer illegal modification or improper operation, resulting in misallocation of maintenance resources, low efficiency of after-sales service, and even secondary failures caused by failure to completely resolve potential risks, which seriously affect the reputation of enterprise products and market competitiveness.

[0006] In a first aspect, the present invention provides a working end and a method for determining its working condition, applicable to engineering machinery that uses hydraulic cylinders to drive the movement of a robotic arm to adjust the posture of the working end, comprising: Acquire the stroke data of the hydraulic cylinder and the working status image of the working end; Based on a preset correspondence, standard posture data corresponding to the stroke data of the working end is determined. The preset correspondence is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original working end of the construction machinery. Based on the working status image, determine the actual posture data of the working end; Based on the relationship between the standard attitude data and the actual attitude data, it is determined whether the operating terminal has been replaced.

[0007] In one possible implementation, the working terminal and its operating condition determination method further include: Acquire stress data, which is obtained by stress gauges arranged on the robotic arm; Based on the changes in the stress data, it is determined whether the working end has been replaced or is operating under the preset normal working conditions.

[0008] In one possible implementation, the working terminal and its operating condition determination method further include: Based on the working status image, the motion trajectory of the working end is determined; Based on the motion trajectory, it is determined whether the working end is operating under preset normal conditions.

[0009] In one possible implementation, the working terminal and its operating condition determination method further include: Based on the working status image, determine the environmental information of the working terminal; Based on the stress data, the deformation information of the robotic arm is determined; Based on the environmental information and the deformation information, it is determined whether the working end is operating under the preset normal working conditions.

[0010] In one possible implementation, the working terminal and its operating condition determination method further include: When it is determined that the working end has been replaced or is not working under the preset normal working conditions, an anomaly record is made, and when the stress data exceeds the preset upper limit, the engineering machinery is triggered to reduce the output power.

[0011] In one possible implementation, the method for determining the preset correspondence includes: Corresponding to the travel data, the position and orientation of each branch arm component that makes up the robotic arm are obtained as the attitude vector of each branch arm component; Transform all the aforementioned attitude vectors to the same coordinate system; Based on the attitude vector transformed to the same coordinate system, the attitude data of the original working end is determined; The attitude data of the original working end is used as the standard attitude data and matched with the travel data to obtain the preset correspondence.

[0012] In one possible implementation, determining whether the working end is operating under a preset normal working condition based on the motion trajectory includes: Input the motion trajectory into the trained working condition recognition model; Based on the output of the working condition identification model, it is determined whether the working terminal is working under the preset normal working condition; The working condition identification model is a deep learning model trained on sample data. The sample data includes target point cloud data of different models of target working terminals, point cloud data sequences of the target working terminals operating under preset normal and abnormal working conditions, target point cloud data after first preprocessing, and point cloud data sequences after second preprocessing. The first preprocessing includes random rotation, scaling, and adding noise. The second preprocessing includes random cropping or repeating parts of the point cloud data sequence.

[0013] In one possible implementation, the working terminal and its operating condition determination method further include: The travel data, the working status image, and the stress data are aligned using timestamps.

[0014] Secondly, the present invention provides a working end and its working condition discrimination system, applied to engineering machinery that uses hydraulic cylinders to drive the movement of a robotic arm to adjust the posture of the working end, comprising: The data acquisition unit is used to acquire the stroke data of the hydraulic cylinder and the working status image of the working end; The first data processing unit is used to determine the standard posture data of the working end corresponding to the stroke data based on a preset correspondence relationship, wherein the preset correspondence relationship is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original working end of the construction machinery. The second data processing unit is used to determine the actual posture data of the working end based on the working status image; The third data processing unit is used to determine whether the working end has been replaced based on the relationship between the standard attitude data and the actual attitude data.

[0015] Thirdly, the present invention provides an engineering machinery, including the engineering machinery body and the working end and its working condition discrimination system provided in the second aspect of the present invention.

[0016] The present invention provides a working end and its working condition determination method, applicable to construction machinery that uses hydraulic cylinders to drive the robotic arm to adjust the posture of the working end. It acquires the stroke data of the hydraulic cylinders and the working state image of the working end. Then, based on the correspondence between the hydraulic cylinder stroke data and the posture data of the original working end of the construction machinery, it determines the standard posture data of the working end corresponding to that stroke data. Based on the working state image, it determines the actual posture data of the working end. Finally, based on the relationship between the standard posture data and the actual posture data, it determines whether the working end has been replaced. Thus, by combining the hydraulic cylinder stroke data of the construction machinery with the working state image of the working end, accurate determination of the working end is achieved, with low implementation difficulty and high operating efficiency. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 The diagram shows a flowchart of a working terminal and its working condition determination method provided by an embodiment of the present invention.

[0019] Figure 2 The image shown is a diagram of the original bucket's posture when the hydraulic cylinder of an excavator extends to a preset length, according to an embodiment of the present invention.

[0020] Figure 3 The diagram shown illustrates the technical route of the working end and its working condition discrimination method provided in this embodiment of the invention, using an excavator as an example.

[0021] Figure 4 The diagram shown is a structural diagram of a working terminal and its working condition discrimination system provided in an embodiment of the present invention.

[0022] Figure 5 The diagram shown is a structural diagram of another type of working terminal and its working condition discrimination system provided in an embodiment of the present invention.

[0023] Figure 6 The diagram shown is a structural schematic of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] Unless otherwise defined, the technical or scientific terms used in the embodiments of this specification shall have the ordinary meaning understood by one of ordinary skill in the art to which this specification pertains. The terms "first," "second," and similar terms used in the embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to avoid confusion of constituent elements.

[0025] Unless the context otherwise requires, throughout this specification, "a plurality of" means "at least two," and "including" is interpreted as open-ended or encompassing, that is, "including, but not limited to." In the description of this specification, terms such as "one embodiment," "some embodiments," "exemplary embodiment," "example," "specific example," or "some examples" are intended to indicate that a particular feature, structure, material, or characteristic associated with that embodiment or example is included in at least one embodiment or example of this specification. The illustrative representations of the above terms do not necessarily refer to the same embodiment or example.

[0026] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0027] Understandably, by identifying the operating end of construction machinery and monitoring its working condition in real time, it is possible to monitor whether the machinery has been illegally modified or is operating beyond its limits. This facilitates determining whether a malfunction such as cracking of the working device is due to a design or manufacturing defect or human error.

[0028] Furthermore, current monitoring of construction machinery operating conditions typically employs a combination of image acquisition and operational data. This involves processing and detecting images of the acquired machinery to determine its posture, then comparing this posture with the corresponding operational stage to determine if the posture matches the operational stage. However, this monitoring method requires the coordinated operation of multiple modules, including image acquisition, image processing, image detection, and data processing, which presents challenges such as difficulty in inter-module coordination, complex integration, and high development costs.

[0029] This invention aims to solve the aforementioned problems by combining the stroke data of the hydraulic cylinders of construction machinery with working status images. This reduces the need for modular applications and enables accurate monitoring of whether the working end of construction machinery has been illegally replaced. The method is simple and easy to implement. Simultaneously, it addresses the issue that current methods cannot distinguish between design and manufacturing defects, unauthorized customer modifications, and improper operation when construction machinery malfunctions. This leads to misallocation of maintenance resources, inefficient after-sales service, and even secondary malfunctions caused by failure to completely resolve potential risks, severely impacting the company's product reputation and market competitiveness.

[0030] The present invention provides a working terminal and its working condition determination method, which is executed on an electronic device. The electronic device can be a control module of engineering machinery, or a smart terminal device such as a laptop, mobile phone, or tablet connected to the control module, or a server remotely connected to the control module.

[0031] It should be noted that the method provided in this embodiment of the invention is applicable to construction machinery that uses hydraulic cylinders to drive the movement of the robotic arm to adjust the posture of the working end. For ease of understanding, the technical solution of this invention will be specifically described below using an excavator as an example. In the case of an excavator, the robotic arm consists of a boom and a stick, and the working end is a bucket.

[0032] See Figure 1 , Figure 1This is a flowchart of a working terminal and its working condition determination method provided by an embodiment of the present invention, such as... Figure 1 As shown, the process of a working terminal and its working condition determination method provided in this embodiment of the invention mainly includes the following steps: S100: Acquire the stroke data of the hydraulic cylinder and the working status image of the working end.

[0033] Specifically, the excavator controls the height and angle of the bucket by changing the extension length of the hydraulic cylinder; that is, the stroke data of the hydraulic cylinder corresponds to the position of the bucket in space.

[0034] Furthermore, the excavator's operating conditions and the bucket's posture can also be obtained through images of the bucket's working status.

[0035] In some possible embodiments, the stroke data of the hydraulic cylinder can be detected by the hydraulic cylinder sensor, while the working status image of the bucket can be acquired by a camera mounted on the front end of the excavator boom.

[0036] S110. Based on the preset correspondence, determine the standard posture data corresponding to the stroke data operation end. The preset correspondence is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original operation end of the construction machinery.

[0037] Specifically, after pre-storing the correspondence between the cylinder stroke data and the standard bucket posture data—that is, the bucket posture data of an excavator operating under normal conditions with its original bucket—when the cylinder sensor displays different stroke values, the standard angle and shape that the original bucket should have can be determined based on the correspondence. In other words, the standard bucket posture data corresponding to the acquired cylinder stroke data can be determined. For example, for a certain excavator, when the cylinder extends a preset length, the posture of the original bucket is as follows: Figure 2 As shown.

[0038] S120. Based on the working status image, determine the actual posture data of the working end.

[0039] Specifically, after obtaining an image of the bucket's working state, the actual posture data of the bucket can be determined by extracting features such as the bucket's edge contour and tilt angle from the image.

[0040] In some possible embodiments, taking excavators as an example, considering that the geometric differences between different models of buckets are subtle and difficult to distinguish accurately through two-dimensional images, a 3D vision sensor is selected to collect images of the bucket's working status.

[0041] Specifically, the method for determining the actual attitude data of the bucket based on the working state image includes the following steps: 1. Standardize the point cloud data collected by the 3D vision sensor: Normalize the collected point cloud data and remove redundant points from the point cloud data based on the target box preset based on the bucket shape to reduce the amount of computation. 2. A representative portion of the point cloud data processed in step 1 is selected from the original point cloud, thus reducing the amount of data while preserving the overall structural features. Then, using the sampled points as centers, local regions are divided using algorithms such as spherical neighborhood or K-nearest neighbors. Points within each local region are combined into a fixed-size point set. Finally, local features are extracted again for each local point set, and these extracted local features are aggregated into a single feature vector, thereby capturing the overall shape of the bucket.

[0042] 3. Determine the actual attitude data of the bucket by capturing the overall shape of the bucket.

[0043] S130. Based on the relationship between standard attitude data and actual attitude data, determine whether the operating terminal has been replaced.

[0044] Specifically, by comparing the standard attitude data determined based on travel data with the actual attitude data determined based on the working state image of the bucket, it can be determined whether the excavator's bucket is the original bucket, that is, whether the bucket has been replaced without authorization.

[0045] In some possible embodiments, a preset threshold range is set, and then it is determined whether the difference between the standard posture data and the actual posture data exceeds the preset threshold range to determine whether the bucket has been replaced. That is, if the difference between the standard posture data and the actual posture data exceeds the preset threshold range, it is determined that the bucket has been replaced; if the difference between the standard posture data and the actual posture data does not exceed the preset threshold range, it is determined that the bucket has not been replaced, i.e., it is the original bucket.

[0046] Furthermore, after determining that the bucket has been replaced, abnormal time points can be marked, on-site camera footage can be saved as evidence, and the excavator's location can be recorded. Then, a violation record can be generated and uploaded to the excavator scene management platform, which facilitates violation reminders to users and the determination of responsibility when the excavator malfunctions.

[0047] In this embodiment, the original working end's posture data of the hydraulic cylinder at each stroke is pre-stored as the standard posture data of the working end. Then, during machinery operation, the cylinder's stroke data and the working end's working status image are simultaneously acquired. The actual posture data of the working end is determined from the working status image and compared with the standard posture data corresponding to the stroke data to determine if the working end has been replaced. This achieves accurate identification of the working end by determining its precise position using stroke data and comparing it with the actual posture data acquired through images. Furthermore, compared to pre-storing images of the working end's shape at each stroke—that is, establishing a correlation between stroke data and images corresponding to the current position—to determine if the working end has been replaced, this method overcomes the limitation of the number of image samples on the accuracy of the identification.

[0048] In a preferred embodiment, the method for determining the preset correspondence includes: Corresponding to the travel data, the position and orientation of each branch arm component that makes up the robotic arm are obtained as the attitude vector of each branch arm component; Transform all attitude vectors to the same coordinate system; Based on the attitude vector transformed to the same coordinate system, the attitude data of the original working end is determined; The attitude data from the original operating terminal is used as the standard attitude data and matched with the travel data to obtain a preset correspondence.

[0049] In this embodiment, the robotic arm of the construction machinery, such as the working device of an excavator, is regarded as a linkage mechanism in three-dimensional space. Then, the position and direction of each link are described by vectors, that is, the attitude vector of each branch arm is obtained. Then, the attitude vectors of each branch arm are transformed to the same three-dimensional coordinate system. Finally, the three-dimensional coordinates of the end of the robotic arm are solved based on the attitude vectors of the branch arm in the same three-dimensional coordinate system, that is, the accurate position coordinates of the original working end, that is, the standard attitude data.

[0050] In a preferred embodiment, the working terminal and its operating condition determination method further include: Stress data is acquired by stress sensors mounted on the robotic arm. Based on changes in stress data, determine whether the working end has been replaced or is operating under preset normal conditions.

[0051] It is understandable that the boom of an excavator experiences stress changes due to the forces applied during operation. Sudden stress changes can occur when the bucket is changed or abnormal operations are performed. Therefore, by placing stress gauges on the boom, stick, and other components to collect stress data in real time, changes in this data can be used to determine whether the workpiece has been replaced or is operating under preset normal conditions.

[0052] In some possible embodiments, the process of acquiring stress data from stress gauge detection includes: attaching strain gauges to key stress-bearing parts of the excavator boom, such as hinge points and stress concentration areas; converting the resistance changes of the strain gauges into voltage signals through a signal conditioning circuit; filtering and amplifying the signals; and finally converting the signals into digital signals through an ADC module.

[0053] Furthermore, by adjusting the sampling frequency of the stress gauge, stress changes at different precision levels of the robotic arm can be captured.

[0054] In a preferred embodiment, the working terminal and its operating condition determination method further include: Based on the working status image, determine the motion trajectory of the working end; Based on the motion trajectory, determine whether the working end is operating under the preset normal working conditions.

[0055] It should be noted that the preset normal operating conditions refer to operating conditions without violations. For example, for excavators, unilateral digging and overload digging are both abnormal operating conditions, which are operating conditions involving violations.

[0056] Specifically, excavator operating conditions are spatially and temporally correlated; that is, violations such as unilateral excavation or overload excavation often manifest as a series of continuous, abnormal action sequences. For example, during unilateral excavation, the bucket's trajectory in consecutive frames will deviate to one side. Based on this, by determining the bucket's trajectory through images of its operating status, it is possible to determine whether the bucket is operating under abnormal conditions.

[0057] In a preferred embodiment, the motion trajectory is input into a trained deep learning model to distinguish between normal mining and illegal operations. Specifically, based on the motion trajectory, it is determined whether the work unit is operating under preset normal conditions, including: Input the motion trajectory into the trained working condition recognition model; Based on the output of the working condition identification model, determine whether the working end is working under the preset normal working condition; The working condition recognition model is a deep learning model trained on sample data. The sample data includes target point cloud data of different types of target working terminals, point cloud data sequences of target working terminals operating under preset normal and abnormal working conditions, target point cloud data after first preprocessing, and point cloud data sequences after second preprocessing. The first preprocessing includes random rotation, scaling, and adding noise, and the second preprocessing includes random cropping or repeating part of the point cloud data sequence.

[0058] In this embodiment, taking an excavator as an example, the sample data for training the working condition recognition model is obtained in the following way: 1. Data labeling Bucket model labeling: Label the point cloud data of different bucket models with category labels and label the 3D bounding boxes.

[0059] Working condition annotation: By observing the excavator's working images, mark the start and end frames of working conditions such as normal excavation, single-sided excavation, and overload excavation, and annotate the corresponding point cloud sequences.

[0060] 2. Data Augmentation Point cloud enhancement: Randomly rotate (e.g., flip ±15°), scale (e.g., 0.8, 1, 1.2 times), and add Gaussian noise to point cloud data to simulate different working angles and environmental interference.

[0061] Spatiotemporal augmentation: Randomly cropping continuous frame sequences from the working condition data, or repeating some frames to create abnormal timing, thereby enhancing the model's adaptability to temporal changes.

[0062] Specifically, training the working condition recognition model using sample data obtained through the above processing can improve the stability and accuracy of the training, thereby helping to more accurately judge the working condition of the excavator.

[0063] Furthermore, in a preferred embodiment, the working terminal and its operating condition determination method further include: Based on the working status image, determine the environmental information of the working end; Based on stress data, the deformation information of the robotic arm is determined; Based on environmental and deformation information, determine whether the operating terminal is working under preset normal conditions.

[0064] In this embodiment, by identifying the surrounding environment of the working end in the working state image, environmental information is obtained. Then, combined with the deformation information of the robotic arm determined by stress data, it is possible to accurately determine whether the working end is working under abnormal conditions.

[0065] In a preferred embodiment, the working terminal and its operating condition determination method further include: When it is determined that the working end has been replaced or is not working under the preset normal operating conditions, an anomaly record is made, and when the stress data exceeds the preset upper limit, the construction machinery is triggered to reduce the output power.

[0066] Specifically, by recording anomalies when the operating end is replaced or not operating under preset normal conditions, it is easier to alert users to usage risks. It also facilitates the identification of the cause of malfunctions in construction machinery, thereby avoiding misallocation of maintenance resources and inefficient after-sales service due to ambiguous diagnosis, as well as secondary malfunctions caused by failure to completely resolve potential risks, which seriously affect the company's product reputation and market competitiveness. By triggering the construction machinery to reduce output power when stress data exceeds a preset upper limit, the probability of machinery failure is reduced.

[0067] See Figure 3 This is a technical roadmap using an excavator as an example, illustrating the operation terminal and its working condition determination method provided in the above embodiments of this application. Figure 3 As shown, the working terminal and its working condition discrimination method provided in this application summarize the standard position information of the bucket obtained by the stroke data of the cylinder stroke sensor, the image information of the bucket obtained by the 3D vision sensor, the working condition information around the bucket, and the boom deformation information obtained by the stress data of the stress plate, and then determine whether there is unauthorized replacement of the bucket, operation under harsh working conditions, or rough operation. Then, it is connected to the engine control module (ECM) and the electronic control system (EPS) to reduce the output power and / or record data when the control requirements are met, and retain the image information as evidence, thus forming a closed-loop control of the construction machinery.

[0068] Considering that the working end and its working condition discrimination method provided in the above embodiments of this application require multi-dimensional data fusion of stroke data, working status image and stress data, and the sampling frequency of strain gauge, hydraulic cylinder stroke sensor and 3D vision data may be different, it is necessary to ensure multi-dimensional data alignment in order to improve discrimination accuracy.

[0069] Based on this, in a preferred embodiment, the working terminal and its operating condition determination method further include: Align travel data, working status images, and stress data using timestamps.

[0070] Specifically, by keeping the "clocks" of strain gauges, hydraulic cylinder sensors, and 3D vision sensors consistent, time misalignment caused by their respective timing deviations can be avoided.

[0071] In some possible embodiments, keeping the "clocks" of the strain gauges, hydraulic cylinder sensors, and 3D vision sensors synchronized can be achieved using any of the following methods: 1. Connect the strain gauges, hydraulic cylinder sensors, and 3D vision sensors to the same hardware clock source, such as a GPS timing module or a PTP precision clock protocol device, so that when collecting data, they are all timestamped based on this unified clock.

[0072] 2. By setting any one of the strain gauges, hydraulic cylinder sensors, and 3D vision sensors as the "master clock" through software, clock calibration signals are periodically sent to the other two devices to reduce time errors between devices.

[0073] Furthermore, after aligning the "clocks," data points from each device can be aligned using software interpolation. Specifically, for data of different frequencies, algorithms fill in the "gaps" in low-frequency data and match the sampling points of high-frequency data.

[0074] Taking "cylinder stroke data (low frequency) matching strain data (high frequency)" as an example, the following method is used to achieve interpolation alignment: 1. Data processing: Extract the "timestamp + data value" lists for strain data (e.g., 1000Hz, 1000 points per second) and cylinder stroke data (e.g., 100Hz, 100 points per second).

[0075] 2. Find the corresponding interval: For each high-frequency strain data point, find the timestamp and value of the two cylinder stroke data points before and after it. For example, if there is a strain data point at 1.05 seconds, the corresponding cylinder stroke data is 5mm at 1.0 seconds and 6mm at 1.1 seconds.

[0076] 3. Linear interpolation calculation: The "estimated value" of the cylinder stroke data at 1.05 seconds is calculated using a linear formula, that is, (6-5)×(1.05-1.0) / (1.1-1.0) +5 =5.5mm, so that each strain data point can correspond to a cylinder stroke data point, thus achieving alignment.

[0077] As can be seen from the above embodiments, the method for judging the working end and its working condition provided in this application has high data processing efficiency, strong real-time performance, and is easy to implement and expand.

[0078] The following describes a working terminal and its working condition discrimination system provided by an embodiment of the present invention. The working terminal and its working condition discrimination system described below can be considered as a module architecture for implementing the working terminal and its working condition discrimination method provided by the embodiment of the present invention; the following description can be referred to in conjunction with the above.

[0079] Optional, see Figure 4 , Figure 4 This is a structural block diagram of a working end and its working condition judgment system provided in an embodiment of the present invention. It is applied to engineering machinery that uses hydraulic cylinders to drive the movement of a robotic arm to adjust the posture of the working end, such as... Figure 4 As shown, the system may include: The data acquisition unit 10 is used to acquire the stroke data of the hydraulic cylinder and the working status image of the working end; The first data processing unit 20 is used to determine the standard posture data corresponding to the stroke data working end based on a preset correspondence relationship. The preset correspondence relationship is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original working end of the construction machinery. The second data processing unit 30 is used to determine the actual posture data of the working end based on the working status image; The third data processing unit 40 is used to determine whether the working end has been replaced based on the relationship between standard attitude data and actual attitude data.

[0080] Optionally, the data acquisition unit 10 is also used to acquire stress data, which is obtained by stress plates arranged on the robotic arm. The third data processing unit 40 is also used to determine whether the working end has been replaced or is operating under preset normal conditions based on changes in stress data.

[0081] Optionally, the second data processing unit 30 is also used for: Based on the working status image, determine the motion trajectory of the working end; Based on the motion trajectory, determine whether the working end is operating under the preset normal working conditions.

[0082] Optionally, the second data processing unit 30 is also used to determine the environmental information of the working end based on the working status image; The third data processing unit 40 is also used to determine the deformation information of the robotic arm based on stress data; and to determine whether the working end is working under preset normal conditions based on environmental information and deformation information.

[0083] See Figure 5 , Figure 5 The diagram shown is a structural block diagram of another type of working terminal and its working condition discrimination system provided by an embodiment of the present invention. Figure 4 Based on the illustrated embodiment, the system further includes: The execution unit 50 is used to record abnormalities when it is determined that the working end has been replaced or is not working under the preset normal working conditions, and to trigger the construction machinery to reduce the output power when the stress data exceeds the preset upper limit.

[0084] Optionally, methods for determining the preset correspondence include: Corresponding to the travel data, the position and orientation of each branch arm component that makes up the robotic arm are obtained as the attitude vector of each branch arm component; Transform all attitude vectors to the same coordinate system; Based on the attitude vector transformed to the same coordinate system, the attitude data of the original working end is determined; The attitude data from the original operating terminal is used as the standard attitude data and matched with the travel data to obtain a preset correspondence.

[0085] Optionally, the second data processing unit 30 is specifically used to input the motion trajectory into the trained working condition recognition model; and based on the output of the working condition recognition model, to determine whether the working end is working under the preset normal working condition. The working condition recognition model is a deep learning model trained on sample data. The sample data includes target point cloud data of different types of target working terminals, point cloud data sequences of target working terminals operating under preset normal and abnormal working conditions, target point cloud data after first preprocessing, and point cloud data sequences after second preprocessing. The first preprocessing includes random rotation, scaling, and adding noise, and the second preprocessing includes random cropping or repeating part of the point cloud data sequence.

[0086] Optional, see Figure 5 The system also includes: The data alignment unit 60 is used to align the travel data, working status image and stress data using timestamps.

[0087] Optionally, embodiments of the present invention also provide an engineering machinery, which includes an engineering machinery body and a working end and its working condition discrimination system as provided in any of the above embodiments.

[0088] Below, for reference Figure 6 The electronic device provided in the embodiments of this application can be described as follows: at least one processor 100, at least one communication interface 200, at least one memory 300 and at least one communication bus 400; In this embodiment of the invention, the number of processor 100, communication interface 200, memory 300, and communication bus 400 is at least one, and the processor 100, communication interface 200, and memory 300 communicate with each other through communication bus 400; obviously, Figure 6 The communication connections shown for the processor 100, communication interface 200, memory 300, and communication bus 400 are optional. Optionally, the communication interface 200 can be an interface of a communication module, such as the interface of a GSM module; the processor 100 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.

[0089] The memory 300 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0090] Specifically, the processor 100 is used to execute the application program in the memory to implement the steps of the above-mentioned working end and its working condition determination method.

[0091] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0092] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0093] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0094] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0095] It should be understood that the qualifiers “first,” “second,” “third,” “fourth,” “fifth,” and “sixth” used in the description of the embodiments of this application are only used to more clearly illustrate the technical solutions and are not intended to limit the scope of protection of this application.

[0096] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. A working end and its working condition determination method, applied to engineering machinery that uses hydraulic cylinders to drive the movement of a robotic arm to adjust the posture of the working end, characterized in that, include: Acquire the stroke data of the hydraulic cylinder and the working status image of the working end; Based on a preset correspondence, standard posture data corresponding to the stroke data of the working end is determined. The preset correspondence is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original working end of the construction machinery. Based on the working status image, determine the actual posture data of the working end; Based on the relationship between the standard attitude data and the actual attitude data, it is determined whether the operating terminal has been replaced.

2. The method according to claim 1, characterized in that, Also includes: Acquire stress data, which is obtained by stress gauges arranged on the robotic arm; Based on the changes in the stress data, it is determined whether the working end has been replaced or is operating under the preset normal working conditions.

3. The method according to claim 2, characterized in that, Also includes: Based on the working status image, the motion trajectory of the working end is determined; Based on the motion trajectory, it is determined whether the working end is operating under preset normal conditions.

4. The method according to claim 3, characterized in that, Also includes: Based on the working status image, determine the environmental information of the working terminal; Based on the stress data, the deformation information of the robotic arm is determined; Based on the environmental information and the deformation information, it is determined whether the working end is operating under the preset normal working conditions.

5. The method according to claim 4, characterized in that, Also includes: When it is determined that the working end has been replaced or is not working under the preset normal working conditions, an anomaly record is made, and when the stress data exceeds the preset upper limit, the engineering machinery is triggered to reduce the output power.

6. The method according to claim 1, characterized in that, The method for determining the preset correspondence includes: Corresponding to the travel data, the position and orientation of each branch arm component that makes up the robotic arm are obtained as the attitude vector of each branch arm component; Transform all the aforementioned attitude vectors to the same coordinate system; Based on the attitude vector transformed to the same coordinate system, the attitude data of the original working end is determined; The attitude data of the original working end is used as the standard attitude data and matched with the travel data to obtain the preset correspondence.

7. The method according to claim 3, characterized in that, The step of determining whether the working end is operating under preset normal conditions based on the motion trajectory includes: Input the motion trajectory into the trained working condition recognition model; Based on the output of the working condition identification model, it is determined whether the working terminal is working under the preset normal working condition; The working condition identification model is a deep learning model trained on sample data. The sample data includes target point cloud data of different models of target working terminals, point cloud data sequences of the target working terminals operating under preset normal and abnormal working conditions, target point cloud data after first preprocessing, and point cloud data sequences after second preprocessing. The first preprocessing includes random rotation, scaling, and adding noise. The second preprocessing includes random cropping or repeating parts of the point cloud data sequence.

8. The method according to claim 3, characterized in that, Also includes: The travel data, the working status image, and the stress data are aligned using timestamps.

9. A working end and its working condition judgment system, applied to engineering machinery that uses hydraulic cylinders to drive the movement of a robotic arm to adjust the posture of the working end, characterized in that, include: The data acquisition unit is used to acquire the stroke data of the hydraulic cylinder and the working status image of the working end; The first data processing unit is used to determine the standard posture data of the working end corresponding to the stroke data based on a preset correspondence relationship, wherein the preset correspondence relationship is the correspondence between the stroke data of the hydraulic cylinder and the posture data of the original working end of the construction machinery. The second data processing unit is used to determine the actual posture data of the working end based on the working status image; The third data processing unit is used to determine whether the working end has been replaced based on the relationship between the standard attitude data and the actual attitude data.

10. An engineering machinery, characterized in that, It includes the main body of the engineering machinery and the working end and its working condition discrimination system as described in claim 9.

Images