Method and system for automatically calculating number of loading buckets and loading quantity

Abstract

The invention provides a method and system for automatically calculating the number of loading buckets and the loading quantity, and relates to the technical field of equipment monitoring, and the method comprises the steps of 1, obtaining bucket excavation pilot pressure and bucket rod excavation pilot pressure, 2, obtaining movable arm rising pilot pressure, and triggering a camera to recognize the state of materials in a bucket when the pilot pressure accords with a threshold value; 3, the state recognition result of the materials in the bucket is obtained, if the recognition result is that the materials exist, the unloading pilot pressure of the bucket continues to be obtained, and when the threshold value is met, a camera is triggered to recognize the ore card and the state of the materials in the ore card; step 4, acquiring a mine card identification result, and when the identification result is that the mine card exists, counting the number of effective loading buckets; and step 5, obtaining a state identification result of the materials in the mine card, if the identification result is that no load is changed into full load, adding one to the effective loading quantity, and reading the remaining oil quantity after loading. According to the invention, automatic statistics of the loading bucket number and the loading quantity of the large mining excavator can be realized, and the accuracy is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of equipment monitoring, and in particular relates to a method and system for automatically calculating the number of loading buckets and the loading quantity. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] During mine construction, trucks are often loaded with excavators. However, for large mining excavators, manual measurement is usually used to calculate the effective loading quantity. This method not only consumes a lot of manpower costs, but is also prone to statistical errors, resulting in low efficiency.

[0004] Existing excavators are generally equipped with pressure sensors, oil level sensors, machine controllers (MCU), on-board communication terminals (TBOX), etc. for monitoring. The effective number of loading buckets can be calculated through the pilot pressure value, main pump pressure value and boom IMU angle. However, in most practical applications, the boom IMU angle is used to judge whether it is in the loading state. When the excavator and the mining truck are at the same height, misjudgment may occur. When judging whether it is fully loaded by the main pump pressure value, the full load state judgment deviation may occur due to different materials, different gears, and different operations, and the effective number of loading buckets cannot be accurately judged.

[0005] However, the method of performing AI image recognition only through cameras will also have blind spots in operation judgment due to the limited field of view of the camera. When the excavator operation cannot be recognized within the field of view of the camera, there will be blind spots in operation judgment. Summary of the invention

[0006] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for automatically calculating the number of loading buckets and the number of loaded vehicles. Through the calculation of the pilot pressure value combined with image recognition and a specific judgment process, it makes up for the operation judgment of the blind spot and filters out the wrong operation, so as to realize the automatic statistics of the number of loading buckets and the number of loaded vehicles of large-scale mining excavators, make the statistical results more accurate and effective, and can upload the information to the cloud system in real time.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a method for automatically calculating the number of loading buckets and the loading quantity.

[0008] A method for automatically calculating the number of loading buckets and the loading quantity, comprising: Step 1, obtaining the bucket excavation pilot pressure and the arm excavation pilot pressure, and when the two meet the first set threshold and the second set threshold respectively, continue to obtain the boom raising pilot pressure; Step 2: Obtain the boom raising pilot pressure, and when it meets the first set threshold, trigger the camera to identify the state of the material in the bucket; Step 3, obtaining the result of identifying the state of the material in the bucket. If the result of the identification is that there is no material, it is determined to be invalid loading. If the result of the identification is that there is material, the bucket unloading pilot pressure is continued to be obtained. When it meets the second threshold, the camera is triggered to respectively identify the mining truck and the state of the material in the mining truck. Step 4, obtaining the mining card identification result, when the identification result is that there is a mining card, the effective loading bucket number is counted; when the identification result is that there is no mining card, it is determined to be an invalid loading bucket number and is not counted; Step 5: Obtain the material status recognition result in the mining truck. If the recognition result is from full load to empty load, read the remaining oil volume before loading; if the recognition result is from empty load to full load, add one to the effective loading quantity and read the remaining oil volume after loading.

[0009] A second aspect of the present invention provides a system for automatically calculating the number of loading buckets and the number of loading buckets.

[0010] A system for automatically calculating the number of buckets loaded and the number of loaded items, comprising: The excavator controller is configured as a running carrier, interacts with the pressure recognition unit for pressure value and triggers the image recognition unit to perform image recognition; a pressure identification unit configured to identify a pilot pressure through a pressure sensor; The bucket excavation pilot pressure and the arm excavation pilot pressure are obtained. When the two pressures meet the first set threshold and the second set threshold, respectively, the boom raising pilot pressure is continued to be obtained. an image recognition unit, configured to perform image recognition through an AI camera; The boom raising pilot pressure is obtained, and when it meets the first set threshold, the camera is triggered to identify the state of the material in the bucket; Obtain the result of identifying the material status in the bucket. If the result is that there is no material, it is determined to be invalid loading. If the result is that there is material, continue to obtain the bucket unloading pilot pressure. When it meets the second threshold, trigger the camera to identify the mining truck and the material status in the mining truck respectively. Obtain the mining card identification result. When the identification result is that there is a mining card, count the valid loading buckets; when the identification result is that there is no mining card, determine it as an invalid loading bucket and do not count; An oil level identification unit is configured to read the remaining oil level before and after loading through an oil level sensor; Obtain the material status recognition result in the mining truck. If the recognition result is from full load to empty load, read the remaining fuel before loading; if the recognition result is from empty load to full load, add one to the effective loading quantity and read the remaining fuel after loading. The communication unit is configured to upload data via the vehicle-mounted communication terminal.

[0011] One or more of the above technical solutions have the following beneficial effects: By combining the calculation of the pilot pressure value with image recognition and a specific judgment process, the blind spot operation judgment is compensated, the wrong operation is filtered out, and the automatic statistics of the number of loading buckets and the number of loadings of large mining excavators are realized, making the statistical results more accurate and effective, and the information can be uploaded to the cloud system in real time.

[0012] When the excavator operation cannot be identified within the camera's field of view, the pilot pressure value is used to judge the blind spot operation, improving the scope and effectiveness of the judgment. By combining the effective value of pressure with the effective value duration to filter out the handle's misoperation and trigger image recognition, the accuracy of the judgment can be improved, and the effective loading bucket number can be accurately judged.

[0013] By switching the material status in the same mining truck compartment, the loading quantity can be measured and the loading fuel consumption can be calculated, so that the loading quantity can be calculated quickly and effectively.

[0014] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0016] Figure 1 The present invention is a flowchart of a method for automatically calculating the number of loading buckets and the number of loading buckets in the first embodiment of the present invention. DETAILED DESCRIPTION

[0017] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0018] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0019] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0020] The overall idea proposed by the present invention is: The bucket excavation pilot pressure and the arm excavation pilot pressure are obtained. When both meet the set thresholds, the boom rise pilot pressure is obtained. When they also meet the set thresholds, the camera is triggered to identify the state of the material in the bucket. According to the state of the material in the bucket, it is determined whether the bucket has been loaded, and the number of buckets loaded is calculated. At the same time, the loading state is determined, that is, the identification results of the mining truck and the material state in the mining truck are obtained, and the loading quantity is calculated. Finally, the loading fuel consumption is calculated according to the identification results of the material state in the mining truck.

[0021] Embodiment 1 This embodiment discloses a method for automatically calculating the number of loading buckets and the loading quantity.

[0022] In order to explain this embodiment more clearly, Figure 1 As shown, a method for automatically calculating the number of loading buckets and the number of loading buckets can be specifically described as follows: Step 1: Obtain the bucket excavation pilot pressure and the arm excavation pilot pressure. When the two pressures meet the first set threshold and the second set threshold, respectively, continue to obtain the boom raising pilot pressure.

[0023] In this embodiment, the bucket excavation pilot pressure and the arm excavation pilot pressure are obtained through the pressure sensor. When the bucket excavation pilot pressure meets the first set threshold and the arm excavation pilot pressure meets the third set threshold, the boom raising pilot pressure continues to be obtained.

[0024] The first set threshold is a value greater than 10 and maintained for more than 0.5 seconds.

[0025] The second set threshold is a value greater than 5 and maintained for more than 0.2 seconds.

[0026] Because the handle may be misoperated when driving an excavator, the threshold and holding time are set to filter out the operator's misoperation that may affect program judgment.

[0027] The bucket excavation pilot pressure, arm excavation pilot pressure and boom rise pilot pressure represent the movements of the excavator boom, arm and bucket respectively.

[0028] As a fixed operation process: when the excavator is loading, it will first perform digging operations (bucket digging, dipper arm digging), and after the digging is completed, the boom will be raised; By obtaining the bucket excavation pilot pressure, the arm excavation pilot pressure and the boom rise pilot pressure, the program can better judge the current operating status of the excavator, thereby activating image judgment; the boom rise pilot pressure represents the end of the excavation operation, and the state of the material in the bucket will not change at this time. Therefore, performing image judgment after the excavation operation is completed can improve the efficiency and accuracy of identifying the number of loading buckets and the loading quantity.

[0029] Step 2: Obtain the boom raising pilot pressure. When it meets the first set threshold, trigger the camera to identify the state of the material in the bucket.

[0030] In this embodiment, the boom raising pilot pressure is obtained through a pressure sensor, and when it meets a first set threshold, the camera is triggered to identify the state of the material in the bucket.

[0031] Wherein, similar to step 1, the first set threshold is a value greater than 10 and maintained for more than 0.5 seconds.

[0032] Step 3, obtaining the result of identifying the state of the material in the bucket. If the result of the identification is that there is no material, it is determined to be invalid loading. If the result of the identification is that there is material, the unloading pilot pressure of the bucket is continuously obtained. When it meets the second set threshold, the camera is triggered to respectively identify the mining truck and the state of the material in the mining truck. In this embodiment, the state of the material in the bucket is identified by an AI camera, and whether to proceed to the next step of obtaining the pilot pressure value is determined based on the identification result of whether there is material.

[0033] If the identification result is that there is no material, it is judged as invalid loading; If the recognition result is that there is material, it is effective loading, and the bucket unloading pilot pressure continues to be obtained. When it meets the second set threshold, the camera is triggered to identify the mining truck and the material status in the mining truck respectively; The third threshold is set as follows: the value is greater than 5 and is maintained for more than 0.5 seconds.

[0034] Effective loading means that the excavator unloads the material in the bucket into the truck compartment. If there is no material in the bucket to load, it is empty excavation, and no material is unloaded into the truck compartment, which will not increase the loading rate of the truck. Therefore, by identifying the state of the material in the bucket, abnormal loading operations can be effectively filtered out, thereby avoiding the waste of computing resources and improving the efficiency of identification.

[0035] When it is recognized that there is material in the bucket and its bucket unloading pilot pressure meets the second set threshold, the status of the mine truck and the material in the mine truck is identified to determine whether loading is completed. When the status of the material in the mine truck changes from empty to full, it means that loading is completed.

[0036] Step 4: Obtain the mining truck identification result. If the identification result is that there is a mining truck, count the number of valid loading buckets; When the identification result is that there is no mining truck, it is judged as invalid loading bucket number and will not be counted; In this embodiment, the camera is used to perform image recognition to obtain the mining truck recognition result. When the recognition result is that there is a mining truck, the number of valid loading buckets is counted and increased by one. When the recognition result is that there is no mining truck, it is judged as an invalid loading bucket number and is not counted.

[0037] Step 5: Obtain the material status recognition result in the mining truck. If the recognition result is that the truck changes from full load to empty load, trigger the oil level sensor to read the remaining oil volume before loading. If the recognition result is that the vehicle changes from empty to fully loaded, the effective loading quantity is increased by one, and the remaining oil volume after loading is read through the oil level sensor.

[0038] In this embodiment, the remaining oil volume before loading and the remaining oil volume after loading are both read by the oil volume sensor.

[0039] The material status recognition results in the mining truck are obtained by calling the open source model YOLOv8 to perform frame image recognition.

[0040] The YOLOv8 model is a target detection model that can be used for tasks such as image classification, object detection, and instance segmentation.

[0041] In the training process of the YOLOv8 model, in order to maintain the authenticity of the data and improve the accuracy of the model, Mosaic data enhancement is no longer used in the last 10 epochs of the training cycle, so as to ensure that the model can better learn the essential characteristics of the data and avoid the introduction of noise or misleading information due to excessive enhancement. In the network structure design of YOLOv8, the changes to the backbone network are relatively small, and the main improvement is to replace the original C3 module with the C2f module. On the basis of inheriting the design concept of the C3 module, the C2f module integrates the design elements of ELAN (EnhancedLayered Aggregation Network) in YOLOv7. By introducing more jump connections, the C2f module not only enhances the gradient flow and improves the efficiency of information transmission, but also maintains the lightweight characteristics of the model. In addition, YOLOv8 also retains the SPPF (Spatial Pyramid Pooling Form) module, which shortens the processing time of the module and improves the overall computing efficiency without sacrificing performance. These structural optimizations enable YOLOv8 to capture richer feature information while maintaining computational efficiency, further improving the performance of the model.

[0042] In addition, in order to simplify the structure, the two convolutional connection layers before the upsampling operation of YOLOv8 are removed. In the design of the head network (Head), YOLOv8 draws on the decoupled head strategy of YOLOX and adopts two parallel branches to handle the extraction of category and position features respectively. This design takes into account the different focuses of classification and positioning tasks: the classification task focuses on comparing the similarity of features with known categories, while the positioning task focuses on the precise adjustment of the bounding box. By processing separately, the two branches can focus on their respective tasks, thereby improving the convergence speed and prediction accuracy of the model during training. YOLOv8 also adopts an anchor-free structure, which means that the model no longer relies on predefined anchor boxes, but directly predicts the center point of the target.

[0043] Therefore, using the YOLOv8 model for image detection can significantly improve the accuracy and efficiency of the model while maintaining good compatibility with different hardware platforms.

[0044] In this embodiment, the loading fuel consumption can also be calculated: the loading fuel consumption is obtained by subtracting the remaining fuel amount after loading from the remaining fuel amount before loading.

[0045] When the material identification in the mining truck changes from full load to empty load, it means that the loading operation of the second truck has started. The start of the loading operation is the remaining fuel before loading. It can effectively help mines calculate costs and profits. The judgment of whether the mining truck is full is based on standards, that is, the amount of earthwork required to fill a truck is fixed, and the fuel consumption of a single truck for loading is also one of the data for cost measurement.

[0046] The above steps are designed based on the actual process of the excavator in normal loading operations. Currently, the prototype has been tested and the accuracy rate has reached 100%.

[0047] Embodiment 2 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0048] Embodiment 3 The purpose of this embodiment is to provide a computer-readable storage medium.

[0049] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0050] Embodiment 4 The purpose of this embodiment is to provide a system for automatically calculating the number of loading buckets and the number of loading buckets.

[0051] A system for automatically calculating the number of buckets loaded and the number of loaded items, comprising: The excavator controller is configured as a running carrier, interacts with the pressure recognition unit for pressure value and triggers the image recognition unit to perform image recognition; a pressure identification unit configured to identify a pilot pressure through a pressure sensor; The bucket excavation pilot pressure and the arm excavation pilot pressure are obtained. When the two pressures meet the first set threshold and the second set threshold, respectively, the boom raising pilot pressure is continued to be obtained. an image recognition unit, configured to perform image recognition through an AI camera; The boom raising pilot pressure is obtained, and when it meets the first set threshold, the camera is triggered to identify the state of the material in the bucket; Obtain the result of identifying the material status in the bucket. If the result is that there is no material, it is determined to be invalid loading. If the result is that there is material, continue to obtain the bucket unloading pilot pressure. When it meets the second threshold, trigger the camera to identify the mining truck and the material status in the mining truck respectively. Obtain the mining card identification result. When the identification result is that there is a mining card, count the valid loading buckets; when the identification result is that there is no mining card, determine it as an invalid loading bucket and do not count; An oil level identification unit is configured to read the remaining oil level before and after loading through an oil level sensor; Obtain the material status recognition result in the mining truck. If the recognition result is from full load to empty load, read the remaining fuel before loading; if the recognition result is from empty load to full load, add one to the effective loading quantity and read the remaining fuel after loading.

[0052] The communication unit is configured to upload data via the vehicle-mounted communication terminal.

[0053] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0054] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0055] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for automatically calculating the number of loading buckets and the number of loading buckets, characterized in that: The steps include: step 1, obtaining the bucket excavation pilot pressure and the boom excavation pilot pressure, and when the two pressures meet the first set threshold and the second set threshold, respectively, continuing to obtain the boom lifting pilot pressure; Step 2: Obtain the boom raising pilot pressure, and when it meets the first set threshold, trigger the camera to identify the state of the material in the bucket; Step 3, obtaining the result of identifying the state of the material in the bucket. If the result of the identification is that there is no material, it is determined to be invalid loading. If the result of the identification is that there is material, the unloading pilot pressure of the bucket is continuously obtained. When it meets the third set threshold, the camera is triggered to respectively identify the mining truck and the state of the material in the mining truck. Step 4, obtaining the mining card identification result, when the identification result is that there is a mining card, the effective loading bucket number is counted; when the identification result is that there is no mining card, it is determined to be an invalid loading bucket number and is not counted; Step 5: Obtain the material status recognition result in the mining truck. If the recognition result is from full load to empty load, read the remaining oil volume before loading; if the recognition result is from empty load to full load, add one to the effective loading quantity and read the remaining oil volume after loading.

2. A method for automatically calculating the number of loading buckets and the number of loading buckets as claimed in claim 1, characterized in that: The first set threshold is a value greater than 10 and maintained for more than 0.5 seconds; the second set threshold is a value greater than 5 and maintained for more than 0.2 seconds; and the third set threshold is: a value greater than 5 and maintained for more than 0.5 seconds.

3. The method for automatically calculating the number of loading buckets and the number of loading buckets as claimed in claim 1, characterized in that: The material status recognition result in the mining truck is performed by calling the open source model YOLOv8 model for frame image recognition.

4. The method for automatically calculating the number of loading buckets and the number of loading buckets as claimed in claim 1, characterized in that: It also includes calculating the loading fuel consumption: the loading fuel consumption is obtained by subtracting the remaining fuel before loading from the remaining fuel after loading.

5. A system for automatically calculating the number of buckets loaded and the number of loads loaded, characterized in that: include: The excavator controller is configured as a running carrier, interacts with the pressure recognition unit for pressure value and triggers the image recognition unit to perform image recognition; The pressure identification unit is configured to obtain the bucket excavation pilot pressure and the arm excavation pilot pressure, and when the two pressures meet the first set threshold and the second set threshold, respectively, continue to obtain the boom raising pilot pressure; The image recognition unit is configured to obtain the boom raising pilot pressure, and when it meets the first set threshold, trigger the camera to recognize the state of the material in the bucket; Obtain the result of identifying the material status in the bucket. If the result is that there is no material, it is determined to be invalid loading. If the result is that there is material, continue to obtain the bucket unloading pilot pressure. When it meets the second threshold, trigger the camera to identify the mining truck and the material status in the mining truck respectively. Obtain the mining card identification result. When the identification result is that there is a mining card, count the valid loading buckets; when the identification result is that there is no mining card, determine it as an invalid loading bucket and do not count; The oil quantity identification unit is configured to obtain the material status identification result in the mining truck, and if the identification result is that the material status changes from full load to empty load, the remaining oil quantity before loading is read; if the identification result is that the material status changes from empty load to full load, the effective loading quantity is increased by one, and the remaining oil quantity after loading is read; The communication unit is configured to upload data via the vehicle-mounted communication terminal.

6. The automatic calculation system for the number of loading buckets and the number of loading vehicles as claimed in claim 5, characterized in that: The bucket digging pilot pressure, the arm digging pilot pressure and the boom raising pilot pressure are obtained through the pressure sensor.

7. The automatic calculation system for the number of loading buckets and the number of loading vehicles as claimed in claim 5, characterized in that: The remaining oil volume before and after loading is read through the oil level sensor.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 4 are performed.

Images