High-adaptability shovel loading robot intelligent sensing system based on on-demand triggering sensing

Abstract

The invention relates to the technical field of shovel loading robots, in particular to a high-adaptability shovel loading robot intelligent sensing system based on on-demand triggering sensing. Comprising a sensing module used for acquiring information of each sensing device of the shovel loading robot and the mine card, surrounding environment information and device state monitoring information; the edge processing module is used for preprocessing the information transmitted by the sensing module so as to improve the data quality, extracting key features of the data and preliminarily integrating the information of different sensing devices; the server module is used for deeply fusing multi-source data and realizing service logic execution, data storage management and system coordination communication; and the front-end module is used for providing an operation and interaction interface for a user and visually displaying the data processed by the system to the user.

Description

Technical Field

[0001] The present invention relates to the technical field of shovel loading robots, and in particular to a highly adaptable shovel loading robot intelligent sensing system based on on-demand trigger sensing. Background Art

[0002] In the shovel loading robot operation environment, shovel loading operation is an important basic link, and its efficiency and safety directly affect the overall progress and benefits of the project. Traditional shovel loading operations mainly rely on manual operation, which has problems such as high labor intensity, low efficiency, and many safety hazards. With the continuous development of technology, automated and intelligent shovel loading robots are gradually applied to actual production, greatly improving the working environment and production efficiency. However, the existing shovel loading robot operation environment perception system still has some shortcomings. On the one hand, the sensors of most systems are in a continuous full-power operation state, and they continue to collect data regardless of whether they are in the actual operation stage. This not only leads to excessive energy consumption of sensors and shortens the service life of equipment, but also generates a large amount of redundant data, increasing the burden of data processing and storage. For example, when the shovel loading robot is waiting for loading or transportation, sensors such as 3D laser radar continue to scan at a high frequency, consuming a lot of electricity, and the massive environmental data generated at the same time has no practical use at this time. On the other hand, the various modules of the system lack an intelligent on-demand working mechanism. In terms of data processing, edge processing modules often process all sensor data in real time without distinguishing the importance and urgency of the data, which slows down the processing speed when the data volume is large, affecting the real-time and responsiveness of the system. On the server side, all business service functions are in operation. Even when some functions are not needed in the current operation scenario, they will occupy system resources and reduce the overall operation efficiency of the system. In addition, there are unreasonable aspects in the data display and interactive operations of the front-end module. Some systems load all data at once for display, regardless of whether the user needs to view the information, which not only increases the pressure of data transmission, but also causes the front-end interface to load slowly, affecting the user experience. The interactive operation function is not reasonably configured according to user permissions and actual operation requirements, which may lead to the risk of misoperation and increase the complexity of operation. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing.

[0004] The present invention adopts the following technical solution: a highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing, comprising: The perception module is used to obtain information about the various sensor devices of the shovel loader and mining truck, the surrounding environment information, and the equipment status monitoring information; The edge processing module pre-processes the information transmitted by the perception module to improve data quality, extracts key features of the data, and preliminarily integrates the information of different sensor devices; The server module deeply integrates multi-source data to implement business logic execution, data storage management, and system coordination and communication; The front-end module provides users with an operation and interaction interface, and intuitively displays the data processed by the system to users.

[0005] In some embodiments, the perception module includes: The perception module I includes a laser radar A and a laser radar B respectively installed on the left and right positions of the front body of the shovel loader robot, a laser radar C installed directly behind the body, a laser radar D installed on the top of the cab or on the top of the body through a pole bracket to achieve 360° rotation scanning and overlap with the scanning range of the laser radars A, B, and C by 10%-50%, and an IMU inertial measurement unit installed on the solid-state laser radar body for laser radar attitude correction; Perception module II, including anti-vibration and anti-shake cameras installed on the outside of the bottom plate around the shovel loader robot body, a body inertial navigation system that accurately locates through four markers around the body, a rotary encoder installed at the bottom of the upper vehicle near the top of the crawler track, a pushing mechanism encoder installed on the pushing mechanism of the shovel loader robot, and a solid-state laser radar depth camera A installed in the middle and upper part of the boom to identify the fullness rate of the material in the bucket; The perception module III includes a solid-state LiDAR depth camera B and a solid-state LiDAR depth camera C installed on the left and right front guardrails of the roof, respectively, and a mining truck inertial navigation system for accurately locating the position of the mining truck.

[0006] In some embodiments, the edge processing module includes: A data processing unit, used to receive data from the laser radar, anti-vibration and anti-shake camera, IMU inertial measurement unit, laser radar depth camera, encoder and inertial navigation system, and perform point cloud processing, motion compensation and attitude estimation on the data; The feature extraction and fusion unit is used to perform target detection, target extraction and point cloud segmentation on the data processed by the data processing unit, and realize multi-sensor data fusion to obtain information related to the working environment of the shovel loader robot and the status information of the equipment itself.

[0007] In some embodiments, the data processing unit is specifically used to receive point cloud data collected by a solid-state laser radar for preprocessing operations, receive image data collected by an anti-vibration and anti-shake camera for denoising, enhancement, and correction, receive attitude data output by an IMU inertial measurement unit for calibration and compensation, receive depth image data acquired by a solid-state laser radar depth camera for hole filling and smoothing, receive position and angle information fed back by an encoder for analysis and conversion, receive position information collected by an inertial navigation system for calibration and compensation, and perform time synchronization and spatial alignment on the above processed data to achieve motion compensation and attitude estimation; The feature extraction and fusion unit is specifically used to process the data of the laser radar and the anti-vibration and anti-shake camera based on the deep learning algorithm to extract the feature information of the working environment, identify the target object, and fuse the multi-source data to improve the accuracy of environmental model reconstruction and target recognition accuracy; use deep learning to extract the posture and other information of the shovel robot from related equipment and calculate the posture and full bucket rate; extract the feature information of the mining truck from the related equipment based on the deep learning algorithm and determine the position of the mining truck and the distribution of materials in the vehicle; deeply fuse the multi-sensor data to generate the working environment model, the posture of the shovel robot, the full bucket rate of the bucket, the posture of the mining truck and the distribution of materials in the vehicle.

[0008] In some embodiments, the server module includes: The data receiving and storage unit receives various data preliminarily processed by the edge processing module through the transmission protocol, stores the point cloud data, image data, and sensor measurement values ​​in the database, and establishes corresponding indexes; The data analysis and processing unit pre-processes the data transmitted by the edge processing module, analyzes the data according to the priority and arrival time of the task, and dynamically allocates computing resources; The business logic processing unit monitors the potential safety hazards in the working environment in real time based on the results of the data analysis and processing unit, arranges the operation and planning of the shovel loading robot in a reasonable manner according to the requirements of the working tasks and the real-time status information of the equipment, and provides functions such as model reconstruction, environmental perception, personnel and obstacle warning, shovel truck posture, bucket fullness rate, mining truck posture, truck material distribution monitoring, reporting, and permission. The system management unit is responsible for managing the operating parameters of the server module, setting and managing user access rights, assigning different operating permissions to users of different roles, and recording error information during system startup, data reception and processing, and storing it in the database.

[0009] In some embodiments, the front end module includes: The user interface display module visualizes various data of the server module, displays the statistics of the full bucket rate through charts, displays the material distribution in the mining truck in the form of a three-dimensional model, and displays the reconstruction results of the large-scale operation model, the shovel robot and the mining truck posture information in the form of a map; The interactive operation module provides human-computer interaction functions, allowing operators to perform operations within their authority and interact with the system by sending remote control commands; The alarm and prompt module uses the SRS streaming service to display the video images collected by the vibration-resistant camera in real time and set alarm thresholds to control the personnel and obstacle alarm functions; User management module, log in according to user role, and can only see and operate operations and data within the scope of authority.

[0010] In some embodiments, the edge processing module assigns a priority to each task based on a priority scheduling algorithm and drives the corresponding sensor device of the perception module to work. The steps of the priority scheduling algorithm are as follows: Define task nodes: The large-scale operation environment model reconstruction and recognition task is marked as T1; the shovel robot posture and full bucket rate detection task is marked as T2; the mining truck posture and material distribution detection in the bucket is marked as T3; Priority setting and task scheduling and execution: Job preparation phase: Set the priority of T1 to 8, the priority of T2 to 3, and the priority of T3 to 2; initialize the task queue and insert tasks in order of priority; the scheduler takes the T1 task from the head of the queue for execution and removes the task after completion; During the shoveling operation phase: adjust the priority of T1 to 6, the priority of T2 to 9, and the priority of T3 to 2 or increase it to 3; update the task queue and move T2 to the head of the queue; the scheduler takes out the T2, T1, and T3 tasks in turn for execution, and removes the corresponding tasks after completion; Unloading operation stage: adjust the priority of T1 to maintain at 6 or reduce it appropriately, the priority of T2 to 4, and the priority of T3 to 10; update the task queue and move T3 to the head of the queue; the scheduler takes out the T3, T1, and T2 tasks in turn for execution, and removes the corresponding tasks after completion; Task processing completed: After the task is executed, it is removed from the task queue. The scheduler continues to check the queue. If it is not empty, the task extraction and execution steps are repeated; Data transmission: The data collected and pre-processed by the sensing devices corresponding to each task are packaged and sent to the server module.

[0011] In some embodiments, the data receiving and storage unit performs a preliminary check on the received data, and stores structured business data in a Mysql database according to the type and characteristics of the data, stores data with high real-time requirements in a Redis database, and stores monitoring data during system operation in a Prometheus database; the data analysis and processing unit uniformly converts the raw data of different sensors into a standard data format, uses corresponding algorithms to process different types of data, and allocates computing nodes based on a dynamic resource allocation strategy of queuing theory and load balancing algorithm; the business logic processing unit uses a rule engine algorithm to formulate trigger rules for each business, and executes corresponding functions according to user requests on the front-end interface.

[0012] In some embodiments, when a user logs in, the interactive operation module obtains the user role and permission information from the system management unit of the server, verifies and processes it, stores it in the front-end Prometheus database, and gives feedback information during operation; the alarm and prompt module receives the alarm information from the server, pops up an alarm prompt box on the interface and plays an alarm sound; the user management module sends the information to the server for verification when the user logs in, and verifies the format of the user input information when registering and sends it to the server for processing.

[0013] Compared with the prior art, the present invention has the following beneficial effects: 1. Optimize energy efficiency of sensor equipment The on-demand triggering sensing mechanism is realized through the priority scheduling algorithm, and the working status of the sensor is dynamically adjusted according to different operation stages. Redundant sensors are automatically reduced or shut down during non-critical mission stages, reducing overall energy consumption by 40%-60% and effectively extending the service life of the equipment.

[0014] 2. Improved data processing efficiency The three-level edge-server collaborative processing architecture is adopted. The edge module completes data preprocessing and feature extraction, and the server implements load balancing based on dynamic resource allocation strategy. According to the test, the data processing response speed of complex scenarios is increased by 3.5 times, and the CPU resource utilization rate is increased by 60%.

[0015] 3. Enhanced multi-source perception accuracy The innovative configuration of a four-directional laser radar array and a dual-mode depth camera, combined with a multi-sensor time-space synchronization algorithm, enables the environment modeling accuracy to reach ±2cm, and the full bucket detection error is <3%. Through multi-source data fusion, the obstacle recognition accuracy is increased to 99.2%, an increase of 28% compared to the traditional single radar solution.

[0016] 4. Intelligent allocation of system resources Based on the improved priority scheduling algorithm, the posture detection task priority is raised to level 9 during the shovel loading phase to ensure core task computing resources. The system can automatically switch working modes according to the task queue, improving overall resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the system structure of the present invention; Figure 2 It is a schematic diagram of the installation of each sensing device in the sensing module; Figure 3 It is the flow chart of the priority scheduling algorithm of the edge processing module; In the figure, 100-perception module; 110-perception module II; 120-perception module III; 130-perception module III; 200-edge processing module; 210-data processing unit; 220-feature extraction and fusion unit; 300-server module; 310-data receiving and storage unit; 320-data analysis and processing unit; 330-business logic processing unit; 340-system management unit; 400-front-end module; 410-user interface display module; 420-interactive operation module; 430-alarm and prompt module; 440-user management module. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] like Figure 1 As shown, a highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing includes a perception module 100, an edge processing module 200, a service end module 300 and a front end module 400.

[0020] The perception module includes a perception module I110, a perception module II120, and a perception module III130.

[0021] The perception module I110 is a large-scene working environment model reconstruction and recognition module, including laser radar A, laser radar B, laser radar C, laser radar D, and IMU inertial measurement unit.

[0022] LiDAR A, LiDAR B, LiDAR C, and LiDAR D all use Helios32 solid-state LiDAR with a detection range of 0.2~100m; Laser radar A and laser radar B are installed on the left and right positions of the front of the shovel robot body respectively; The laser radar C is installed directly behind the shovel-loading robot body; Lidar D is installed on the top of the cab or on the top of the vehicle body through a pole bracket, which can realize 360° rotation scanning, and the scanning range of Lidar A, Lidar B and Lidar C overlaps by 10%~50%; The IMU inertial measurement unit is installed on the solid-state lidar body and is used for lidar attitude correction.

[0023] The perception module II120 is a shovel robot posture and full bucket rate detection module, including anti-vibration and anti-shake cameras (8), vehicle body inertial navigation system, rotary encoder, push mechanism encoder, solid-state laser radar depth camera A (dual angle); The anti-vibration and anti-shake camera is DS-2XC7045, with a horizontal field of view of 79.6°, a vertical field of view of 43.7°, and a detection range of 0.74~3.1m. It is installed on the outside of the bottom plate around the vehicle. The vehicle body inertial navigation system is the Huace CGI-430 inertial navigation system, which integrates and solves GNSS, INS, and DR information to provide continuous high-precision navigation information such as position, speed, and attitude. It can accurately locate the vehicle through four markers around the vehicle body, and its base station is installed at the location of the electric pole. The rotary encoder is built into the shovel-loading robot and is installed at the bottom of the upper vehicle near the top of the track to identify the rotation angle of the upper vehicle and chassis, thereby identifying the posture of the entire vehicle. The push mechanism encoder is built into the shovel loader robot and is installed on the shovel loader robot’s push mechanism. It can detect the displacement and angle information of the push mechanism in real time and accurately control the cutting depth and push distance of the bucket. Solid-state LiDAR depth camera A (dual angle), installed in the middle and upper part of the boom, with a detection range of 0.25m~9.2m, a LiDAR field of view of 70°×55°, and a visible light field of view of 70°×43°, used to identify the fullness of the bucket; The perception module III130 is a mining truck posture and in-vehicle material distribution detection module, including a solid-state laser radar depth camera B, a solid-state laser radar depth camera C, and a mining truck inertial navigation system; Solid-state LiDAR depth camera B and solid-state LiDAR depth camera C are a fusion of solid-state LiDAR and visible light camera, installed on the left and right front guardrails of the roof respectively, to identify the distance between the mining truck and the shovel loading robot, and the distribution of the material pile in the truck; The mining truck’s inertial navigation system is the Huace CGI-430 inertial navigation system with two antennas. The system fuses and solves GNSS, INS, and DR information, and can provide continuous and high-precision navigation information such as position, speed, and attitude, and accurately locate the mining truck.

[0024] The edge processing module 200 assigns a priority to each task based on the priority scheduling algorithm, and drives the corresponding sensor device of the perception module 100 to work. The data processing unit 210 performs pre-processing such as noise reduction, filtering, and calibration on the transmitted data. Then, the feature extraction and fusion unit 220 extracts feature information and performs deep fusion to transmit it to the database corresponding to the server module 300. The specific steps of the priority scheduling algorithm are as follows: Define task nodes: The large-scale operation environment model reconstruction and recognition task is marked as T1; the shovel robot posture and full bucket rate detection task is marked as T2; the mining truck posture and material distribution detection in the bucket is marked as T3; Priority setting and task scheduling and execution: 1. Operation preparation stage (1) Task and initial priority setting: Large-scale working environment model reconstruction and recognition task: priority value is set to 8; shovel robot posture and bucket fullness detection task: priority value is set to 3; mining truck posture and material pile distribution detection task: priority value is set to 2; (2) Task queue initialization: The system initializes an empty task queue and inserts the task into the queue; The large-scene operation environment model reconstruction and recognition task arrives first. Since the queue is empty, it is directly inserted into the head of the task queue; Next, the shovel robot posture and bucket fullness rate detection task arrives. Its priority 3 is lower than the priority 8 of the task at the head of the queue, so it is inserted into the tail of the queue. Then, the task of detecting the position of the mining truck and the distribution of the material pile in the truck bed arrives. Its priority 2 is lower than the priority of the existing tasks in the queue, so it is also inserted into the tail of the queue.

[0025] At this time, the order of the task queue is: large-scene working environment model reconstruction and recognition task, shovel robot posture and bucket fullness rate detection task, mining truck posture and material pile distribution detection task.

[0026] (3) Task execution: The scheduler takes out the large-scene operating environment model reconstruction and recognition task from the head of the task queue, then encapsulates the task instruction (including the task identifier and priority value), sends it to the device corresponding to the perception module I110 through wireless transmission, and drives the device to collect data, and then transmits the collected data to the data processing unit 210 and the feature extraction and fusion unit 220 of the edge processing module 200 for processing. When the task is completed, it is removed from the task queue. At this time, the order of the task queue becomes: shovel robot posture and bucket fullness detection task, mining truck posture and material pile distribution detection task in the bucket.

[0027] 2. Loading operation stage (1) Task and priority adjustment: Scene operation environment model reconstruction and identification task: The priority value is reduced to 6. At this time, the environmental model has been established, and only some necessary updates and monitoring are required, so the priority is reduced; Shovel loading robot posture and bucket fullness detection task: The priority value is increased to 9. Because this stage requires real-time and accurate control of the robot's posture and bucket fullness to ensure the efficiency and quality of the shovel loading operation, the priority is greatly increased; Mining truck posture and material pile distribution detection task: The priority value is maintained at 2 or appropriately increased to 3 (depending on the specific situation, such as when the mining truck begins to approach the operation area, etc.). The mining truck has not yet entered the main operation link, so the priority is relatively low.

[0028] (2) Task queue update: Since the priority of the shovel robot posture and bucket fullness detection task is increased, it is removed from the task queue (assuming it was in the queue before) and then reinserted into the queue head. The priority 6 of the large-scale working environment model reconstruction and recognition task is higher than the priority of the mine truck posture and the distribution of material piles in the bucket detection task (assuming it is 3), so the large-scale working environment model reconstruction and recognition task is ranked before the mine truck posture and the distribution of material piles in the bucket detection task.

[0029] At this time, the order of the task queue is: shovel robot posture and bucket fullness rate detection task, large-scene operation environment model reconstruction and recognition task, mine truck posture and material pile distribution detection task.

[0030] (3) Task execution: The scheduler takes out the shovel robot posture and bucket full rate detection task from the head of the queue for execution, then encapsulates the task instruction (including task identifier and priority value), sends it to the device corresponding to the perception module II120 through wireless transmission, and drives the device to collect data, then calculates the robot posture and bucket full rate, and the feature extraction and fusion unit 220 further optimizes and verifies the data. After the task is executed, it is removed from the queue, and then the scheduler takes out the large-scale operation environment model reconstruction and recognition task for processing, and updates and monitors the environment model. After completion, the task is removed again. At this time, only the mining truck posture and the material pile distribution detection task in the bucket remains in the queue.

[0031] 3. Unloading operation stage (1) Task and priority adjustment: Large-scale operation environment model reconstruction and identification task: The priority value is maintained at 6 or appropriately reduced according to actual conditions. The environmental model has been basically stable, and the urgency of updating it has been reduced; Shovel loading robot posture and bucket fullness detection task: The priority value is reduced to 4, the shovel loading operation has been completed, and the importance of this task has decreased; Mining truck posture and material pile distribution in the bucket detection task: The priority value is increased to 10. In the unloading operation stage, it is necessary to accurately grasp the posture of the mining truck and the distribution of the material pile in the bucket to ensure accurate unloading, so it has the highest priority.

[0032] (2) Task queue update: Since the priority of the mining truck posture and material pile distribution detection task is increased to 10, it is removed from the queue (assuming it was in the queue before) and then inserted into the head of the queue.

[0033] The priority 6 of the large-scale working environment model reconstruction and recognition task is higher than the priority 4 of the shovel robot posture and bucket fullness detection task, so the large-scale working environment model reconstruction and recognition task is placed before the shovel robot posture and bucket fullness detection task. At this time, the task queue order is: mining truck posture and material pile distribution detection task, large-scale working environment model reconstruction and recognition task, shovel robot posture and bucket fullness detection task.

[0034] (3) Task execution: The scheduler first takes out the task of detecting the position and distribution of the material pile in the truck bucket, then encapsulates the task instruction (including the task identifier and priority value), sends it to the device corresponding to the perception module III130 through wireless transmission, and drives the device to collect data, so as to determine the position and distribution of the mining truck and the material pile in the truck bucket. After the task is completed, it is removed, and then the large-scale operation environment model reconstruction and recognition task is executed, and finally the shovel robot position and bucket fullness detection task is executed.

[0035] Task processing is completed: When the task is completed, it is removed from the task queue. The scheduler continues to check the queue. If the queue is not empty, the above task removal and execution steps are repeated; Data transmission: The data collected by the sensing devices corresponding to each task (the data pre-processed by the data processing unit 210 and the feature extraction and fusion unit 220) are encapsulated (the encapsulation process includes task identification, priority value, and time information), and then sent to the server module 300.

[0036] The server module 300 is responsible for data transmission with the edge processing module 200 through a reliable network protocol (TCP / IP protocol); The data receiving and storage unit 310 receives the data transmitted by the edge processing module, performs preliminary verification on the data, checks whether the data format is correct and whether the data source is reliable; selects a suitable database storage according to the type and characteristics of the data. The structured business data (point cloud data, image data, bucket fullness rate data, truck material distribution data, personnel and obstacle warning data) are stored in the Mysql database, and some data with high real-time requirements and fast reading and writing (posture information of the shovel loading robot, location information of the mining truck, etc.) are stored in the Redis database; the monitoring data during the system operation (such as the CPU and memory usage of the server itself, user login and registration information) are stored in a special Prometheus database.

[0037] The data analysis and processing unit 320 performs preprocessing operations on the received data and converts the raw data of different sensors into a standard data format; For large-scale working environment model reconstruction and recognition data, 3D modeling algorithms and image recognition algorithms are used to process environmental data and build an accurate working environment model. For the shovel robot posture and bucket fullness rate detection data, sensor fusion algorithms and machine learning algorithms are used to fuse and analyze multi-sensor data to accurately calculate the robot's posture and bucket fullness rate. For the mining truck posture and material pile distribution detection data in the truck bucket, target detection algorithms and spatial analysis algorithms are used to determine the mining truck's posture and the distribution of material piles in the truck bucket; The dynamic resource allocation strategy based on queuing theory and load balancing algorithm stores tasks in corresponding queues according to their priority and arrival time. When new data processing tasks arrive, the load balancing algorithm allocates tasks to computing nodes that meet the task processing requirements based on the load level of the computing nodes and the resource requirements of the tasks.

[0038] The business logic processing unit 330 provides a variety of business functions, including model reconstruction, environmental perception, personnel and obstacle warning, forklift posture, bucket fullness statistics, mining truck posture, mining truck material distribution monitoring, reports, permissions, etc. According to the results of the data analysis and processing unit 320, the rule engine algorithm is used to formulate trigger rules for each business, and when the rule conditions are met, the corresponding business function is automatically triggered. The corresponding function can be executed according to the corresponding request issued by the user on the front-end interface.

[0039] The system management unit 340 is responsible for managing the operating parameters of the server module, setting and managing user access rights, assigning different operating permissions to users of different roles, and recording error information during system startup, data reception and processing, and storing it in the Prometheus database.

[0040] The front-end module 400 is responsible for receiving point cloud data, position information, bucket fullness information, alarm information, mining truck material distribution information, report information and authority management information transmitted by the server module.

[0041] The front-end interface is built based on Vue+Three.js, and HTTP / HTTPS requests are sent to the server to obtain real-time and historical data. The user interface display module 410 intuitively displays the three-dimensional scene, bucket fullness, material distribution of the mining truck, and the position information of the shovel robot and the mining truck; When a user logs in, the interactive operation module 420 obtains the user role and permission information from the system management unit (340) on the server side, verifies and processes it, and stores it in the front-end Prometheus database. Then, the processing result is returned to the front-end, and feedback information can be given in time when the user performs an operation; The alarm and prompt module 430 receives the alarm information sent by the server, pops up a striking alarm prompt box on the interface, and plays an alarm sound to attract the user's attention; The user management module 440 displays the user login and registration interface. When the user logs in, the user information is sent to the system management unit 340 of the server through HTTP / HTTPS request for verification. After successful verification, the user jumps to the main interface. When the user registers, the front end verifies the format of the information entered by the user to ensure the legitimacy of the information. After the verification is passed, the information is sent to the server for registration processing.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing, characterized in that: include: A sensing module (100) is used to obtain information of various sensor devices of the shovel loading robot and the mining truck, surrounding environment information, and equipment status monitoring information; An edge processing module (200) pre-processes the information transmitted by the sensing module (100) to improve data quality, extracts key features of the data, and preliminarily integrates information from different sensing devices; The server module (300) performs deep integration of multi-source data to implement business logic execution, data storage management, and system coordination and communication; The front-end module (400) provides an operation and interaction interface for the user, and intuitively displays the data processed by the system to the user.

2. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 1 is characterized in that: The perception module (100) comprises: The perception module I (110) comprises a laser radar A and a laser radar B respectively installed at the left and right positions of the front part of the shovel loader robot body, a laser radar C installed directly behind the body, a laser radar D installed on the top of the cab or on the top of the body through a pole bracket to achieve 360° rotation scanning and overlap with the scanning range of the laser radars A, B, and C by 10%-50%, and an IMU inertial measurement unit installed on the solid-state laser radar body for laser radar posture correction; The perception module II (120) includes an anti-vibration and anti-shake camera installed on the outer side of the bottom plate around the body of the shovel loader robot, a body inertial navigation system for accurate positioning through four marks around the body, a rotary encoder installed at the bottom of the upper vehicle near the top of the crawler track, a pushing mechanism encoder installed on the pushing mechanism of the shovel loader robot, and a solid-state laser radar depth camera A installed at the middle and upper part of the boom for identifying the fullness rate of the material in the bucket; The perception module III (130) includes a solid-state laser radar depth camera B and a solid-state laser radar depth camera C respectively installed at the left and right front side guardrails of the vehicle roof, and a mining truck inertial navigation system for accurately locating the position of the mining truck.

3. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 1 is characterized in that: The edge processing module (200) comprises: A data processing unit (210) is used to receive data from a laser radar, an anti-vibration and anti-shake camera, an IMU inertial measurement unit, a laser radar depth camera, an encoder, and an inertial navigation system, and perform point cloud processing, motion compensation, and attitude estimation on the data; The feature extraction and fusion unit (220) is used to perform target detection, target extraction and point cloud segmentation on the data processed by the data processing unit, and to achieve multi-sensor data fusion to obtain information related to the working environment of the shovel loader robot and the status information of the device itself.

4. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 3 is characterized in that: The data processing unit (210) is specifically used to receive point cloud data collected by a solid-state laser radar for preprocessing operations, receive image data collected by an anti-vibration and anti-shake camera for denoising, enhancement and correction, receive attitude data output by an IMU inertial measurement unit for calibration and compensation, receive depth image data obtained by a solid-state laser radar depth camera for hole filling and smoothing, receive position and angle information fed back by an encoder for analysis and conversion, receive position information collected by an inertial navigation system for calibration and compensation, and perform time synchronization and spatial alignment on the above-mentioned processed data to achieve motion compensation and attitude estimation; The feature extraction and fusion unit (220) is specifically used to process the data of the laser radar and the anti-vibration and anti-shake camera based on a deep learning algorithm to extract the feature information of the working environment, identify the target object, and fuse the multi-source data to improve the accuracy of environmental model reconstruction and target recognition accuracy; use deep learning to extract the position and posture of the shovel robot from related equipment and calculate the position and full bucket rate; extract the feature information of the mining truck from the related equipment based on the deep learning algorithm and determine the position of the mining truck and the distribution of materials in the vehicle; perform deep fusion of multi-sensor data to generate the working environment model, the position of the shovel robot, the full bucket rate of the bucket, the position of the mining truck and the distribution of materials in the vehicle.

5. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 1 is characterized in that: The server module (300) comprises: A data receiving and storage unit (310) receives various types of data preliminarily processed by the edge processing module through a transmission protocol, stores the point cloud data, image data, and sensor measurement values ​​in a database, and establishes corresponding indexes; A data analysis and processing unit (320) performs preprocessing operations on the data transmitted by the edge processing module (200), performs data analysis according to the priority and arrival time of the task, and dynamically allocates computing resources; The business logic processing unit (330) monitors the potential safety hazards in the working environment in real time according to the results of the data analysis and processing unit (320), arranges the operation and planning of the shovel loading robot in a reasonable manner according to the requirements of the working task and the real-time status information of the equipment, and provides functions such as model reconstruction, environmental perception, personnel and obstacle warning, shovel truck posture, bucket fullness rate, mining truck posture, truck material distribution monitoring, reporting, and authority; The system management unit (340) is responsible for managing the operating parameters of the server module (300), setting and managing the user's access rights, assigning different operation permissions to users of different roles, and recording error information during system startup, data reception and processing, and storing it in a database.

6. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 1 is characterized in that: The front-end module (400) comprises: The user interface display module (410) visualizes various data of the server module (300), displays the statistics of the full bucket rate through charts, displays the distribution of materials in the mining truck in the form of a three-dimensional model, and displays the reconstruction results of the large-scale operation model, the shovel robot and the mining truck posture information in the form of a map; An interactive operation module (420) provides a human-computer interaction function, allowing operators to perform operations within their authority and also to interact with the system by sending remote control commands; An alarm and prompt module (430) uses an SRS streaming media service to display the video images collected by the anti-vibration camera in real time, and sets an alarm threshold to control the personnel and obstacle alarm functions; The user management module (440) logs in according to the user role and can only see and operate the operations and data within the authority.

7. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to any one of claims 1 to 6, characterized in that: The edge processing module (200) assigns a priority to each task based on a priority scheduling algorithm, and drives the corresponding sensor device of the perception module (100) to work. The steps of the priority scheduling algorithm are as follows: Define task nodes: The large-scale operation environment model reconstruction and recognition task is marked as T1; the shovel robot posture and full bucket rate detection task is marked as T2; the mining truck posture and material distribution detection in the bucket is marked as T3; Priority setting and task scheduling and execution: Job preparation phase: Set the priority of T1 to 8, the priority of T2 to 3, and the priority of T3 to 2; initialize the task queue and insert tasks in order of priority; the scheduler takes the T1 task from the head of the queue for execution and removes the task after completion; During the shoveling operation phase: adjust the priority of T1 to 6, the priority of T2 to 9, and the priority of T3 to 2 or increase it to 3; update the task queue and move T2 to the head of the queue; The scheduler takes out tasks T2, T1, and T3 in turn for execution, and removes the corresponding tasks after completion; Unloading operation stage: adjust the priority of T1 to maintain at 6 or reduce it appropriately, the priority of T2 to 4, and the priority of T3 to 10; update the task queue and move T3 to the head of the queue; the scheduler takes out the T3, T1, and T2 tasks in turn for execution, and removes the corresponding tasks after completion; Task processing completed: After the task is executed, it is removed from the task queue. The scheduler continues to check the queue. If it is not empty, the task extraction and execution steps are repeated; Data transmission: The data collected and pre-processed by the sensing devices corresponding to each task are packaged and sent to the server module.

8. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 5 is characterized in that: The data receiving and storage unit (310) performs preliminary verification on the received data, and stores structured business data in a Mysql database, data with high real-time requirements in a Redis database, and monitoring data during system operation in a Prometheus database according to the type and characteristics of the data; the data analysis and processing unit (320) uniformly converts the raw data of different sensors into a standard data format, processes different types of data using corresponding algorithms, and allocates computing nodes based on a dynamic resource allocation strategy of queuing theory and load balancing algorithms; the business logic processing unit (330) uses a rule engine algorithm to formulate trigger rules for each business, and executes corresponding functions according to user requests on the front-end interface.

9. The highly adaptable shovel loading robot intelligent perception system based on on-demand trigger sensing according to claim 6 is characterized in that: When a user logs in, the interactive operation module (420) obtains the user role and permission information from the system management unit (340) on the server side, verifies and processes it, stores it in the front-end Prometheus database, and gives feedback information during operation; the alarm and prompt module (430) receives the alarm information from the server side, pops up an alarm prompt box on the interface and plays an alarm sound; the user management module (440) sends the information to the server side for verification when the user logs in, and verifies the format of the user input information when registering and sends it to the server side for processing.

Images