Cooperative control method and system for mining grouting equipment based on multi-modal data

Abstract

The invention discloses a cooperative control method and system for mining grouting equipment based on multi-modal data, and relates to the technical field of multi-modal data, and the method comprises the steps: determining a plurality of remaining work contents according to a work state, the work progress of each work project and the corresponding project content; according to the technical scheme, the multiple cooperative positions are determined based on tracing of the multiple remaining working contents, the multi-modal data are determined according to the cooperative positions, the multiple remaining working contents and the working state of the mining grouting equipment, the corresponding cooperative control contents are determined based on identification of the multi-modal data, and the identification accuracy of the cooperative control contents is improved. Therefore, corresponding cooperative equipment is determined according to the cooperative control content and the functional tool corresponding to the mining grouting equipment, and a cooperative control sequence table is determined based on the current working state of each cooperative equipment, the current working state of the mining grouting equipment and each cooperative position; and on the basis of the cooperative control sequence table, each cooperative device and the mining grouting device determine a corresponding cooperative control mode.

Description

Technical Field

[0001] The present invention relates to the technical field of multimodal data, and in particular to a collaborative control method and system for mining grouting equipment based on multimodal data. Background Art

[0002] With the development of science and technology, mining grouting equipment is used as mining equipment for grouting reinforcement and water blocking operations. The mining grouting equipment injects slurry in the working project. In the existing technology, multiple working data of the mining grouting equipment are collected, and the working status of the mining grouting equipment is determined based on the multiple working data. The working efficiency of the mining grouting equipment is regulated based on the adjustment of the working status of the mining grouting equipment. The collaborative control content is not taken into consideration, and the single operation of the mining grouting equipment is overly dependent, which affects the collaboration of various collaborative equipment and the mining grouting equipment, and reduces the accuracy of the collaborative control mode. Summary of the Invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art, and the present invention provides a collaborative control method and system for mining grouting equipment based on multimodal data.

[0004] An embodiment of the present invention provides a collaborative control of mining grouting equipment based on multimodal data, including: determining a work item sequence list of the mining grouting equipment based on a current working event of the mining grouting equipment and a current position of the mining grouting equipment, and marking the work progress of each work item; determining a working state of the mining grouting equipment according to multiple working data of the mining grouting equipment, and determining multiple remaining work contents according to the working state, the work progress of each work item and the corresponding project content; determining multiple collaborative positions based on the tracing of the multiple remaining work contents, determining multimodal data according to each collaborative position, the multiple remaining work contents and the working state of the mining grouting equipment, and determining corresponding collaborative control content based on the identification of the multimodal data; determining corresponding collaborative devices according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and determining a collaborative control sequence list based on the current working state of each collaborative device, the current working state of the mining grouting equipment and each collaborative position; determining a corresponding collaborative control mode based on the collaborative control sequence list, each collaborative device and the mining grouting equipment, so as to autonomously regulate the work progress of each collaborative device and the work progress of the mining grouting equipment.

[0005] An embodiment of the present invention provides a collaborative control system for mining grouting equipment based on multimodal data. The collaborative control system for mining grouting equipment based on multimodal data is applied to the collaborative control method for mining grouting equipment based on multimodal data described above. The collaborative control system for mining grouting equipment based on multimodal data includes: A work item sequence list module is used to determine a work item sequence list of the mining grouting equipment based on the current working event of the mining grouting equipment and the current position of the mining grouting equipment, and mark the work progress of each work item; The remaining work content module is used to determine the working status of the mining grouting equipment according to multiple working data of the mining grouting equipment, and determine multiple remaining work contents according to the working status, the work progress of each work project and the corresponding project content; A collaborative control content module is used to determine multiple collaborative positions based on the tracing of multiple remaining work contents, determine multimodal data according to each collaborative position, multiple remaining work contents and the working status of the mining grouting equipment, and determine corresponding collaborative control content based on the recognition of the multimodal data; A collaborative control sequence table module is used to determine the corresponding collaborative equipment according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and to determine the collaborative control sequence table based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment, and each collaborative position; The collaborative control mode module is used to determine the corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device and the mining grouting equipment, so as to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment.

[0006] Compared with the prior art, the present invention has the following beneficial effects: In an embodiment of the present invention, through the method in the embodiment of the present invention, multiple collaborative positions are determined based on the tracing of multiple remaining work contents, multimodal data is determined according to each collaborative position, multiple remaining work contents and the working status of the mining grouting equipment, and the corresponding collaborative control content is determined based on the identification of the multimodal data. The working status of the mining grouting equipment and multiple remaining work contents are introduced, and the overall consideration of each collaborative position, multiple remaining work contents and the working status of the mining grouting equipment is compatible, thereby improving the recognition accuracy of the collaborative control content.

[0007] Therefore, the corresponding collaborative equipment is determined according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and the collaborative control sequence table is determined based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment and each collaborative position; based on the collaborative control sequence table, each collaborative equipment and the mining grouting equipment, the corresponding collaborative control mode is determined, and the collaborative control sequence table is introduced to realize the overall consideration of the collaborative control sequence table, each collaborative equipment and the mining grouting equipment, improve the accuracy of the collaborative control mode, and realize the autonomous regulation of the working progress of each collaborative equipment and the working progress of the mining grouting equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 11 is a flow chart of a collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 2 1 is a flow chart of step S11 in the collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 3 1 is a flow chart of step S12 in the collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 4 1 is a flow chart of step S13 in the collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 5 1 is a flow chart of step S14 in the collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 6 1 is a flow chart of step S15 in the collaborative control method for mining grouting equipment based on multimodal data in an embodiment of the present invention; Figure 7 It is a schematic diagram of the structural composition of a collaborative control system for mining grouting equipment based on multimodal data in an embodiment of the present invention. DETAILED DESCRIPTION

[0009] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0010] See also Figures 1 to 7 A collaborative control method for mining grouting equipment based on multimodal data is applied to the collaborative control scenario of mining grouting equipment; the collaborative control method for mining grouting equipment based on multimodal data includes: Step S11: determining a work item sequence list of the mining grouting equipment based on the current work event of the mining grouting equipment and the current position of the mining grouting equipment, and marking the work progress of each work item; Step S12: determining a working state of the mining grouting equipment according to a plurality of working data of the mining grouting equipment, and determining a plurality of remaining work contents according to the working state, the work progress of each work item and the corresponding project content; Step S13: determining multiple collaborative positions based on tracing the multiple remaining work contents, determining multimodal data according to each collaborative position, the multiple remaining work contents, and the working state of the mining grouting equipment, and determining corresponding collaborative control content based on the identification of the multimodal data; Step S14: determining corresponding collaborative devices according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and determining a collaborative control sequence table based on the current working status of each collaborative device, the current working status of the mining grouting equipment, and each collaborative position; Step S15: determining a corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device and the mining grouting equipment, so as to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment; refer to Figure 2 In step S11, the specific steps are: S111: collecting the current location of the mining grouting equipment, determining multiple completed events and current execution content of the mining grouting equipment based on the tracing of the current location of the mining grouting equipment, and determining the current working event of the mining grouting equipment based on the multiple completed events and current execution content of the mining grouting equipment; S112: Determine multiple current work items based on the analysis of the current work event of the mining grouting equipment, and determine a work item sequence list of the mining grouting equipment according to the synthesis of the contents and sequences of the multiple current work items; S113: In the work item sequence table of the mining grouting equipment, the work progress of each work item is dynamically updated as each current work item is carried out, and the work track of the mining grouting equipment in each work item is recorded.

[0011] In the embodiments of the present application, the precise geographic location information of the equipment in the mine is obtained; since GPS signals are usually not available underground, it is necessary to rely on underground-specific positioning technology; common technologies include: UWB (ultra-wideband) positioning: multiple UWB base stations are deployed in the mine tunnels, and UWB tags are installed on the grouting equipment. By measuring the precise time of flight (TOF) between the tag and multiple base stations, the three-dimensional coordinates of the tag can be calculated; the accuracy can usually reach the centimeter level.

[0012] The collected location data is usually three-dimensional coordinates (X, Y, Z) and direction (posture) information, and is timestamped to indicate the specific moment the device was located at that location. The device's position change trajectory over a period of time is used to infer what actions it has completed (completed events) and what operations it is currently performing (current execution content). This requires combining the mine's geographic information (such as tunnel maps and working face layouts) with the workflow logic of the grouting equipment.

[0013] At this point, the system will obtain the equipment's location history for the most recent period (for example, the past 5 minutes or 10 minutes), which forms a spatial trajectory; the mine map is divided into different functional areas, such as: equipment parking area, material preparation area, entrance to the working face to be grouting, grouting operation point, completed grouting area, maintenance area, etc. By matching the equipment's historical location with these predefined areas, it can be determined which areas the equipment has entered; combined with regional information and typical work processes, events are inferred.

[0014] At the same time, if the trajectory shows that the device has moved from the parking area to the material preparation area and stayed there for a period of time, the "Completed: Device moved to the material area" and "Completed: Material loading" events are inferred; if the trajectory shows that the device has moved from the material preparation area to a specific grouting operation point (for example, the coordinates are near [X1, Y1, Z1]), the "Completed: Moved to the target grouting point" event is inferred; if the trajectory shows that the device is currently staying near the grouting operation point and its position fluctuates within a small range (indicating that a fixed operation is in progress), then the "Current Execution Content" is "Grouting operation in progress".

[0015] In addition to location, other sensor data can also be combined to assist in determining the current execution content. The camera can capture images of the equipment connecting pipes, operating the control panel, or pumping slurry; such as the pump's operating status (start / stop), motor current, pressure sensor reading, etc. For example, if the pump is running and the pressure is rising, "Current execution content: Grouting in progress" is strongly supported.

[0016] The precise current location, the inferred list of "multiple completed events" and the description of "current execution content" are input into the decision module together; the system maintains a standard list of work events, such as "equipment on standby", "moving to the material area", "loading materials", "moving to the grouting point", "preparing for grouting", "executing grouting", "grouting paused", "grouting completed", "cleaning equipment", "moving to the next grouting point", etc.; the decision module determines the work event that best suits the current situation based on the completed events and the current execution content, combined with the sequential logic of the workflow. For example, if the completed event includes "moving to the target grouting point" and the current execution content is "grouting in progress", then the current work event is "executing grouting"; if the completed events include "loading materials" and "moving to the grouting point", but the current execution content is "the pump operation status is stopped and the operator is checking the pipeline", then the current work event is "preparing for grouting" or "grouting paused"; if the equipment is currently in the parking area and there are no other completed events, and the current execution content is "stationary", then the current work event is "equipment on standby".

[0017] Furthermore, the system maintains an "event-work item" mapping rule library or knowledge base. This library defines which specific work items correspond to each "current work event". For example, the event "moving to grouting point 2" needs to be decomposed into specific items such as "starting the mobile motor", "adjusting the moving direction", "monitoring the moving path", and "stopping the movement". Each "work item" should have a clear definition, including: Name: such as "starting the mobile motor" and "adjusting the moving direction"; Content / Goal: the specific operation to be performed or the state to be achieved, such as "the motor speed reaches 500rpm" and "adjusting the vehicle head to the coordinates (130, 40, -150)"; Required Resources / Conditions: the equipment status or external conditions required to execute the project, such as "the motor is in standby state" and "there are no obstacles on the path"; Expected Results / Verification Methods: how to judge whether the project has been completed or successful, such as "the motor speed sensor reading is stable at 500rpm" and "the GPS / IMU shows that the device is facing the target coordinates". Based on the current "work event", the system queries the mapping rule library and dynamically generates a corresponding "current work item" list. This list is "current", which means that it will change with the changes of the "work event".

[0018] The order of work items is usually determined by the following factors: internal dependencies: some items must be started after other items are completed, for example, "precise positioning and stopping" must be after "starting physical movement". This dependency is usually determined when the work items are defined; external process requirements: according to the standard process or safety regulations of grouting operations, certain steps must be performed in a specific order, for example, the target point must be reached and stopped stably before subsequent grouting preparation can be started; optimization considerations: on the premise of meeting the dependencies and process requirements, sometimes the optimization order is considered, for example, steps that are time-consuming or prone to errors are brought forward to enable problems to be discovered as early as possible.

[0019] According to the determined sequential logic, the system arranges the items in the "Current Work Items" list in the correct order and generates a "Work Item Sequence List". This list contains not only the project names, but also the expected start time, estimated duration, priority and other information of each project. This sequence list is "current" and will be regenerated or updated as the "Current Work Event" changes (for example, from "Move to Grouting Point 2" to "Prepare for grouting at Grouting Point 2").

[0020] Therefore, in the work item sequence list of the mining grouting equipment, the work progress of each work item is dynamically updated as each current work item is carried out, and the work track of the mining grouting equipment in each work item is recorded, which introduces the recording of the work track of the mining grouting equipment in each work item.

[0021] At this point, define how to measure the progress of each work item; progress can be a percentage (such as 0% to 100%) or a stage (such as not started, in progress, completed, failed); the measurement method depends on the nature of the work item; the system needs to continuously collect data from device sensors (such as motor encoders, GPS / UWB positioning modules, gyroscopes, accelerometers, pressure sensors, etc.) to determine the degree of completion of the currently executing work item; based on the real-time data collected, the system calculates the progress of the current work item according to the predefined measurement method; when the progress reaches a certain threshold (usually 100%), the system updates the project status to "completed"; if an abnormality is detected (such as sensor failure, serious deviation from the path, motor overload, etc.), the system updates the status to "failed" and triggers an alarm or terminates subsequent projects; the updated progress information will be fed back to the control center or displayed on the operation interface.

[0022] Record the working trajectory of the mining grouting equipment in various work projects. The working trajectory usually includes: timestamp: record the time when each data point occurs; location information: GPS / UWB coordinates, depth, etc.; status information: name of the currently executed work project, progress percentage / status; sensor data: key sensor readings related to the current work project (such as motor speed, pressure, temperature, attitude angle, distance sensor reading, etc.); event log: specific events that occurred (such as reaching a path point, adjustment start / end, alarm triggering, etc.); control instructions: record of instructions sent to equipment actuators (such as motors, valves).

[0023] refer to Figure 3 In step S12, the specific steps are: S121: real-time monitoring of the working process of the mining grouting equipment and collection of multiple working data of the mining grouting equipment. Here, the multiple working data include grouting pressure, grouting speed, grouting path, grouting volume, and stirring speed. S122: constructing corresponding working data combinations based on the plurality of working data and each working node of the mining grouting equipment, determining a plurality of working features at the same time node based on identification of each working data combination, and determining the working state of the mining grouting equipment based on detection of the plurality of working features at the same time node; S123: Real-time monitoring of each work project, collecting the work progress of each work project, marking the work content of each work project, determining the unfinished work content of each work project based on the work progress of each work project and the corresponding project content, and determining a plurality of remaining work contents based on the unfinished work content of each work project and the working status of the mining grouting equipment; In an embodiment of the present application, the working process of the mining grouting equipment is monitored in real time, and the state changes and activities of the equipment are actively and continuously observed; the system needs to know whether the equipment is currently in a moving state, a grouting state, a stirring state, a standby state or a fault state, which is usually achieved by reading the status flag of the internal control unit of the equipment (such as a PLC or an embedded system), analyzing the pattern of the sensor data (for example, if only the position data is changing and the pressure and flow data are 0, it is judged to be moving) or receiving status instructions from the equipment operation interface; optionally, it is achieved by polling the device interface, subscribing to the status topic published by the device (in IoT protocols such as MQTT) or establishing a direct serial port / Ethernet connection; the monitoring frequency needs to be determined according to the application requirements. A high frequency is required for rapidly changing processes (such as pressure fluctuations), and a lower frequency can be used for slowly changing processes (such as equipment position).

[0024] Predefined key values ​​that reflect the equipment's performance and status are extracted from the various sensors and actuators installed on the equipment. These values ​​are objective and quantitative. Each data point corresponds to one or more sensors / actuators. The system needs to be configured with the communication protocols (such as Modbus, CAN bus, I2C, SPI, RS485, etc.) for these hardware and set the frequency and accuracy of data collection. After data collection, preliminary filtering (to remove noise) and unit conversion are usually required to ensure that the data conforms to the system's internal data format requirements. Specific working data includes grouting pressure, grouting speed (flow), grouting path, grouting volume, and stirring speed.

[0025] Furthermore, "working node" refers to the key stage or status point in the equipment workflow, for example, "moving to the target point", "reaching the target point", "starting stirring", "stirring completed", "starting grouting", "grouting in progress", "grouting completed", "pressure check", etc.; the system needs to know which node the current equipment is at (this is usually combined with the results of S111 and S112); then, the "multiple working data" (pressure, speed, path, quantity, stirring speed) collected by S121 at that time point are associated with the current working node to form a meaningful data packet or record. This combination is not just a simple accumulation of data, but gives the data the context of a specific working stage.

[0026] A mapping table or rule base can be established to define which data are key data at each work node. For example, at the "move to target point" node, the key data are mainly "grouting path" (current position and target position) and "grouting speed" (if the equipment's own moving speed can be measured); at the "grouting" node, the key data are "grouting pressure", "grouting speed", "grouting volume" and "stirring speed" (if stirring and grouting are carried out simultaneously or there is intermittent stirring); the system extracts the corresponding data based on the currently judged work node to form a "work data combination".

[0027] "Working characteristics" are key attributes extracted from the "working data combination" that can describe the specific behavior or performance of the equipment at that node. For example, from the "grouting" node combination (pressure 5.8 MPa, speed 12.5 L / min, volume 785.3 L, stirring 0 rpm), the following characteristics can be identified: Feature 1: Currently in the "grouting execution" sub-state; Feature 2: The grouting pressure is in the "normal range" (assuming the system sets the normal range to be 4-6 MPa); Feature 3: The grouting speed is in the "normal range" (assuming the normal range is 10-15 L / min); Feature 4: The agitator is in the "stop" state; Feature 5: The cumulative grouting volume is close to the "target volume" (assuming the target volume is 800 L).

[0028] At this time, the Rule Engine can be used, for example, "IF pressure > 4 AND pressure < 6AND speed > 10 AND speed < 15 THEN feature = {normal grouting, normal speed, normal pressure}".

[0029] The "working status" is a summary description of the current overall situation of the equipment. It integrates multiple "working features" identified from the "working data combination"; it is more macroscopic than a single feature and directly reflects whether the equipment is operating normally, experiencing some abnormality, or in a specific functional stage. For example, based on the above features 1-5, the current working status can be determined to be "normal grouting, close to the target amount"; if a feature is abnormal, such as the pressure feature changes to "pressure too high" (assuming the pressure suddenly rises to 7 MPa), the overall working status becomes "grouting abnormality: pressure too high, requiring suspension or adjustment"; at the same time, a rule engine or state machine can be used; the rule is: "IF feature contains {normal grouting} AND feature contains {normal pressure} AND feature contains {normal speed} THEN working status = normal grouting"; or, "IF feature contains {pressure too high} THEN working status = abnormal grouting: pressure too high"; the state machine defines the transition conditions between different states, which are often based on the detected working features.

[0030] Therefore, each work project is monitored in real time, and the work progress of each work project is collected, and the work content of each work project is marked. The unfinished work content of each work project is determined according to the work progress of each work project and the corresponding project content. The multiple remaining work contents are determined according to the unfinished work content of each work project and the working status of the mining grouting equipment. This is compatible with the overall consideration of the unfinished work content of each work project and the working status of the mining grouting equipment, ensuring the accuracy of multiple remaining work contents.

[0031] At this point, the system needs to keep an eye on each item in the work item sequence list generated by S112. Through S113, the system already knows the progress percentage of each item. Now, it is necessary to read this progress information in real time, for example, to check whether the "Navigation and Path Tracking" item has been completed 100%, whether the "Start Mixing" item has started (progress > 0%), etc. This is usually achieved through an interface with the equipment control unit or status monitoring module; mark the work content of each work item.

[0032] For example: Project 1: Navigation and path tracking - Content: Move from the current position to the specified coordinates of grouting point 2; Project 2: Start stirring - Content: Start the stirring motor and reach the preset speed; Project 3: Continuous stirring - Content: Keep the stirring motor running until grouting starts; Project 4: Start grouting - Content: Open the grouting valve and start the grouting pump; Project 5: Continuous grouting - Content: Keep the grouting pump running, control the pressure and flow until the target grouting volume is reached or a stop command is received; Project 6: Stop grouting - Content: Close the grouting valve and stop the grouting pump; Project 7: Clean equipment - Content: Drain the remaining slurry, flush the pipes and agitator; Project 8: Move to grouting point 3 - Content: Move from grouting point 2 to the specified coordinates of grouting point 3.

[0033] Based on the current progress and project content, determine whether each project has remaining work. For completed projects (100% progress), the unfinished content is blank. For ongoing projects (0% < progress < 100%), the unfinished content is the unexecuted portion of the project content. For unstarted projects (0% progress), the unfinished content is the entire project content. Example of determining unfinished content: Item 1: Unfinished content - Empty (Completed); Item 2: Unfinished content - Empty (Completed); Item 3: Unfinished content - Empty (Completed); Item 4: Unfinished content - Empty (Completed); Item 5: Unfinished content - Keep the grouting pump running, control the pressure and flow until the target grouting volume (2250L - 785.3L = 1464.7L) is reached or a stop command is received; (In progress); Item 6: Unfinished content - Close the grouting valve and stop the grouting pump; (Not started); Item 7: Unfinished content - Drain the remaining slurry, flush the pipes and agitator; (Not started); Item 8: Unfinished content - Move from grouting point 2 to the specified coordinates of grouting point 3; (Not started).

[0034] The system needs to arrange the list of tasks to be performed next based on the unfinished content of each current project and the current working status of the equipment (the result of S122); the working status will affect the priority or execution conditions of the task. For example, if the working status is "grouting abnormality: pressure is too high", then "stop grouting" will be raised to the highest priority; if the working status is "normal grouting, close to the target amount", then the system will expect to complete the current grouting soon and then execute subsequent tasks; example of the remaining work content: current working status (S122 result): normal grouting, close to the target amount; unfinished content of each project (step 3 result): Project 5: complete the remaining 1464.7L grouting; Project 6: stop grouting; Project 7: clean the equipment; Project 8: move to grouting point 3.

[0035] Determine the remaining work content: 1. Continue to execute Item 5: Continue grouting - This is the current task in progress and needs to continue to complete the remaining 1464.7L of grouting; the working status "near target volume" also confirms this; 2. Prepare to execute Item 6: Stop grouting - Once Item 5 is completed (reaching 2250L or receiving instructions), this task needs to be executed immediately; 3. Prepare to execute Item 7: Clean up the equipment - This task is usually executed immediately after Item 6 is completed; 4. Prepare to execute Item 8: Move to grouting point 3 - After Item 7 is completed, the equipment needs to be moved to the next work point.

[0036] refer to Figure 4 In step S13, the specific steps are: S131: Collecting multiple remaining work contents, tracing the multiple remaining work contents, determining multiple remaining work nodes based on the tracing of the multiple remaining work contents, and determining multiple coordinated positions based on the positions of the multiple remaining work nodes, the corresponding remaining work contents, and the current position of the mining grouting equipment; S132: At each collaborative position, a first collaborative content is determined for each collaborative position based on a match between each collaborative position and a plurality of remaining work contents, a second collaborative content is determined for each collaborative position based on each collaborative position and the working state of the mining grouting equipment, and multimodal data is determined based on a synthesis of each collaborative position, the corresponding first collaborative content, and the second collaborative content. S133: In the multimodal data, multiple collaborative paths are determined based on the autonomous identification of the multimodal data, and the corresponding collaborative control content is marked for each collaborative path. At this time, the corresponding collaborative control content is constructed based on the synthesis of each collaborative path, each collaborative position and multiple remaining work contents.

[0037] In an embodiment of the present application, multiple remaining work contents are collected and a list of work items that have not yet been completed is extracted from the output of S123; the system needs a data structure (such as a list or queue) to store these contents; optionally, the system reads from the output of S123 and obtains the following list of remaining work contents: Work content 1: Continue to complete the current grouting (remaining amount: 465L); Work content 2: Prepare to stop grouting; Work content 3: Clean equipment; Work content 4: Move to grouting point 3.

[0038] Trace back multiple remaining work contents, considering all necessary factors required to execute the work content, such as resources, permissions, environmental conditions, cooperation with other equipment, etc.; consider what the status of the equipment will be after the work is completed, and what new tasks or decision points will be triggered; determine whether it is necessary to communicate with external systems, such as requesting permission, sending status, receiving instructions, etc.

[0039] Furthermore, the first collaborative content of each collaborative location is determined based on the matching of each collaborative location and multiple remaining work contents. In order to complete the remaining work content related to the collaborative location, we need to interact with "who" or "what system" at the location. This usually involves functional matching: what functions or services does the collaborative location provide, and what functions or services are required for our work content; the first collaborative content is the specific interactive action or request that needs to be executed after this matching; optionally, based on a preset rule library (for example, the work content of "cleaning equipment" needs to be matched to the function of the "maintenance team information terminal", that is, "requesting the maintenance team to assist in cleaning"), or through an intelligent judgment model to infer based on the work content and location attributes; the first collaborative content is a specific instruction, request, query, or some kind of interactive process that needs to be triggered.

[0040] The second collaborative content of each collaborative position is determined based on the working status of each collaborative position and the mining grouting equipment. According to the current actual operating status of the equipment (for example, grouting, idle, faulty, or the real-time value of certain parameters), the information that needs to be transmitted to the collaborative position to reflect the current status of the equipment is determined. This information helps the responding party at the collaborative position to better understand the situation and make more appropriate decisions or responses; the second collaborative content is information related to these states; optionally, it is necessary to clarify which equipment status information is important for a specific collaborative position. For example, when requesting movement permission from the dispatch room, information such as the current position, speed, expected arrival time, and whether the equipment is intact is important; and when reporting cleaning needs to the maintenance team, information such as the current cleaning status of the equipment and the expected cleaning time is more important; the second collaborative content is usually data, parameters, status descriptions, timestamps, etc., which are an objective reflection of the equipment status.

[0041] Multimodal data is determined based on the synthesis of each collaborative location, the corresponding first collaborative content, and the second collaborative content, integrating the information obtained in the first two steps. The identification of the collaborative location, the first collaborative content to be executed (interaction action / request), and the second collaborative content to be transmitted (status information) are packaged into a unified data structure. This data structure contains multiple data types, such as text (instructions), values ​​(parameters), geographic location (location information), timestamps, etc., and is therefore called "multimodal data". This data will become the basic input for path planning and instruction generation in subsequent steps. Optionally, a clear data structure is designed to accommodate this information, such as a JSON object or database record. The data also contains multiple information forms such as descriptive text, quantitative values, geographic location coordinates, etc.

[0042] Specifically, S131 identified three collaborative locations: Collaborative location 1: East District Management Center of the mine; Collaborative location 2: West District Maintenance Station of the mine; Collaborative location 3: Central Dispatching Room of the mine; The current working status of equipment A (according to S123): Grouting in progress, pressure 6.2 MPa, flow rate 50 L / min, 1535.3 L / 2000 L injected, and the equipment operating normally.

[0043] For collaborative location 1: Mine East District Management Center Determine the first collaborative content: This location is related to the remaining work content 2 (prepare to stop grouting) and B (cleaning preparation); the regional management terminal is usually responsible for equipment management and status monitoring in the area; in order to "prepare to stop grouting", it is necessary to report to it that the grouting is about to be completed; in order to "clean up preparation", it is necessary to notify the regional management terminal in advance that the equipment is about to enter the cleaning stage; the first collaborative content: report the status of the grouting being about to stop and notify that cleaning is about to begin.

[0044] Determine the second collaborative content: transmit status information related to "prepare to stop grouting" and "cleaning preparation"; the regional management terminal needs to know the grouting progress and equipment status to update the monitoring information, and needs to know the approximate time of cleaning to coordinate regional resources; the second collaborative content: grouting is currently in progress, pressure, flow, injected volume, target volume, estimated remaining time, and equipment are normal. The next task is cleaning, which is expected to start in 10 minutes; combine the location, the first content, and the second content, and construct multimodal data.

[0045] By associating and combining the collaborative location, work content, and device status, multimodal data is generated for each collaborative location. This data not only includes the actions to be performed (first collaborative content), but also the background information required to perform the actions (second collaborative content), and clearly indicates the target location of the interaction.

[0046] Therefore, in multimodal data, multiple collaborative paths are determined based on the autonomous identification of multimodal data, and the corresponding collaborative control content is marked for each collaborative path. At this time, the corresponding collaborative control content is constructed according to the synthesis of each collaborative path, each collaborative position and multiple remaining work contents, which is compatible with the overall consideration of the synthesis of each collaborative path, each collaborative position and multiple remaining work contents, and ensures the accuracy of the corresponding collaborative control content. At the same time, the working status of the mining grouting equipment and multiple remaining work contents are introduced, which is compatible with the overall consideration of each collaborative position, multiple remaining work contents and the working status of the mining grouting equipment, and improves the recognition accuracy of the collaborative control content.

[0047] At this time, the multimodal data generated by S132, which includes location, content, and status, is used to infer or calculate the path to reach each collaborative location and conduct effective interaction. The "path" here can be in various forms, such as: physical path, logical communication path, and task execution path.

[0048] Match the location information in the multimodal data with the pre-stored map or network topology to find a reachable path; alternatively, determine the path based on preset rules (such as "going to the West District Maintenance Station must pass through the West District Main Tunnel" or "communication with the East District Management Center must pass through the underground 5G base station B").

[0049] Convert the abstract "request path planning" into a specific MQTT message payload, convert the "request device maintenance" into a specific maintenance request message, and convert the "physical moving path" into a specific navigation point sequence and speed instructions; Ensure that the collaborative control content is designed for a specific path. For example, the collaborative control content of the physical moving path is a navigation instruction, and the collaborative control content of the logical communication path is a network message; The control content should contain all the information required to perform the action, such as the target address, request parameters, device ID, timestamp, etc.; Optionally, for collaborative path 1: Mark the collaborative control content: Generate an MQTT message based on the first collaborative content (request path planning) and the second collaborative content (current location, target point).

[0050] The corresponding collaborative control content is constructed based on the synthesis of each collaborative path, each collaborative position and multiple remaining work contents. The collaborative control content for a specific path has been generated, but here it is necessary to combine these contents more closely with the collaborative position information and the original remaining work content associated with it to form a final and more complete collaborative control instruction package, which helps to clarify the context of the instruction package during execution.

[0051] refer to Figure 5 In step S14, the specific steps are: S141: Collecting collaborative control content, determining a corresponding collaborative action based on the analysis of the collaborative control content, and determining a corresponding collaborative function coefficient based on the collaborative action and the current working state of the mining grouting equipment; determining a corresponding collaborative device based on the collaborative function coefficient, the mapping relationship between the functional tool corresponding to the mining grouting equipment, and the collaborative device; S142: Determine a plurality of first collaborative control data based on the current working status of each collaborative device and the current working status of the mining grouting equipment; S143: Determine a plurality of second collaborative control data according to the matching of the current working status of each collaborative device and each collaborative position, and determine a collaborative control sequence table based on the synthesis of the plurality of first collaborative control data and the plurality of second collaborative control data.

[0052] In an embodiment of the present application, to collect collaborative control content, the system needs to extract the specific collaborative control instructions or plans that need to be executed currently from the results generated in the previous stage (S13, especially S133). These contents are usually stored in a structured form, such as database records, messages in a message queue, or configuration items in a file; the system needs to accurately read and parse this data to provide input for subsequent steps.

[0053] Decompose and refine the collected collaborative control content; the system needs to decompose high-level instructions (such as "move and request replacement") into a series of more specific and executable operational steps (i.e., "collaborative actions"), which usually involves parsing the "decomposed action list" or similar fields in the "collaborative control content"; each collaborative action should have a clear action type, target object, and expected result.

[0054] The collaborative function coefficient is a quantitative indicator used to represent the "difficulty", "resource consumption" or "risk" of performing a specific collaborative action under the current equipment status; this coefficient can be used for subsequent resource allocation and priority sorting. The determination of this coefficient needs to consider: the collaborative action itself: the mobile action considers the distance and path complexity, and the communication action considers the data volume and network conditions; the current working status of the mining grouting equipment: for example, if the battery of the device is low, the "difficulty" coefficient of the mobile action will increase; if the communication module of the device fails, the "difficulty" coefficient of the communication action will be very high (even infinite or unexecutable).

[0055] Optionally, assume that the current real-time status of the mining grouting equipment is as follows: Location: East District Working Face (approximately 2 kilometers from the West District Maintenance Station); Battery Level: 30%, Communication Module: Normal; Mobility: Normal; Now evaluate the synergy coefficient of the two coordinated actions: Action 1 (Act_001): Physically move to the West Maintenance Station; Factors considered: Distance (2 km), Battery Level (30%), Mobility (Normal); Calculation / Evaluation: The system's built-in algorithm calculates this as follows: Base Movement Factor (based on distance) + Battery Level Penalty Factor (a bonus if battery level is below 50%) + Mobility Modifier Factor; Assume Base Movement Factor is 2.0 (kilometer factor); Battery Level Penalty Factor is 1.5 (because it is below 50%); Mobility Modifier Factor is 1.0 (Normal); Synergy Factor = 2.0 1.5 1.0 = 3.0.

[0056] Action 2 (Act_002): Communicate with the West District Maintenance Station control system to request replacement of the standby grouting pump; factors to consider: communication module status (normal), expected data volume (medium), network status (assuming the system detects that the current network signal is good); Calculation / Evaluation: The system's built-in algorithm will calculate as follows: basic communication coefficient (based on data volume) + communication module status correction coefficient + network status correction coefficient; assuming the basic communication coefficient is 0.5 (medium data volume); the communication module status correction coefficient is 1.0 (normal); the network status correction coefficient is 1.0 (good); the collaborative function coefficient = 0.5 1.0 1.0 = 0.5, this coefficient means that under the current state, the "difficulty" or "resource consumption" of executing the communication request is very low.

[0057] The system will decide whether higher-performance or more specialized collaborative equipment is needed based on the coefficient. For example, if the movement coefficient is very high (due to long distance, low battery, or poor road conditions), the system will need to request more powerful transportation equipment (such as mining trucks) to assist in movement, rather than relying solely on the equipment itself.

[0058] Mining grouting equipment carries some tools or interfaces that can cooperate with specific collaborative equipment. For example, the equipment comes with a standard communication interface and can only communicate with the maintenance station control system that is compatible with this interface. The collaborative equipment mapping relationship is a pre-established database or configuration table that defines: what types of collaborative equipment are usually required for each collaborative action type; what collaborative equipment is available at each target location (such as the West District Maintenance Station) and its current status (idle, busy, faulty); the compatibility of interfaces between devices (for example, which maintenance station system can my communication interface be connected to); based on this information, the system will select the most suitable, most available, and functionally satisfactory collaborative equipment.

[0059] Furthermore, the system needs to query or receive in real time the current status information of the collaborative devices previously determined in S141 (for example, truck Truck-05 and maintenance station control system MS-System-C). This information usually comes from the sensors, monitoring systems or management platforms of these devices themselves; status information includes: operating status: idle, executing tasks, faulty, under maintenance, standby, etc.; location information: the current precise coordinates or area; resource status: such as whether the truck is full of fuel, whether the maintenance station has a spare pump inventory, whether the system is online, etc.; performance indicators: such as truck driving speed, system response time, etc.

[0060] The system needs to obtain its own (mining grouting equipment) current status information. This information has been partially determined in S122, but a more comprehensive or latest status is needed here, including: equipment status: running, paused, stopped, faulty (such as the current Err-503, grouting pump failure), maintenance; location information: current location coordinates; task status: currently executed tasks, completed progress, remaining progress; resource status: such as whether there are sufficient materials (cement, water), the status of other subsystems (such as mixing system).

[0061] The system compares and analyzes the status of the collaborative equipment with the status of the mining grouting equipment, evaluates the feasibility and urgency of the collaborative operation, and generates the first collaborative control data. This data includes: determining whether the collaborative equipment is truly available (for example, although the status of Truck-05 is idle, it is far away and the response time is long; the maintenance station system MS-System-C is available); determining whether the collaborative equipment has the resources to perform the required actions (for example, whether the maintenance station has the correct backup pump model JZB-2000); evaluating the time required for the collaborative equipment to arrive or start collaboration, and the degree of match with the current status of the mining grouting equipment and the task progress (for example, if equipment JZB-01 has stopped, the waiting time is acceptable; if the equipment is still running on the critical path, the waiting time is not allowed); identifying existing conflicts (for example, whether Truck-05 is scheduled for other tasks, whether the maintenance station has other urgent repairs) or operational risks (for example, whether moving equipment JZB-01 to the maintenance station requires special path planning or safety measures); based on the status evaluation, the priority of the collaborative action needs to be adjusted; Therefore, multiple second collaborative control data are determined based on the current working status of each collaborative device and the matching of each collaborative position, and the collaborative control sequence table is determined based on the synthesis of multiple first collaborative control data and multiple second collaborative control data, which is compatible with the overall consideration of the synthesis of multiple first collaborative control data and multiple second collaborative control data, and ensures the accuracy of the collaborative control sequence table.

[0062] At this time, determine the distance and direction between the current position of the collaborative device and the target collaborative position, and plan a rough moving path (if movement is required), which can be done with the help of GPS, an underground positioning system, or a preset path network; estimate the time required for the device to move from the current position to the target collaborative position based on the device's moving speed, path complexity (such as congestion in underground tunnels), waiting time, etc.; reconfirm whether the device has the resources required to complete the task after arriving at the target position (such as whether the maintenance station has a spare pump after the truck arrives at the maintenance station; whether the maintenance station system can complete the pump replacement preparation under remote control), which requires considering the resource status of the target position (collaborative position); combine the above-mentioned positioning, time, resource matching and other information with the current status of the device (from S142) to generate a second set of collaborative control data, which focuses more on the feasibility of the spatial and temporal dimensions.

[0063] Combine the "first collaborative control data" generated by S142 (device status, availability, resource matching, etc.) and the "second collaborative control data" just generated (spatial position matching, time estimation, path planning, etc.), conduct comprehensive evaluation and sorting, and finally form a list of steps executed in chronological order - a collaborative control sequence list.

[0064] At this point, the information in the two sets of data is correlated. For example, the first data for Truck-05 indicates that it is available but far away, while the second data provides the specific arrival time and route. The first data for the maintenance station system MS-System-C indicates that it is fully available, while the second data indicates that it can start working immediately. The order of operations is determined based on the urgency of the task, the arrival time of the equipment, and resource dependencies. For example, if the preparation of the backup pump can be completed before the truck arrives, then this part of the operation can be performed first. Check whether there are any operations with time or resource conflicts, such as whether two devices need to occupy the same resource at the same time, whether the paths intersect and cause congestion, and whether resolving the conflict requires adjusting the order or waiting. The sorted operation steps, along with the necessary context information (such as the required equipment, location, expected time point, person in charge / system, etc.), are written into the collaborative control sequence table. Optional: Truck-05: Available, but takes 45 minutes to reach the West District Maintenance Station, and needs to wait for maintenance personnel after arrival; resource matching (pump available); Maintenance Station System MS-System-C: Available, can start work immediately, and can remotely prepare a backup pump.

[0065] Generate a sequence listing, presented as follows: Step 1 (time point: immediately): executed by the maintenance station system MS-System-C; Task: remotely check the status of the JZB-2000 standby pump at the West District Maintenance Station and confirm its availability; if available, remotely unlock the pump's fixture and prepare for lifting; Basis: the system is immediately available, and preparations can be made in advance for the arrival of the truck; Step 2 (time point: immediately): executed by the dispatch center (or the grouting equipment JZB-01 control system); Task: send a command to truck Truck-05 to start the journey to the West District Maintenance Station; Basis: the truck is available, and although the distance is far, it must start moving; Step 3 (time point: estimated arrival time - 5 minutes): executed by the maintenance station system MS-System-C; Task: reconfirm the status of the standby pump and notify maintenance personnel (if necessary) to prepare to welcome the truck; Basis: the truck is about to arrive and final confirmation and personnel preparation are required; Step 4 (Time: Truck-05's estimated arrival time): Executed by Truck-05 (physical action); Task: Arrive at the West District Maintenance Station and load the spare pump; Basis: Truck arrival; Step 5 (Time: Truck loading completed): Executed by the dispatch center; Task: Instruct Truck-05 to proceed to the East District fault point (where JZB-01 is located); Basis: The pump has been loaded; Step 6 (Time: Truck-05 arrives at the East District fault point): Executed by Truck-05 (physical action); Task: Unload the spare pump to the designated location; Basis: Truck arrival at the destination; Step 7 (time point: pump unloading is completed): performed by maintenance personnel (or remote guidance); Task: install the backup pump to the position of JZB-01, connect the pipeline and power supply; Basis: the pump is in place and requires manual installation; Step 8 (time point: pump installation is completed): performed by the grouting equipment JZB-01 control system; Task: switch to the newly installed backup pump, perform functional testing, and confirm that the fault has been eliminated; Basis: the new pump is installed and grouting can be resumed.

[0066] refer to Figure 6 In step S15, the specific steps are: S151: Determine a plurality of collaborative items in the collaborative control sequence table based on parsing the collaborative control sequence table, and determine a first collaborative control coefficient according to the plurality of collaborative items and the corresponding collaborative devices; S152: Determine a second collaborative control coefficient based on the multiple collaborative projects and the mining grouting equipment, and determine a corresponding collaborative control mode based on a mapping relationship among the first collaborative control coefficient, the second collaborative control coefficient, and the collaborative control mode; S153: In this collaborative control mode, the collaborative control relationship between each collaborative device and the mining grouting equipment is collected, and the working progress of each collaborative device and the working progress of the mining grouting equipment are determined based on the detection of the collaborative control relationship. The autonomous control logic of the work progress is determined according to the working progress of each collaborative device, the working progress of the mining grouting equipment and the corresponding remaining work content, and the working progress of each collaborative device and the working progress of the mining grouting equipment are autonomously controlled based on the autonomous control logic.

[0067] In the embodiment of the present application, the system first obtains the collaborative control sequence table generated in S143. This sequence table is a list of operation steps arranged in chronological order, including information such as what operation is performed, who performs it (which collaborative device or person), where it is performed, and when it is expected to start and end; The system needs to identify each independent, executable operation step in the sequence list as a "collaborative project". These projects should be relatively independent, can be assigned to a certain device or person for execution, and have clear start and end marks; for each identified collaborative project, the system needs to extract its key attributes, such as: Project ID: unique identifier, such as "CP-001"; Project name / description: such as "Truck Truck-05 departs from the West District Warehouse and goes to the East District Fault Point"; Execution Equipment: Equipment responsible for executing the project, such as "Truck-05"; Target Location: Destination or work area for project execution, such as "East District Fault Point (near JZB-01)"; Estimated Start Time: The time point when the project is scheduled to start; Estimated End Time: The time point when the project is scheduled to complete; Dependency: Whether the project depends on the completion of a previous project in the sequence list, for example, "Truck Arrives at the Fault Point" depends on the completion of "Truck Departure".

[0068] Collect the collaborative control sequence table, which is shown in Table 1:

[0069]

[0070] The system parses this table, identifying 10 independent collaborative projects (CP-001 to CP-010) and extracting attributes for each project, such as the executing equipment, target location, and time window. The system explicitly associates each collaborative project identified in the previous step with its corresponding executing equipment. For example, project CP-001 (truck departure) is associated with equipment Truck-05, and project CP-007 (installation start) is associated with equipment maintenance personnel. For each project-equipment pair, the system evaluates the factors necessary to control the equipment when executing that specific project. This is typically based on the characteristics of the project itself and the capabilities / status of the equipment. The evaluation dimensions include: Task complexity: Is the project a simple movement (such as a truck departure) or a complex operation (such as installing a pump)? Complex operations require more sophisticated and frequent control instructions; Equipment autonomy: Is the equipment highly autonomous (such as a self-driving truck) or requires a lot of manual intervention (such as maintenance personnel)? Equipment with low autonomy requires more instructions and status confirmation; Environmental factors: What are the environmental conditions in the project execution area? For example, moving a truck in a narrow alley is more difficult to control than in an open area and requires more cautious instructions; Time sensitivity: Does the project must be completed within a specific time window? Urgent tasks require stronger control and faster response; Resource requirements: Does executing the project require calling specific functions or resources of the equipment? For example, installing a pump requires the use of a lifting function, which affects the control method; Equipment status: Although S142 has taken into account the current status of the equipment, this article focuses more on the impact of the equipment status on the control method when executing the project. For example, if the truck is low on fuel, the control coefficient of the departure project needs to consider energy-saving driving strategies.

[0071] Based on this assessment, the system calculates or determines a "first collaborative control coefficient" for each project-equipment pair. This coefficient is a quantitative value (can be a number between 0 and 1, or other ranges) that represents the "strength," "precision," "priority," or "complexity" required to control the equipment to execute the project. A higher coefficient generally indicates the need for more sophisticated, more frequent, and more prioritized control. The system calculates a first collaborative control coefficient for each of the ten project-equipment pairs. These coefficients are used in the next step, S152, along with the coefficients for projects related to mining grouting equipment (JZB-01) (such as CP-009 and CP-010) to determine a higher-level control mode. At this point, S151 analyzes the sequence table, breaking down the macro-plan into micro-tasks (collaborative projects), and assessing the specific control requirements of each task for the executing equipment, quantifying them as a "first collaborative control coefficient." This step lays the foundation for the subsequent selection of an appropriate control mode (S152), ensuring differentiated control strategies for different tasks and equipment.

[0072] Furthermore, the system needs to analyze each collaborative project and clarify the role played by the mining grouting equipment (JZB-01) in the project; it is: Affected party: the execution of the project will directly affect its status (such as waiting for truck unloading, waiting for pump installation); Executing party: the project is executed by the grouting equipment itself (such as fault diagnosis, system self-test, and grouting recovery); Monitoring party: the project requires the grouting equipment to monitor the status of other collaborative equipment (less common); Unrelated party: the project has no direct connection with the grouting equipment.

[0073] Combined with the current status of the grouting equipment (such as fault status, maintenance history, performance parameters, current task priority, etc.), evaluate its involvement and sensitivity in related projects; if the grouting equipment is in a critical fault state, any operation that affects its repair (such as truck unloading, pump installation) will be given higher attention; if the grouting equipment is about to resume operation, the demand for monitoring its testing and startup procedures will be high.

[0074] Based on the above role and status assessment, a "second collaborative control coefficient" is calculated for each collaborative project. This coefficient reflects the importance, sensitivity or involvement of the mining grouting equipment in the project. It is not a fixed value, but changes dynamically according to the relationship between the project and the grouting equipment. The coefficient range can usually be set between 0 and 1. Close to 1: It means that the project is critical to the mining grouting equipment and requires high attention and control, for example, installing a backup pump (CP-006), testing a new pump (CP-008), and resuming grouting (CP-010); close to 0: It means that the project has little impact or is irrelevant to the mining grouting equipment, for example, the truck departs from the warehouse (CP-001) and the truck waits in the east area (CP-002); intermediate value: It means that there is a certain correlation and moderate attention is needed, for example, the truck approaches the east area (CP-003) and unloading the backup pump (CP-004).

[0075] For each collaborative project, the system now has two coefficients: the first collaborative control coefficient (reflecting the control requirements of the task itself) and the second collaborative control coefficient (reflecting the importance / sensitivity of the task to the mining grouting equipment). The collaborative control mode mapping relationship is a predefined rule base or decision table that defines which collaborative control mode should be selected based on the combination of these two coefficients. These modes represent different control strategies and levels of interaction. Common modes include: Mode A: Autonomous Collaboration: The collaborative equipment is highly autonomous, and the system only sends target instructions and does not intervene in the specific process. This is suitable for projects with low first and second coefficients (e.g., trucks driving along a predetermined route within a mining area, unrelated to the grouting equipment). Mode B: Monitored Collaboration: The collaborative equipment executes according to instructions, and the system closely monitors its status and progress, but generally does not actively intervene unless an anomaly occurs. This is suitable for projects with medium first and second coefficients, or high first and low second coefficients (e.g., trucks approaching a designated location to unload cargo, where location and status monitoring is required but not closely related to the grouting equipment; or manual pump installation, where monitoring of the operation is required to ensure that it is carried out according to procedures, which is important to the grouting equipment). Mode C: Command Collaboration: The system needs to send detailed, step-by-step instructions to the collaborative equipment and confirm the completion of each step. This is suitable for projects with a high first coefficient and a medium or high second coefficient (such as complex manual pump installation operations that require precise instructions, or complex self-test procedures for grouting equipment itself). Mode D: Forced Control Collaboration: The system has complete control over the collaborative equipment and needs to take over partial control or perform real-time intervention. This is suitable for projects with high first and second coefficients (such as forcing a collaborative device to stop in an emergency to protect grouting equipment, or accurately setting and monitoring key parameters of grouting equipment). The system searches for the collaborative control mode mapping based on the two coefficient values ​​of the current project and determines the most suitable collaborative control mode for the project.

[0076] Therefore, in this collaborative control mode, the collaborative control relationship between each collaborative equipment and the mining grouting equipment is collected, and the work progress of each collaborative equipment and the mining grouting equipment is determined based on the detection of the collaborative control relationship. The autonomous control logic of the work progress is determined according to the work progress of each collaborative equipment, the work progress of the mining grouting equipment and the corresponding remaining work content, and the work progress of each collaborative equipment and the mining grouting equipment are autonomously controlled based on the autonomous control logic. The overall consideration of the work progress of each collaborative equipment, the work progress of the mining grouting equipment and the corresponding remaining work content is compatible, ensuring the accuracy of the autonomous control logic of the work progress. At the same time, a collaborative control sequence table is introduced to realize the overall consideration of the collaborative control sequence table, each collaborative equipment and the mining grouting equipment, improve the accuracy of the collaborative control mode, and realize the autonomous control of the work progress of each collaborative equipment and the working progress of the mining grouting equipment.

[0077] At this point, the collaborative control relationship between each collaborative device and the mining grouting equipment is collected. The system needs to clarify the dependency relationship between each device based on the currently executed collaborative project (for example, CP-006: the truck transports the backup pump to the grouting equipment) and the selected collaborative control mode (for example, mode B: monitoring collaboration). Optionally, the movement and unloading of truck Truck-05 are key steps. The mining grouting equipment JZB-01 needs to wait for the truck to complete unloading before proceeding to the next step (CP-007: manual installation of the backup pump). The maintenance station system MS-System-C has completed the preparation of the backup pump (assuming this is a prerequisite and has been met). Manual personnel (although not explicitly modeled as "equipment", as executors, their availability is also part of the relationship) need to wait for the truck to arrive before starting installation.

[0078] The system needs to know how to obtain status information from each device. For example, Truck-05 uses onboard GPS and sensors to obtain its current location, speed, whether it has stopped, the status of the cargo door (open or not), and whether the cargo has been unloaded. Mining grouting equipment JZB-01 uses local sensors to obtain its current status (standby, waiting, faulty, ready to start, etc.). The maintenance station system MS-System-C uses a network interface to confirm the readiness of the backup pump. Human workers (using wearable devices or handheld terminals) can report status such as arrival at the site, commencement of installation preparations, and completion of installation (if the system interacts with personnel). The system internally establishes a data structure to represent these relationships, such as a simple graph model where nodes represent devices / tasks and edges represent dependencies (e.g., "Truck unloading completed" > "Grouting equipment allowed to begin installation").

[0079] The system continuously obtains real-time status data from each device through the interface defined in the previous step. The collected status data is mapped to the execution stage of the collaborative project (CP) and quantified into progress values ​​(for example, percentages or stage identifiers). For example: Truck-05: Status: Driving, 500 meters from the target point; > Progress: 50% (assuming total distance is 1000 meters); Status: Arrived, slowing down to stop; > Progress: 90%; Status: Stopped safely, cargo door opened; > Progress: 95%; Status: Stopped safely, cargo door opened, pump removed; > Progress: 100%; Mining grouting equipment JZB-01: Status: Standby, waiting for truck to arrive; > Progress: 0%; Status: Standby, truck has arrived and stopped; > Progress: 50% (waiting conditions have been met); Status: Standby, truck has unloaded; > Progress: 100% (conditions for entering the next stage have been met).

[0080] The system checks whether the equipment status satisfies the previously defined collaborative control relationship. For example, when it detects that the truck progress has reached 100% (unloaded), the system confirms this event and updates related dependencies (JZB-01 can proceed to the next step); optionally, it obtains GPS and status information from Truck-05 and calculates its progress in reaching the point specified by JZB-01; assuming the current progress is 75%; obtains the status from JZB-01 and confirms that it is still in the "waiting for truck arrival" state with a progress of 0% (for the CP-006 project, its own "waiting" progress is not counted, or it is defined as the state from the start of waiting to the satisfaction of the conditions); when the system detects that Truck-05's progress has reached 100% (unloaded), it confirms the key event of "truck unloading completed."

[0081] The system analyzes the current progress of each device, the remaining tasks to be performed (according to the collaborative control sequence table), and the currently selected collaborative control mode (the result of S152). It determines whether there are progress delays, resource conflicts, or potential risks. For example, if the progress of a certain device is far below expectations (such as a truck being delayed for some reason), multiple tasks require the same resource (such as only one installer), or the current progress deviates significantly from the time point planned in the sequence table, it generates control suggestions based on preset rules, such as: Acceleration strategy: If a task on a non-critical path is completed ahead of schedule, can resources be freed up for the critical path, and can manual preparation be notified? Waiting strategy: If a critical piece of equipment (truck) is delayed, should subsequent tasks (installation) be notified to pause to avoid ineffective waiting or resource occupation? Rescheduling strategy: If a serious delay occurs, whether the sequence list needs to be re-evaluated and the task priority or allocation needs to be adjusted; Compensation strategy: If a task is delayed, whether subsequent tasks can process more content in parallel.

[0082] Convert the above policy into executable logic rules, for example: "If the truck unloading progress is delayed for more than 15 minutes and the installer has arrived at the site, notify the installer to suspend preparation and wait for further instructions."

[0083] When a trigger condition is monitored (such as progress reaching a certain point or delay exceeding a threshold), the system automatically sends control instructions to related equipment or systems based on the control logic generated in the previous step; the content of the instructions varies according to the collaborative control mode and control requirements; including: status notification: notifying the device that it can start / pause / continue a task; parameter adjustment: adjusting equipment operating parameters (such as adjusting truck path planning to avoid congestion, if the system has this capability; or adjusting certain parameters of the grouting equipment self-test); resource coordination: requesting the release or allocation of resources (such as requesting the dispatch of another truck as a backup, or notifying personnel to change the order of work); priority change: temporarily increasing or lowering the priority of a task; closed-loop feedback: the system needs to receive the device's response to the control instruction and the new status after execution to form a closed-loop control to ensure effective control.

[0084] See also Figure 7 , Figure 7 : is a schematic diagram of the structural composition of a collaborative control system for mining grouting equipment based on multimodal data in an embodiment of the present invention; the collaborative control system for mining grouting equipment based on multimodal data includes: A work item sequence list module 21 is used to determine a work item sequence list of the mining grouting equipment based on the current working event of the mining grouting equipment and the current position of the mining grouting equipment, and mark the work progress of each work item; The remaining work content module 22 is used to determine the working state of the mining grouting equipment according to multiple working data of the mining grouting equipment, and determine multiple remaining work contents according to the working state, the work progress of each work project and the corresponding project content; A collaborative control content module 23 is configured to determine multiple collaborative positions based on the tracing of multiple remaining work contents, determine multimodal data based on each collaborative position, the multiple remaining work contents, and the working state of the mining grouting equipment, and determine corresponding collaborative control content based on the identification of the multimodal data; The collaborative control sequence table module 24 is used to determine the corresponding collaborative equipment according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and to determine the collaborative control sequence table based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment and each collaborative position; The collaborative control mode module 25 is used to determine the corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device and the mining grouting equipment, so as to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment.

[0085] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A collaborative control method for mining grouting equipment based on multimodal data, characterized in that: include: Determine a work item sequence list of the mining grouting equipment based on the current work event of the mining grouting equipment and the current position of the mining grouting equipment, and mark the work progress of each work item; Determine the working status of the mining grouting equipment according to a plurality of working data of the mining grouting equipment, and determine a plurality of remaining work contents according to the working status, the work progress of each work item and the corresponding project content; Determine multiple collaborative positions based on tracing multiple remaining work contents, determine multimodal data based on each collaborative position, multiple remaining work contents, and the working state of the mining grouting equipment, and determine corresponding collaborative control content based on the recognition of the multimodal data; Determine the corresponding collaborative equipment according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and determine the collaborative control sequence table based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment and each collaborative position; Based on the collaborative control sequence table, each collaborative device and the mining grouting equipment, a corresponding collaborative control mode is determined to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment.

2. The collaborative control method for mining grouting equipment based on multimodal data according to claim 1 is characterized in that: The method of determining a work item sequence list of the mining grouting equipment based on the current work event of the mining grouting equipment and the current position of the mining grouting equipment, and marking the work progress of each work item, includes: Collect the current location of the mining grouting equipment, determine multiple completed events and current execution content of the mining grouting equipment based on the tracing of the current location of the mining grouting equipment, and determine the current working event of the mining grouting equipment based on the multiple completed events and current execution content of the mining grouting equipment; Determine multiple current work items based on the analysis of the current work events of the mining grouting equipment, and determine a work item sequence list of the mining grouting equipment according to the synthesis of the contents and sequences of the multiple current work items; In the work item sequence table of the mining grouting equipment, the work progress of each work item is dynamically updated as each current work item is carried out, and the work track of the mining grouting equipment in each work item is recorded.

3. The collaborative control method for mining grouting equipment based on multimodal data according to claim 1, characterized in that: The method of determining the working state of the mining grouting equipment according to the plurality of working data of the mining grouting equipment, and determining the plurality of remaining work contents according to the working state, the work progress of each work item and the corresponding project content, includes: Real-time monitoring of the working process of the mining grouting equipment and collection of multiple working data of the mining grouting equipment, including grouting pressure, grouting speed, grouting path, grouting volume and stirring speed; Constructing corresponding working data combinations based on multiple working data and each working node of the mining grouting equipment, determining multiple working features at the same time node based on the identification of each working data combination, and determining the working state of the mining grouting equipment based on the detection of the multiple working features at the same time node; Monitor each work project in real time, collect the work progress of each work project, and mark the work content of each work project. Determine the unfinished work content of each work project based on the work progress of each work project and the corresponding project content. Determine multiple remaining work contents based on the unfinished work content of each work project and the working status of the mining grouting equipment.

4. The collaborative control method for mining grouting equipment based on multimodal data according to claim 1, characterized in that: The method of determining multiple collaborative positions based on tracing multiple remaining work contents, determining multimodal data according to each collaborative position, multiple remaining work contents, and the working state of the mining grouting equipment, and determining corresponding collaborative control content based on the identification of the multimodal data includes: Collect multiple remaining work contents, trace the multiple remaining work contents, determine multiple remaining work nodes based on the tracing of the multiple remaining work contents, and determine multiple coordinated positions based on the positions of the multiple remaining work nodes, the corresponding remaining work contents, and the current position of the mining grouting equipment; In each collaborative position, the first collaborative content of each collaborative position is determined based on the matching of each collaborative position and multiple remaining work contents, the second collaborative content of each collaborative position is determined based on the working status of each collaborative position and the mining grouting equipment, and multimodal data is determined based on the synthesis of each collaborative position, the corresponding first collaborative content and the second collaborative content.

5. The collaborative control method for mining grouting equipment based on multimodal data according to claim 4 is characterized in that: The method further includes determining multiple collaborative positions based on tracing multiple remaining work contents, determining multimodal data according to each collaborative position, the multiple remaining work contents, and the working state of the mining grouting equipment, and determining corresponding collaborative control content based on the identification of the multimodal data. In multimodal data, multiple collaborative paths are determined based on the autonomous identification of multimodal data, and the corresponding collaborative control content is marked for each collaborative path. At this time, the corresponding collaborative control content is constructed based on the synthesis of each collaborative path, each collaborative position and multiple remaining work contents.

6. The collaborative control method for mining grouting equipment based on multimodal data according to claim 1, characterized in that: The method of determining the corresponding collaborative equipment according to the collaborative control content and the functional tool corresponding to the mining grouting equipment, and determining the collaborative control sequence table based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment and each collaborative position includes: Collect collaborative control content, determine the corresponding collaborative action based on the analysis of the collaborative control content, and determine the corresponding collaborative function coefficient based on the collaborative action and the current working status of the mining grouting equipment; determine the corresponding collaborative equipment based on the collaborative function coefficient, the functional tools corresponding to the mining grouting equipment, and the mapping relationship between the collaborative equipment.

7. The collaborative control method for mining grouting equipment based on multimodal data according to claim 6, characterized in that: The method further includes determining corresponding collaborative devices according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and determining a collaborative control sequence table based on the current working status of each collaborative device, the current working status of the mining grouting equipment, and each collaborative position, and further includes: Determine a plurality of first collaborative control data based on the current working status of each collaborative device and the current working status of the mining grouting equipment; A plurality of second collaborative control data are determined according to the matching between the current working status of each collaborative device and each collaborative position, and a collaborative control sequence table is determined based on the synthesis of the plurality of first collaborative control data and the plurality of second collaborative control data.

8. The collaborative control method for mining grouting equipment based on multimodal data according to claim 1, characterized in that: The method of determining a corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device and the mining grouting equipment to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment includes: In the collaborative control sequence table, a plurality of collaborative items are determined based on parsing of the collaborative control sequence table, and a first collaborative control coefficient is determined according to the plurality of collaborative items and corresponding collaborative devices; A second collaborative control coefficient is determined according to multiple collaborative projects and mining grouting equipment, and a corresponding collaborative control mode is determined based on a mapping relationship among the first collaborative control coefficient, the second collaborative control coefficient and the collaborative control mode.

9. The collaborative control method for mining grouting equipment based on multimodal data according to claim 8, characterized in that: The method further includes determining a corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device, and the mining grouting equipment to autonomously control the working progress of each collaborative device and the working progress of the mining grouting equipment. In this collaborative control mode, the collaborative control relationship between each collaborative equipment and the mining grouting equipment is collected, and the working progress of each collaborative equipment and the working progress of the mining grouting equipment are determined based on the detection of the collaborative control relationship. The autonomous control logic of the work progress is determined according to the working progress of each collaborative equipment, the working progress of the mining grouting equipment and the corresponding remaining work content, and the working progress of each collaborative equipment and the working progress of the mining grouting equipment are autonomously controlled based on the autonomous control logic.

10. A collaborative control system for mining grouting equipment based on multimodal data, characterized in that: The collaborative control system for mining grouting equipment based on multimodal data is applied to the collaborative control method for mining grouting equipment based on multimodal data as described in any one of claims 1 to 9, and the collaborative control system for mining grouting equipment based on multimodal data includes: A work item sequence list module is used to determine a work item sequence list of the mining grouting equipment based on the current working event of the mining grouting equipment and the current position of the mining grouting equipment, and mark the work progress of each work item; The remaining work content module is used to determine the working status of the mining grouting equipment according to multiple working data of the mining grouting equipment, and determine multiple remaining work contents according to the working status, the work progress of each work project and the corresponding project content; A collaborative control content module is used to determine multiple collaborative positions based on the tracing of multiple remaining work contents, determine multimodal data according to each collaborative position, multiple remaining work contents and the working status of the mining grouting equipment, and determine corresponding collaborative control content based on the recognition of the multimodal data; A collaborative control sequence table module is used to determine the corresponding collaborative equipment according to the collaborative control content and the functional tools corresponding to the mining grouting equipment, and to determine the collaborative control sequence table based on the current working status of each collaborative equipment, the current working status of the mining grouting equipment, and each collaborative position; The collaborative control mode module is used to determine the corresponding collaborative control mode based on the collaborative control sequence table, each collaborative device and the mining grouting equipment, so as to autonomously regulate the working progress of each collaborative device and the working progress of the mining grouting equipment.

Images