A coal mine coal mining machine remote control method, system, device and medium

Abstract

The application discloses a kind of coal mine shearer remote control method, system, equipment and medium, belong to mining machinery control field, comprising: receiving the update parameter of the relative position relationship of camera device description, simultaneously real-time acquisition shearer operating state parameter;Dynamic decision updates the execution mode of parameter;Based on the transition parameter sequence of decision result generation;The continuous visual reference information required for generating shearer control, while the position instruction of shearer actuator is synchronously compensated, and in switching process, continuously monitor instruction execution deviation to implement exception handling.The application switches strategy by dynamic decision parameter and synchronously compensates actuator position instruction, eliminates the space-time dislocation of control instruction and visual feedback caused by parameter jump, while continuously monitoring the effectiveness of visual information and execution deviation in switching process, and falls back switching or adjusts control mode when abnormal, forms closed-loop safety mechanism, improves the accuracy and safety of shearer remote control.

Description

Technical Field

[0001] This invention relates to the field of mining machinery control technology, specifically to a remote control method, system, equipment, and medium for coal mining machines. Background Technology

[0002] Intelligent coal mining requires the transfer of underground workers to a remote control center on the ground. To achieve remote and precise control of the coal mining machine, the control system needs to acquire real-time information on the relative position of the machine and the coal face to guide adjustments to the drum cutting path and control of the machine's traction speed. Currently, multiple cameras deployed at the longwall face are typically used to collect images, which are then processed to generate visual perception information, serving as a crucial basis for coal mining machine control decisions.

[0003] However, during the cutting operation, the coal mining machine is continuously subjected to cutting load impacts and reciprocates along the working face track, causing real-time drift in the relative spatial position relationship between the cameras installed on the coal mining machine body and hydraulic supports. To ensure the accuracy of the visual perception information upon which the coal mining machine control system relies, the parameters describing the relative position relationship of the cameras need to be dynamically calibrated periodically.

[0004] Existing technologies typically perform parameter calibration by directly replacing old parameters, leading to abrupt changes in visual perception information. In this situation, the automatic cutting system of the coal mining machine still controls the drum position based on the perception information before the change, causing a spatiotemporal misalignment between the coal mining machine's control commands and the current visual perception information. This misalignment, especially during high-intensity operations such as high-speed cutting, can easily lead to uncontrolled drum penetration into the coal face or accidental cutting of the roof and floor, seriously threatening the safety and reliability of remote control of the coal mining machine.

[0005] In addition, the existing parameter calibration strategy uses a fixed threshold triggering method, which does not take the real-time operating conditions of the coal mining machine into the decision-making basis, and introduces unnecessary visual disturbances during high-risk operation phases. At the same time, the calibration process lacks a closed-loop monitoring mechanism for the deviation between the actual position and the commanded position of the coal mining machine drum, and cannot form an adaptive adjustment capability that is deeply integrated with the coal mining machine control.

[0006] Therefore, how to eliminate the spatiotemporal misalignment between the control commands of the coal mining machine and the visual perception information during the dynamic calibration of the relative position parameters of the camera device is a core technical problem that urgently needs to be solved in the field of remote control of coal mining machines. Summary of the Invention

[0007] In view of the above-mentioned problems, the present invention is proposed.

[0008] Therefore, the technical problem solved by this invention is: how to eliminate the spatiotemporal misalignment between the control commands and visual feedback of the coal mining machine caused by the jump in visual reference information during the dynamic calibration of parameters, thereby improving the accuracy and safety of the remote control of the coal mining machine.

[0009] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a remote control method for a coal mining machine, comprising, It receives updated parameters describing the relative positional relationship of the camera devices, and simultaneously collects the operating status parameters of the coal mining machine in real time; The execution method for updating parameters is dynamically determined based on the collected operational status parameters; Generate a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; The system generates continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, it synchronously compensates the position commands of the coal mining machine actuator according to the transition parameter sequence and continuously monitors command execution deviations during the switching process to implement anomaly handling.

[0010] In a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the method of receiving updated parameters describing the relative positional relationship of the camera devices includes... Receive updated parameters output after processing images acquired in real time from multiple cameras; The calculation is used to determine the relative positional relationship between multiple cameras that changes due to the vibration or movement of the coal mining machine, while recording the old parameters currently in use; The operating status parameters of the coal mining machine are acquired in real time at a set sampling period, and the operating condition characteristics of the current coal mining machine are analyzed based on the acquired operating status parameters.

[0011] As a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the execution method of dynamically deciding and updating parameters based on the collected operating status parameters includes: Based on the analysis of the current operating condition characteristics, the stability of the operating condition and the criticality of the task are determined, and the timing for switching decision parameters is determined. The length of the transition time used during the decision-making and parameter update process depends on the level of mission criticality. Based on the traction speed characteristics in the operating status parameters, the decision is made regarding the type of transition curve to use during the parameter switching process.

[0012] This invention achieves adaptive matching between the visual parameter switching strategy and the real-time operating conditions of the coal mining machine in remote control. It determines the execution timing based on the stability of the operating conditions and the criticality of the task, the transition time based on the level of task criticality, and the type of transition curve based on traction speed characteristics. In high-criticality operation phases (such as automatic cutting at high speed), delayed execution with a longer transition time and a soft-start curve prioritizes the stability of the visual reference information upon which the coal mining machine control relies, preventing drum malfunctions due to sudden visual changes. In low-criticality phases (such as no-load or shutdown), immediate execution with a shorter transition time and a linear curve quickly updates parameters, reducing the waiting time of the coal mining machine control system. This adaptive mechanism ensures that the visual parameter switching process meets the control requirements of different operating scenarios of the coal mining machine while avoiding the safety hazards introduced by traditional fixed threshold methods in high-risk phases, thus improving the reliability and efficiency of remote control of the coal mining machine.

[0013] In a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the step of generating a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results includes: The number of transition steps is determined based on the synchronization signal frequency of the current coal mining machine control system and the transition time obtained from the decision. For each synchronization cycle, the transition coefficient corresponding to the current cycle is determined based on the progress position of the current cycle in the total number of transition steps and the transition curve type selected by the decision. The transition parameters for the current period are obtained by performing transition calculations on the old and updated parameters based on the transition coefficient. The transition parameters corresponding to each cycle are stored in cycle order.

[0014] In a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the continuous visual reference information required for generating the coal mining machine control includes: The transition parameters corresponding to the current cycle are obtained from the stored transition parameter sequence and sent to the vision processing unit; The vision processing unit processes the multi-camera images based on the received transition parameters to generate visual reference information for the current period.

[0015] In a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the step of synchronously compensating for the position commands of the coal mining machine's actuator includes: When issuing transition parameters, analyze the changing trend between the old and updated parameters; During the transmission of the transition parameter sequence, the target position calculation results of the coal mining machine actuator are compensated according to the changing trend and the time offset from the start of transmission to the current cycle, so that the output position command is consistent with the visual reference information corresponding to the current cycle.

[0016] This invention analyzes the changing trends between old and updated parameters when transition parameters are issued, and compensates for the target position of the coal mining machine actuator based on the changing trends and time offset during the transition process. This ensures that the output position command is consistent with the visual reference information corresponding to the current cycle. This fundamentally eliminates the spatiotemporal misalignment between coal mining machine control commands and visual feedback caused by visual parameter switching. It allows the automatic cutting system to accurately control the coal mining machine drum position based on real and continuous visual reference information during parameter switching, avoiding drum malfunctions caused by visual jumps. Simultaneously, the compensation mechanism deeply couples visual perception with coal mining machine control, providing reliable command synchronization for remote control of the coal mining machine and improving the accuracy and safety of coal mining machine cutting operations.

[0017] As a preferred embodiment of the remote control method for a coal mining machine according to the present invention, the step of continuously monitoring command execution deviations during the switching process to implement anomaly handling includes, During the transition parameter sequence issuance process, the effectiveness of the visual reference information on which the coal mining machine control relies is continuously evaluated. When the effectiveness is lower than the set requirements, the switching process is paused and the old parameters are restored. Simultaneously, the deviation between the actual position and the commanded position of the coal mining machine actuator is continuously monitored. When the deviation exceeds the safe range, the control mode of the coal mining machine is adjusted and the switching process is terminated.

[0018] This invention provides a remote control system for a coal mining machine.

[0019] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a remote control system for a coal mining machine, comprising: a data collection module, a switching execution module, an update module, and a dynamic compensation module; The data collection module receives updated parameters describing the relative positional relationship of the camera devices, and simultaneously collects the operating status parameters of the coal mining machine in real time. The switching execution module dynamically decides the execution method for updating parameters based on the collected running status parameters; The update module generates a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; The dynamic compensation module generates continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, it synchronously compensates the position commands of the coal mining machine actuator according to the transition parameter sequence, and continuously monitors command execution deviations during the switching process to implement anomaly handling.

[0020] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the remote control method for a coal mining machine.

[0021] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the steps of the remote control method for a coal mining machine.

[0022] The beneficial effects of this invention are as follows: This invention designs the switching process of the camera device's sensing parameters as a continuous transition process scheduled by the coal mining machine control system, and synchronously compensates for the position command of the coal mining machine drum during the switching process, so that the output control command is always consistent with the current visual sensing information. This fundamentally eliminates the problem of spatiotemporal misalignment between control command and visual feedback caused by the switching of sensing parameters, avoids drum malfunction, and improves the accuracy of remote control of the coal mining machine.

[0023] This invention acquires the traction speed, vibration status, and working mode of the coal mining machine in real time, and dynamically decides the timing and transition method for switching sensing parameters. During high-intensity operation stages such as high-speed cutting, it prioritizes the stability of visual perception, and quickly completes the switching during low-intensity operation stages such as no-load operation, thus balancing the safety and efficiency of remote control of the coal mining machine.

[0024] This invention continuously monitors the deviation between the actual position and the commanded position of the coal mining machine drum during the switching of sensing parameters, and promptly reverses the switching, terminates the action, or switches the control mode when the deviation exceeds the standard, ensuring that the remote control of the coal mining machine is always in a controllable state and eliminating safety accidents caused by abnormal sensing information or control deviation. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a general flowchart of a remote control method for a coal mining machine provided in one embodiment of the present invention.

[0027] Figure 2 This is a flowchart illustrating the execution method of dynamic decision-making and parameter updating in a remote control method for a coal mining machine according to an embodiment of the present invention.

[0028] Figure 3This is a flowchart illustrating the synchronous compensation process of a remote control method for a coal mining machine, provided as an embodiment of the present invention. Detailed Implementation

[0029] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0030] Example 1, referring to Figure 1 This is one embodiment of the present invention, which provides a remote control method for a coal mining machine, comprising: S1. Receive updated parameters describing the relative positional relationship of the camera devices, and simultaneously collect the operating status parameters of the coal mining machine in real time; S2. Dynamically decide on the execution method for updating parameters based on the collected operating status parameters; S3. Generate a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; S4. Generate continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, perform synchronous compensation on the position commands of the coal mining machine actuator according to the transition parameter sequence, and continuously monitor command execution deviations during the switching process to implement anomaly handling.

[0031] Example 2, refer to Figures 2-3 As an embodiment of the present invention, a remote control method for a coal mining machine is provided based on the previous embodiment, comprising: Step S1 involves receiving updated parameters describing the relative positional relationship of the camera devices and simultaneously acquiring real-time operating status parameters of the coal mining machine, including the following steps: S11. Receive updated parameters output after solving images acquired in real time by multiple cameras; the solution is used to determine the relative positional relationship between multiple cameras that has changed due to the vibration or movement of the coal mining machine, and at the same time record the old parameters currently in use.

[0032] Specifically, during the advance of the coal mining machine along the working face, the relative positions of the multiple camera devices installed on the machine body and hydraulic supports will drift in real time due to machine vibration and movement. To ensure the accuracy of the visual reference information, the parameters need to be dynamically calibrated periodically. In this embodiment, the parameter calibration unit performs parameter calculation based on images acquired in real time by multiple cameras.

[0033] Suppose a multi-camera system contains V cameras, denoted as V. .

[0034] in, Let i be the identifier of the i-th camera. This represents the total number of cameras in a multi-camera system.

[0035] At some point Each camera acquires one cycle of images. , Let be the two-dimensional image acquired by the i-th camera at time t. For coal mine images acquired underground, feature points mainly originate from the natural textures on the coal face, the edges and corners of the hydraulic supports, the regular structure of the scraper conveyor, and the inherent structural edges of the coal mining machine body.

[0036] Furthermore, well-known image feature extraction methods (Scale-Invariant Feature Transform, such as SIFT) are employed to detect feature points in the image and generate corresponding feature description vectors. For the image The extracted feature point set is denoted as: , , in, The image acquired by the i-th camera at time t The extracted feature point set, each element of which consists of the feature point pixel coordinates and a description vector. For the i-th camera, the i-th... Feature points in the image pixel coordinates in Let be the feature description vector corresponding to the u-th feature point in the i-th camera image. For feature point indexing, within the same image, , For the image of the i-th camera at time t The number of feature points extracted is a positive integer. , These represent the image column coordinates (horizontal direction) and image row coordinates (vertical direction) of the feature points, respectively, in pixels. This is a transpose.

[0037] Furthermore, for any pair of adjacent cameras and ,Will and Feature points in the image are matched. The matching process uses the Euclidean distance between feature description vectors as a similarity measure, and a nearest neighbor distance ratio strategy is used to select reliable matching point pairs. The camera... and The set of all matching point pairs that satisfy the condition is denoted as: , in, For adjacent cameras at time t and The set of feature point pairs that are successfully matched, each element being a pair of pixel coordinates. This indicates the same physical point observed by two cameras. To match point pairs indices, between the same pair of cameras.

[0038] These matching point pairs correspond to the projection points of the same physical location in the downhole scene in different camera images.

[0039] Further analysis of the set of matching point pairs The camera was solved using a parameter-solving method known in the art. Compared to a camera rotation matrix Translation vector This forms the parameters, expressed as: , in, It is a time camera Compared to a camera The rotation matrix is ​​a 3×3 orthogonal matrix. It is a time camera Compared to a camera The translation vector is a three-dimensional column vector. It is a time camera Compared to a camera The parameters represent those from the camera. Coordinate system to camera Coordinate system transformation.

[0040] Specifically, using cameras As a benchmark, for any camera Its parameters relative to the reference camera can be obtained through chain multiplication: , in, It is a time camera Relative to the reference camera The parameters, For camera indexing.

[0041] The set of parameters for all cameras relative to the reference camera is denoted as: , in, At time t, all cameras relative to the reference camera The set of parameters, It is a time camera Relative to the reference camera parameters Further, the current moment The set of parameters obtained by solving Compared to the previous moment set Comparison. For any camera. Calculate the translational change: , in, The time interval represents the difference between two consecutive parameter calculations. For camera The translation vector relative to the reference at time... Compared to the previous moment The change in quantity.

[0042] If there exists any camera such that (For example If the parameter value (in meters) changes significantly, it is considered a significant change, triggering a parameter update. At this point, the parameter calibration unit outputs the updated parameter value. , representing the relative pose between adjacent key cameras.

[0043] in, The threshold value for translational change is derived from field experiments and is preset to a positive real number. These are the updated parameters.

[0044] Further in receiving Previously, the control system stored old parameters that were currently being used to generate visual reference information. (i.e., the one used in the previous moment) When the control system receives through the industrial control network... Record the time of this reception as [time]. and will Update to the previously saved value.

[0045] in, The old parameters currently in use, i.e., those used in the previous time step. ). For the control system to receive updated parameters The moment is used as the starting moment for subsequent sampling.

[0046] S12. Obtain the operating status parameters of the coal mining machine in real time according to the set sampling period, and analyze the current working condition characteristics of the coal mining machine based on the obtained operating status parameters.

[0047] It should be noted that the set acquisition strategy refers to the process by which the control system, based on the pre-set sampling period, acquisition parameter type, and acquisition method, cyclically acquires data from the coal mining machine control system via the industrial fieldbus.

[0048] Specifically, the control system collects the operating status parameters of the coal mining machine in real time via an industrial fieldbus. The sampling period is set to [missing information]. (For example At any moment Collect the following parameters: At any moment The traction speed of the coal mining machine. Coal mining machine body vibration acceleration Current control mode of coal mining machine , is a discrete value, and its value can be set to automatic truncation, remote manual, or no-load.

[0049] in, The sampling sequence number is a non-negative integer. , For the first The time of the next sampling.

[0050] Based on the collected parameters, the control system further analyzes the characteristic information of the current operating condition, specifically including: (1) The root mean square value of the sliding window of vibration acceleration is used to characterize the stability, and the stability index of the working condition is calculated. The expression is: , in, The length of the sliding window (e.g., taking...) (corresponding to a 0.5-second window) At any moment The calculated stability index is defined as the root mean square value of the vibration acceleration within the sliding window; the smaller the value, the more stable the operating condition. For the past The acceleration value at each sampling time. This is the offset index of the sampling point within the window.

[0051] (2) Based on the traction speed and the current control mode, the mission criticality index is determined. The expression is: , in, For at any time The determined task criticality index has values ​​of 3, 2, and 1, representing high, medium, and low criticality, respectively.

[0052] The above and This will serve as the basis for switching the decision-making parameters in subsequent steps. Simultaneously, all collected raw parameters and calculated feature indicators are temporarily stored in the shared memory of the control system in time-series format for later use.

[0053] It should be noted that existing technologies use fixed thresholds to trigger parameter calibration, without considering the real-time operating conditions of the coal mining machine. This invention dynamically decides on the execution method based on the collected operating status parameters, achieving adaptive matching between the switching strategy and the operating conditions. This breakthrough is reflected in three aspects: First, the decision-making basis is expanded from a single parameter (translation change) to multi-dimensional operating condition characteristics (stability, criticality, traction speed); second, the decision-making objective is expanded from whether to switch to a full-process planning of when to switch, for how long to switch, and how to switch; third, the decision logic is optimized from hard threshold judgment to soft decision fusion.

[0054] In existing technologies, visual reference information generation and the coal mining machine control system are disconnected. This invention, while generating transitional visual reference information, dynamically compensates for control commands, ensuring that the output position command remains consistent with the visual coordinate system corresponding to the current cycle. This upgrades visual feedback from passive display to active participation in the control loop, achieving deep coupling between the vision system and the control system.

[0055] In step S2, the execution method for dynamically updating parameters is determined based on the collected operating status parameters, including the following steps: S21. Based on the analysis of the current operating condition characteristics, the stability of the operating condition and the criticality of the task, determine the timing for switching the decision parameters.

[0056] Specifically, the control system scans the latest operating condition characteristic information in the shared memory at fixed intervals (e.g., every 10ms). Let the current time be... The corresponding level of stability is The criticality is .

[0057] The decision-making rules for when to execute are as follows: like (Highly critical) and (For example This indicates that the coal mining machine is in automatic cutting mode and vibrating violently. Performing a parameter switch at this time could cause screen jitter or control malfunction; therefore, it is determined to be a delayed execution. The control system adds this switch request to the delay queue and records the request time. .

[0058] Otherwise, if (Medium to low criticality) and (For example ), or although but If so, it is determined to be executed immediately.

[0059] in, The high threshold for the stability of the operating conditions is a positive real number set based on experience. The low threshold for the stability of the operating conditions is a positive real number set based on experience. Record the moment when the decision is to delay execution.

[0060] Furthermore, regarding requests for delayed execution, the control system continuously monitors the operating conditions at subsequent moments, and at a certain moment... Immediate execution is triggered when any of the following conditions are met: Drop to 2 or below, and .

[0061] Delay time exceeds maximum waiting time (For example (seconds), at this point, to ensure timely parameter updates, immediate execution is forced.

[0062] in, This refers to the subsequent sampling time that the control system continuously monitors during the delay period.

[0063] S22. The length of the transition time used during the switching of decision update parameters is determined according to the level of mission criticality.

[0064] Specifically, transition time (Unit: seconds) refers to the time it takes to gradually transition from the old parameters to the new parameters. The value is determined based on the criticality of the task. (For delayed execution, the criticality of the trigger time is used) and determined according to the following rules: , in, It is a positive number, and the specific value can be set according to the actual working conditions of the mine. For example, take... Second, Second, The more critical the situation, the longer the transition time, in order to ensure that the screen changes smoothly enough and avoid interference with the operator or automatic control. The transition time for switching to the new parameters obtained from the decision. These correspond to high, medium, and low criticality levels, respectively.

[0065] S23. Based on the traction speed characteristics in the operating status parameters, decide the type of transition curve to be used during the switching process of update parameters.

[0066] Specifically, the type of transition curve determines the pattern of change of the transition coefficient with the cycle number during the transition process. In this embodiment, based on the traction speed... (For delayed execution, the speed at the trigger point is used) Decision curve type: like (For example This indicates that the coal mining machine is operating at a higher speed. At this speed, the smoothness of the image transitions is crucial, so an S-curve (also known as a slow start-stop curve) is selected. The transition coefficient of the S-curve... With normalized time The relationship is: , The curve changes slowly at the beginning and end, and changes more rapidly in the middle stage, which can effectively avoid sudden changes in the image.

[0067] in, For at any time The traction speed of the coal mining machine is collected. The traction speed threshold is used to distinguish between high-speed and low-speed operating conditions and is a positive real number set based on experiments.

[0068] like This indicates that the coal mining machine is running at low speed or stationary. In this case, the smoothness requirement is relatively low, and a linear curve can be selected, i.e.: , in, The transition coefficient represents the mixing ratio of parameters in the current cycle. It is a linear transition function. It is an S-shaped transition function. The normalized progress indicates the relative position of the current cycle within the total number of transition steps.

[0069] Linear curves are simple to implement, have low computational overhead, and are sufficient to meet visual continuity requirements under low-speed conditions.

[0070] Further execution timing (immediate or delayed), transition time Transition curve types These parameters will be uniformly recorded and passed to subsequent steps to generate a sequence of transition parameters. If the execution is delayed, the control system will automatically trigger subsequent steps after the conditions are met; if the execution is immediate, it will directly proceed to step S3.

[0071] This invention determines the execution timing based on the stability of the operating conditions and the criticality of the task, ensuring that parameter switching only begins within a safe window. This fundamentally eliminates the visual disturbances and safety hazards caused by forced switching during high-critical operation phases in traditional fixed-threshold triggering methods. Secondly, by dynamically adjusting the transition time according to the task's criticality, it achieves differentiated control, prioritizing visual stability in high-risk scenarios and switching efficiency in low-risk scenarios, thus balancing safety and system response speed. Furthermore, by dynamically selecting the transition curve type based on traction speed characteristics, it matches the visual transition characteristics to the coal mining machine's motion state. An S-shaped curve is used at high speeds to avoid abrupt visual changes, while a linear curve is used at low speeds to reduce computational overhead, achieving synergistic optimization of visual smoothness and computational efficiency.

[0072] In step S3, a transition parameter sequence from the currently used old parameters to the updated parameters is generated based on the decision results, including the following steps: S31. Determine the number of transition steps based on the synchronization signal frequency of the current coal mining machine control system and the transition time obtained from the decision.

[0073] Specifically, the control system obtains the synchronization signal frequency of the current coal mining machine control system. (Unit: cycles / second, for example) The transition time obtained from the S2 decision. The total number of transition steps required to transition from the old parameters to the new parameters is calculated using the following expression: , in, This indicates rounding up to the nearest integer, ensuring the transition time is at least [time value missing]. , This represents the total number of transition steps required to transition from the old parameters to the new parameters. Then take This is to ensure that there is at least one intermediate transition period.

[0074] S32. For each synchronization cycle, determine the transition coefficient corresponding to the current cycle based on the progress position of the current cycle in the total number of transition steps and the transition curve type selected by the decision.

[0075] Specifically, a synchronization cycle is a transition, for the first... cycle The control system first calculates the normalized progress of this cycle in the total number of transition steps. : , in, Indicates the starting point of the transition. Indicates the end point of the transition. This refers to the cycle number during the transition process.

[0076] Further, based on the transition curve type of the decision in S2 Calculate the transition coefficient corresponding to the current cycle. .

[0077] If an S-shaped curve is chosen, then: , The curve in the initial stage ( (approaching 0) and the end phase ( When the temperature approaches 1, the change is slow, while the change is faster in the middle stage, making it suitable for high-speed operating conditions.

[0078] If a linear curve is selected, then: , The curve maintains a constant rate of change throughout the transition process, is simple to implement, and is suitable for low-speed or stationary conditions.

[0079] No. Transition coefficient corresponding to the period The range of values ​​is ,in This indicates that cycle 0 (transition start cycle) uses all old parameters. To splice, Indicates the first The cycle (transition end cycle) fully utilizes the updated parameters. Then, the parts are assembled.

[0080] S33. Perform transition calculations on the old and updated parameters based on the transition coefficients to obtain the transition parameters for the current cycle.

[0081] Specifically, the control system will use the old parameters and update parameters Decomposed into rotation matrices and translation vectors respectively: , in, From the old parameters respectively The rotation matrix and translation vector in the decomposition. respectively from update parameters The rotation matrix and translation vector are decomposed into two parts, where the rotation matrix describes the rotation relationship between the cameras and the translation vector describes the translation relationship between the cameras.

[0082] For the Period, based on the obtained transition coefficient Transition operations are performed on the translation vector and rotation matrix respectively: (1) Translation vector transition: A linear transition is used, and the expression is: , in, It is the first Cycle, for and The translation vector obtained after a linear transition.

[0083] (2) Rotation matrix transition: adopt spherical linear transition.

[0084] First, convert the rotation matrix into a quaternion representation. Let... and They are respectively and The corresponding quaternion, and satisfying (If the dot product is negative, then take) To ensure the shortest transition path. Calculate the angle between two quaternions, expressed as: , in, The angle between two quaternions.

[0085] It should be noted that when using quaternions to represent rotations, and This represents a rotation within the same space (because the negative sign is canceled out when calculating the rotation matrix from quaternions). However, in a linear transition on a sphere, if the dot product of two quaternions is negative, it means they are in opposite directions on the four-dimensional sphere. A direct transition would take a longer route (a longer arc), resulting in a non-shortest rotation path. To ensure the transition follows the shortest path, it is necessary to ensure... If the dot product is negative, one of the quaternions is inverted, making the dot product positive and the transition path shortest. However, the inverted quaternion still represents the same rotation.

[0086] Furthermore, the first Quaternions after periodic transition The expression is: , The first Quaternions after periodic transition Normalized and converted back to rotation matrix The conversion method is well known in the field.

[0087] Further The transition parameter of the period is expressed as: , in, For the first Periodic transition parameters.

[0088] S34. Store the transition parameters corresponding to each cycle in cycle order.

[0089] Specifically, the control system will calculate the... The data is stored sequentially into a pre-allocated transition buffer based on its period number, from smallest to largest. This buffer uses a circular queue structure and has a minimum capacity of [missing information]. This is to ensure that the cycle number can be read correctly in subsequent steps.

[0090] At the same time, the control system initializes the periodic index counter. This is used to indicate the current cycle number to be sent to the vision processing unit. When subsequent steps process a cycle of synchronization signals, they will read this information from the buffer. and will Increment by 1 until... This indicates that the transition sequence has been fully distributed. It is a periodic index.

[0091] At this point, the control system has generated a complete sequence of transition parameters and is ready for subsequent steps to call cycle by cycle.

[0092] This invention accurately calculates the number of transition steps based on the transition time and the synchronization signal frequency of the coal mining machine control system, ensuring that the transition process matches the video output cycle rate and laying the foundation for subsequent cycle synchronization. Secondly, it generates periodically changing transition coefficients based on the transition curve type of the decision, so that the parameters evolve continuously in the transition sequence. Among them, the S-shaped curve effectively suppresses sudden changes in the picture under high-speed conditions, while the linear curve maintains computational efficiency under low-speed conditions. Furthermore, by using spherical linear transition (SLERP) on the rotation matrix and linear transition on the translation vector, the physical rationality and mathematical rigor of the parameter changes are ensured, avoiding image distortion caused by improper transition. The generated transition parameters are stored in a buffer according to the cycle number.

[0093] In step S4, continuous visual reference information required for the control of the coal mining machine is generated based on the transition parameter sequence. Simultaneously, position commands of the coal mining machine actuator are synchronously compensated according to the transition parameter sequence, and command execution deviations are continuously monitored during the switching process to implement anomaly handling. This includes the following steps: S41. Generate continuous visual reference information required for coal mining machine control based on the transition parameter sequence.

[0094] S411. Obtain the transition parameters corresponding to the current cycle from the stored transition parameter sequence and send them to the vision processing unit.

[0095] Specifically, the vision processing unit generates synchronization signals for the coal mining machine control system at a fixed frequency. The control system listens to these synchronization signals and performs the following operations each time it receives a synchronization signal: Read the current cycle index from the transition buffer. Corresponding transition parameters The data is then transmitted to the vision processing unit via a real-time industrial communication interface (Scale-Invariant Feature Transform, such as using the UDP protocol). After transmission, the control system uses a periodic index counter. Increment by 1.

[0096] S412, The vision processing unit processes the multi-camera images based on the received transition parameters to generate visual reference information for the current period.

[0097] Specifically, the vision processing unit receives... Then, this is used as the parameter for synthesizing visual information for the current period. For multiple cameras at different times... Raw images acquired The visual processing unit according to Each image is projected onto a unified visual coordinate system, and visual information is synthesized to generate visual reference information for the current period. This visual reference information is displayed in real time at the remote operating station and can be accessed.

[0098] Furthermore, once the transition steps are completed, the vision processing unit is instructed to directly use the updated parameters to generate visual reference information.

[0099] Specifically, the control system will periodically index the counter. Increasing .

[0100] Further judgment Is it greater than : like Then repeat the above process and wait for the next synchronization signal.

[0101] like This indicates that the entire transition sequence has been issued, at which point the control system sends a switching completion command to the vision processing unit. Upon receiving this command, the vision processing unit directly uses the updated parameters in subsequent cycles. Visual reference information is generated, and no further processing is performed.

[0102] S42. Synchronize and compensate for the position commands of the coal mining machine actuator.

[0103] S421. When issuing transition parameters, analyze the changing trend between the old parameters and the updated parameters.

[0104] Specifically, during the issuance of the transition parameter sequence, the control system simultaneously performs dynamic compensation for the control commands of the coal mining machine actuator to ensure that the control commands remain consistent with the current visual coordinate system.

[0105] At the moment when the transition parameter sequence begins to be issued (i.e., the first issuance) (At the time), the control system uses old parameters and update parameters Calculate the trend of change, including translational and rotational changes.

[0106] The expression for calculating translational changes is: , in, This is the translation vector.

[0107] The relative rotation matrix is ​​calculated using the following expression: , in, It is a relative rotation matrix.

[0108] It should be further explained that, for rotational transformations, the difference of the rotation matrix is ​​converted into a rotational angular velocity vector. .

[0109] Specifically, will Convert to a rotating axis (Unit vector) and rotation angle Then the rate of change of rotation (angular velocity) is: , in, This is the transition period for decision-making.

[0110] S422. During the transmission of the transition parameter sequence, the target position calculation result of the coal mining machine actuator is compensated according to the changing trend and the time offset from the start of transmission to the current cycle, so that the output position command is consistent with the visual reference information corresponding to the current cycle.

[0111] Specifically, during the transition process, for the current period index... (i.e., the number of cycles successfully delivered to the visual processing unit, initial) ), calculate the time offset from the start of distribution to the current time, the expression is: , in, The synchronization signal frequency for the coal mining machine control system. For the first The time offset of the period.

[0112] Furthermore, when calculating the target position of the coal mining machine's actuator (such as the drum), the control system first calculates the target position vector in the old parameter coordinate system based on coal and rock identification or a preset cutting trajectory. (Three-dimensional coordinates relative to the reference camera).

[0113] Let the transition parameter corresponding to the current period be... (i.e., the parameters issued in the previous cycle), the goal of compensation is to make the output command consistent with the visual coordinate system defined by these parameters, so compensation amounts need to be added, expressed as: , in, The spatial position vector of the actuator (drum center) relative to the reference camera (pre-calibrated in the old parametric coordinate system). To ensure the final command is sent to the coal mining machine's actuator, the output position is consistent with the visual coordinate system of the current cycle. If the compensated position command exceeds a preset safety limit, an exception handling is triggered. This compensation ensures the output position command is based on the true visual coordinate system corresponding to the current cycle, guaranteeing synchronization between control commands and visual feedback.

[0114] S43. During the switching process, continuously monitor instruction execution deviations to implement exception handling.

[0115] S431. During the distribution of transition parameter sequences, continuously evaluate the effectiveness of the visual reference information on which the coal mining machine control depends. When the effectiveness is lower than the set requirements, pause the switching process and restore the use of the old parameters.

[0116] Specifically, for the visual reference information generated in each cycle The control system evaluates its quality.

[0117] Using the overlapping area of ​​adjacent cameras as the evaluation object, the normalized cross-correlation (NCC) index is used to calculate the degree of matching, and the expression is: , in, For the degree of matching, For adjacent cameras and The set of pixels in the overlapping area after stitching. For camera Image pixels in overlapping areas grayscale value at that location For camera In the image Corresponding pixel coordinates grayscale value at that location For camera Image in overlapping area The average grayscale value of the pixels within the range. For camera Image in overlapping area The average grayscale value of the pixels within the range.

[0118] Furthermore, if continuous Period (e.g.) If the NCC values ​​of all components are lower than the preset matching threshold NCC1 (the preset matching threshold is a parameter set through testing, for example, NCC1=0.85), then the splicing quality is deemed abnormal. At this point, the control system immediately executes: Pause the issuance of subsequent transition parameters and discard unused transition parameters in the buffer; send a rollback command to the vision processing unit to resume the use of the old parameters. The system is spliced ​​together, and an audible and visual alarm is issued at the remote operation station to alert the operator that the switching process is abnormal.

[0119] S432. Simultaneously monitor the deviation between the actual position and the commanded position of the coal mining machine actuator. When the deviation exceeds the safe range, adjust the control mode of the coal mining machine and terminate the switching process.

[0120] Specifically, the control system collects the actual position of the coal mining machine's actuators at fixed intervals. This position is obtained through sensors such as encoders, laser rangefinders, or UWB positioning. Simultaneously, the position of the command currently issued to the actuator is recorded. (i.e., the target position output after compensation) ).

[0121] The Euclidean norm of the positional deviation is calculated as follows: , If continuous Each sampling period (e.g.) The deviation value e of all values ​​is greater than the preset safety threshold. (For example If the command execution deviation exceeds the limit, the control system will then execute: Immediately terminate the current switching process and stop issuing subsequent transition parameters; forcibly switch the coal mining machine control mode to local manual mode and issue an audible and visual alarm; record the operating data at the time of the anomaly to the historical database for subsequent analysis.

[0122] If the switchover process is completed successfully (i.e., all applications are successfully distributed) (If the periodic transition parameters are not triggered and no abnormalities are found), the control system will mark the status as completed and will then operate in normal mode.

[0123] This invention uses the synchronization signal of the coal mining machine control system as a trigger to issue transition parameters cycle by cycle, ensuring that the visual reference information generated by the vision processing unit in each cycle strictly corresponds to the current transition parameter. This achieves cycle-level synchronization between parameter switching and the control system synchronization signal, ensuring the continuity and real-time nature of the visual reference information transition.

[0124] During the transition process, the parameter change trend is dynamically analyzed, and the target position of the coal mining machine actuator is compensated in real time according to the time offset, so that the output position command is always consistent with the visual coordinate system of the current cycle. This completely eliminates the spatiotemporal misalignment between control commands and visual feedback caused by parameter switching and avoids the risk of drum malfunction.

[0125] This invention continuously monitors the quality of visual reference information and the positional deviation of the actuator throughout the entire process. When the quality indicators continue to decline or the deviation exceeds the standard, it promptly pauses the switching, rolls back the old parameters, or adjusts the control mode, forming a closed-loop safety mechanism to ensure the safety of equipment and personnel under any abnormal circumstances. By integrating visual perception, control execution, and safety monitoring, the parameter switching process possesses high reliability and fault tolerance.

[0126] Example 3 is an embodiment of the present invention, which provides a remote control system for a coal mining machine, including: a data collection module, a switching execution module, an update module, and a dynamic compensation module; The data collection module receives updated parameters describing the relative positional relationship of the camera devices, and simultaneously collects the operating status parameters of the coal mining machine in real time. The switching execution module dynamically decides the execution method for updating parameters based on the collected running status parameters; The update module generates a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; The dynamic compensation module generates continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, it synchronously compensates the position commands of the coal mining machine actuator according to the transition parameter sequence, and continuously monitors command execution deviations during the switching process to implement anomaly handling.

[0127] This embodiment also provides an electronic device applicable to a remote control method for a coal mining machine, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the remote control method for a coal mining machine as proposed in the above embodiment.

[0128] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, it implements a remote control method for a coal mining machine as proposed in the above embodiment.

[0129] The storage medium proposed in this embodiment belongs to the same inventive concept as the remote control method for a coal mining machine proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0130] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0131] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A remote control method for a coal mining machine, characterized in that: include, It receives updated parameters describing the relative positional relationship of the camera devices, and simultaneously collects the operating status parameters of the coal mining machine in real time; The execution method for updating parameters is dynamically determined based on the collected operational status parameters; Generate a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; The system generates continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, it synchronously compensates the position commands of the coal mining machine actuator according to the transition parameter sequence and continuously monitors command execution deviations during the switching process to implement anomaly handling.

2. The remote control method for a coal mining machine as described in claim 1, characterized in that: The received updated parameters describing the relative positional relationships of the camera devices include, Receive updated parameters output after processing images acquired in real time from multiple cameras; The calculation is used to determine the relative positional relationship between multiple cameras that changes due to the vibration or movement of the coal mining machine, while recording the old parameters currently in use; The operating status parameters of the coal mining machine are acquired in real time at a set sampling period, and the operating condition characteristics of the current coal mining machine are analyzed based on the acquired operating status parameters.

3. The remote control method for a coal mining machine as described in claim 2, characterized in that: The method for dynamically updating parameters based on the collected operating status parameters includes: Based on the analysis of the current operating condition characteristics, the stability of the operating condition and the criticality of the task are determined, and the timing for switching decision parameters is determined. The length of the transition time used during the decision-making and parameter update process depends on the level of mission criticality. Based on the traction speed characteristics in the operating status parameters, the decision is made regarding the type of transition curve to use during the parameter switching process.

4. The remote control method for a coal mining machine as described in claim 3, characterized in that: The generation of a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results includes, The number of transition steps is determined based on the synchronization signal frequency of the current coal mining machine control system and the transition time obtained from the decision. For each synchronization cycle, the transition coefficient corresponding to the current cycle is determined based on the progress position of the current cycle in the total number of transition steps and the transition curve type selected by the decision. The transition parameters for the current period are obtained by performing transition calculations on the old and updated parameters based on the transition coefficient. The transition parameters corresponding to each cycle are stored in cycle order.

5. The remote control method for a coal mining machine as described in claim 4, characterized in that: The continuous visual reference information required for generating the coal mining machine control includes... The transition parameters corresponding to the current cycle are obtained from the stored transition parameter sequence and sent to the vision processing unit; The vision processing unit processes the multi-camera images based on the received transition parameters to generate visual reference information for the current period.

6. The remote control method for a coal mining machine as described in claim 5, characterized in that: The synchronous compensation of the position commands of the coal mining machine actuator includes... When issuing transition parameters, analyze the changing trend between the old and updated parameters; During the transmission of the transition parameter sequence, the target position calculation results of the coal mining machine actuator are compensated according to the changing trend and the time offset from the start of transmission to the current cycle, so that the output position command is consistent with the visual reference information corresponding to the current cycle.

7. The remote control method for a coal mining machine as described in claim 6, characterized in that: The method of continuously monitoring instruction execution deviations during the switching process to implement anomaly handling includes, During the transition parameter sequence issuance process, the effectiveness of the visual reference information on which the coal mining machine control relies is continuously evaluated. When the effectiveness is lower than the set requirements, the switching process is paused and the old parameters are restored. Simultaneously, the deviation between the actual position and the commanded position of the coal mining machine actuator is continuously monitored. When the deviation exceeds the safe range, the control mode of the coal mining machine is adjusted and the switching process is terminated.

8. A remote control system for a coal mining machine, employing the remote control method for a coal mining machine as described in any one of claims 1 to 7, characterized in that, include: The module includes a data collection module, a switching execution module, an update module, and a dynamic compensation module. The data collection module receives updated parameters describing the relative positional relationship of the camera devices, and simultaneously collects the operating status parameters of the coal mining machine in real time. The switching execution module dynamically decides the execution method for updating parameters based on the collected running status parameters; The update module generates a transition parameter sequence from the currently used old parameters to the updated parameters based on the decision results; The dynamic compensation module generates continuous visual reference information required for coal mining machine control based on the transition parameter sequence. At the same time, it synchronously compensates the position commands of the coal mining machine actuator according to the transition parameter sequence, and continuously monitors command execution deviations during the switching process to implement anomaly handling.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the remote control method for a coal mining machine according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the remote control method for a coal mining machine according to any one of claims 1 to 7.

Images