Hydraulic support and scraper conveyor cooperative pushing control system based on reinforcement learning

Abstract

The present application relates to a hydraulic support and scraper conveyor collaborative pushing control system based on reinforcement learning, belonging to the field of intelligent coal mine technology. The system comprises: a model construction and training module for constructing and training a hydraulic support and machine adaptive intelligent decision-making model; a data processing module for obtaining the scraper arrangement coordinate list and the pushing cylinder length list at the beginning of the current process cycle as the current state; a state input module for inputting the current state into the model to generate a support moving action list; a control module for controlling the actions of each hydraulic support; and a data acquisition module for acquiring the scraper arrangement coordinate list and the pushing cylinder length list at the beginning of the next process cycle as the new current state of the model, and feeding back the new current state to the state input module until the coal mining machine finishes cutting coal. The support moving action list output by the hydraulic support and machine adaptive intelligent decision-making model can be self-adaptively adjusted according to the complex dynamic environment of the fully mechanized working face.

Description

Technical Field

[0001] This invention relates to the field of intelligent coal mining technology, and in particular to a reinforcement learning-based collaborative pushing control system for hydraulic supports and scraper conveyors. Background Technology

[0002] The control of the movement of hydraulic supports and scraper conveyors is a key control link in the automated operation of fully mechanized mining faces. According to the coal mining process requirements, after the coal cutting machine, the corresponding hydraulic supports need to complete actions such as lowering the supports, moving the supports, raising the supports, and pushing the conveyor according to a preset sequence to achieve the forward movement of the hydraulic supports, the advancement of the scraper conveyor, and timely roof support, while maintaining the coordination of the continuous advancement process of the fully mechanized mining face. Currently, the electro-hydraulic control system of hydraulic supports can achieve automatic following operation through program control. However, after the initial automatic following operation, it is often necessary to readjust the position of the hydraulic supports and the state of the pushing cylinders to correct the alignment deviation between the hydraulic support cluster and the scraper conveyor, meeting the requirements of continuous production. To intuitively illustrate the alignment deviation problems that may occur in the field application of the existing automatic following control scheme, a schematic diagram of the spatial relationship between the coal cutting machine, scraper conveyor, and hydraulic support cluster is drawn, based on field data characteristics and actual alignment observations, after the coal cutting operation is completed and the automatic following control ends. Figure 1 As shown. By Figure 1 It is evident that when the automatic follow-up control ends, there may be significant differences in the length of the pushing cylinders of each hydraulic support, and the arrangement of the hydraulic support cluster and the scraper conveyor may also show bending and local deviations. This indicates that the existing automatic follow-up control results are still difficult to directly meet the requirements of on-site production for the arrangement of hydraulic support clusters.

[0003] Furthermore, existing hydraulic support shifting control schemes primarily rely on fixed coal mining process logic and fixed target values ​​for support shifting control. Under ideal conditions where the initial arrangement meets requirements and execution errors are negligible, if the shifting and pushing actions of a single hydraulic support are both aimed at the coal mining machine's cutting depth, the overall arrangement of the hydraulic support cluster and the scraper conveyor can be maintained. However, longwall mining faces typically contain a large number of hydraulic supports, with initial arrangement deviations. Moreover, each hydraulic support is affected by factors such as roof pressure, floor conditions, and cylinder execution errors, making it difficult to maintain consistent actual shifting distances. For longwall mining faces with multiple hydraulic supports advancing collaboratively, relying solely on fixed target values ​​is insufficient to simultaneously consider the advance of a single support and the overall arrangement of the hydraulic support cluster. Simultaneously, during the pushing process, the scraper conveyor is constrained by the geometric relationship and structural continuity of adjacent central troughs. Advance deviations of local hydraulic supports or scrapers can affect the movement space of adjacent units, causing the hydraulic support cluster arrangement adjustment to exhibit significant coupling characteristics.

[0004] Based on this, and addressing the problem that existing hydraulic support and scraper conveyor pushing control mainly relies on preset process logic and mathematical reasoning decision models, making it difficult to adaptively adjust to the complex dynamic environment of the fully mechanized mining face, this invention addresses the need for coordinated optimization of hydraulic support cluster arrangement and advancement. It constructs a hydraulic support-scraper pushing constraint mathematical model and a state transition model, and combines reinforcement learning methods to form a hydraulic support following machine adaptive intelligent decision model, aiming to provide feasible theoretical methods and technical support for intelligent fully mechanized mining faces. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a reinforcement learning-based collaborative pushing control system for hydraulic supports and scraper conveyors. The technical solution of this invention is as follows: A reinforcement learning-based coordinated moving control system for hydraulic supports and scraper conveyors includes: a model building and training module for building and training a reinforcement learning-based adaptive intelligent decision-making model for hydraulic supports; a data acquisition module for acquiring the initial coordinates of each scraper on the scraper conveyor and the initial moving cylinder length of each hydraulic support in the hydraulic support cluster at the start of the current process cycle based on the field perception system of the longwall mining face; and a data processing module for processing the initial coordinates of each scraper and the initial moving cylinder length of each hydraulic support to obtain a list of scraper arrangement coordinates and a list of moving cylinder lengths at the start of the current process cycle, which serve as the trained data. The system includes: a current state of the hydraulic support following the machine adaptive intelligent decision-making model; a state input module for inputting the current state into the trained hydraulic support following the machine adaptive intelligent decision-making model, which then generates a list of moving actions; a control module for controlling the actions of each hydraulic support in the hydraulic support cluster according to the moving action list; and a data acquisition module for acquiring a list of scraper arrangement coordinates and a list of push cylinder lengths at the start of the next process cycle, which serve as the new current state of the trained hydraulic support following the machine adaptive intelligent decision-making model. This new current state is then fed back to the state input module until the coal cutting by the mining machine ends.

[0006] Preferably, the model building and training module includes: a mathematical model building unit, used to combine the adjacent scraper spacing constraint range and the push cylinder length constraint to build a hydraulic support-scraper push constraint mathematical model, so as to establish the mapping relationship between the scraper arrangement coordinates, the push cylinder length and the hydraulic support arrangement coordinates; The system comprises the following units: a state transition model construction unit, used to construct a state transition model, which takes the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t-th process cycle as state inputs, and obtains the list of moving actions of the hydraulic support cluster in the t-th process cycle as action inputs. Based on the hydraulic support-scraper pushing constraint mathematical model, it generates the scraper arrangement coordinate list and the push cylinder length list at the beginning of the (t+1)-th process cycle as the next state; a decision model construction unit, used to construct a hydraulic support following adaptive intelligent decision model, which takes the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t-th process cycle as inputs, and generates the list of moving actions of the hydraulic support cluster in the t-th process cycle; and a training unit, used to train the hydraulic support following adaptive intelligent decision model based on a reinforcement learning algorithm, to obtain the trained hydraulic support following adaptive intelligent decision model.

[0007] Preferably, the mathematical model construction unit includes: a coordinate system construction subunit, used to establish a reference coordinate system with the center of the first scraper corresponding to the first hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis; and a scraper coordinate acquisition subunit, used to acquire a list of scraper arrangement coordinates at the end of the t-th process cycle under the reference coordinate system. , represented as: (1); In formula (1), n ​​is the number of hydraulic supports in the hydraulic support cluster. Let i be the coordinates of the i-th scraper corresponding to the i-th hydraulic support at the end of the t-th process cycle. The spatial constraint determination sub-unit is used to determine the spatial constraint between the i-th scraper and its adjacent scrapers based on the adjacent scraper spacing constraint range: (2); In formula (2), This represents the allowable limit for the spacing between adjacent scrapers; The first subunit for obtaining the length of the push cylinder is used to obtain a list of push cylinder lengths at the end of the t-th process cycle in the reference coordinate system. , represented as: (3); In formula (3), Let be the length of the pushing cylinder of the i-th hydraulic support at the end of the t-th process cycle. This is the maximum length of the hydraulic cylinder. The first hydraulic support coordinate calculation subunit is used to calculate the coordinates of the i-th scraper at the end of the t-th process cycle. The length of the push cylinder of the i-th hydraulic support at the end of the t-th process cycle Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: (4); The hydraulic support arrangement coordinate generation subunit is used to generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle based on the coordinates of each hydraulic support at the end of the t-th process cycle. , represented as: (5).

[0008] Preferably, the state transition model construction unit includes: a data acquisition subunit, used to acquire the scraper arrangement coordinate list at the start of the t-th process cycle. The list of push cylinder lengths is as follows And obtain the list of moving actions of the hydraulic support cluster in the t-th process cycle. , is represented as; (6); In formula (6), For the moving action of the i-th hydraulic support in the t-th process cycle; Input unit, used to generate a list of scraper arrangement coordinates at the start of the t-th process cycle. List of push cylinder lengths And a list of moving actions of the hydraulic support cluster in the t-th process cycle. Input the state transition model. The state transition model calculates the list of hydraulic support arrangement coordinates at the beginning of the t-th process cycle based on the mathematical model of hydraulic support-scraper pushing constraint. , represented as: (7); In formula (7), Let be the coordinates of the i-th hydraulic support at the beginning of the t-th process cycle; The second hydraulic support coordinate calculation subunit is used to calculate the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: (8), and generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle. , represented as: (9); The second subunit for obtaining the length of the moving cylinder is used to determine the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the length of the pushing cylinder of the i-th hydraulic support after the support is moved and before the slide is pushed in the t-th process cycle. The calculation formula is: (10), and generate a list of the lengths of the pushing cylinders after the t-th process cycle shift. , represented as: (11); The allowable push amount calculation subunit is used to calculate the allowable push amount of the i-th scraper in the t-th process cycle based on the spacing constraints between adjacent scrapers. The calculation formula is: (12); In formula (12), The function is for finding the minimum value; The judgment subunit is used to determine the allowable pushing amount of the i-th scraper in the t-th process cycle. Does it meet the constraints of the maximum length of the push cylinder and the available push distance in front of the scraper? If the length of the push cylinder is after the frame is moved... With allowable push volume The sum is greater than the maximum length of the push cylinder. Alternatively, if a pushing distance B is available in front of the scraper, then the allowable pushing amount will be... Revised to or Otherwise, the allowable pushing amount remains unchanged, and is based on the allowable pushing amount of the i-th scraper in the t-th process cycle. Determine the pushing amount of scraper i in the t-th process cycle. The judgment rule is: ; (13); The update subunit is used to adjust the push amount of each scraper in the t-th process cycle. Update the length of the push cylinder of hydraulic support i at the end of process cycle t. The coordinates of the i-th scraper at the end of the t-th process cycle The calculation formula is: (15); In formulas (14) and (15), m is the total number of actions of the i-th scraper in the t-th process cycle, and d is the index of the number of actions of the i-th scraper in the t-th process cycle.

[0009] Preferably, the state transition model construction unit includes: a data acquisition subunit, used to acquire the scraper arrangement coordinate list at the start of the t-th process cycle. , represented as: (16); In formula (16), For state space, For the action space, This is a state transition relation. For the reward function, Discount factor; The state determination subunit is used to determine the scraper arrangement coordinate list at the start of the t-th process cycle. List of push cylinder lengths Determine the state at the start of the t-th process cycle. (17); The status input subunit is used to input the status at the start of the t-th process cycle. Input the hydraulic support following adaptive intelligent decision model, and output the list of hydraulic support cluster movement actions in the t-th process cycle. As the motion space of the hydraulic support cluster in the t-th process cycle (18).

[0010] Preferably, the training unit includes: a state and action input subunit, used to input the state at the start of the t-th process cycle. As the current state, the list of moving actions of the hydraulic support cluster in the t-th process cycle is input into the state transition model, which outputs the scraper arrangement coordinate list at the end of the t-th process cycle. List of push cylinder lengths and as a new state , represented as: (19); The reward calculation subunit is used to calculate the agent's state. Next action Rewards were then given for deviations in the arrangement of hydraulic support clusters and for the effective advancing distance. The iterative training subunit is used to iteratively train the hydraulic support following adaptive intelligent decision-making model based on reinforcement learning algorithm according to the reward until the preset termination condition is reached, so as to obtain the trained hydraulic support following adaptive intelligent decision-making model.

[0011] Preferably, the reward calculation subunit is used to: calculate the agent's state using a reward function. Next action Subsequent rewards for deviations in the arrangement of hydraulic support clusters and for effective advancing distance The reward function is expressed as: (20); In formula (20), and These are the weighting coefficients for the ranking reward and the progress distance reward, respectively. and These are the reward functions for hydraulic support cluster arrangement and the reward function for advance distance, respectively. (twenty one); In formula (21), This is a list of scraper arrangement coordinates at the end of the t-th process cycle. This is a list of the lengths of the push cylinders at the end of the t-th process cycle. This indicates that the hydraulic support cluster arrangement is extremely poor. For normalization; (twenty two); In formula (22), This is a list of moving actions of the hydraulic support cluster in the t-th process cycle. This is a list of the lengths of the push cylinders at the start of the t-th process cycle. The function is a summation function.

[0012] Preferably, the model building and training module further includes: a sample building unit, used to build multiple randomly arranged samples and multiple extreme arranged samples, wherein the randomly arranged samples include a randomly generated list of scraper arrangement coordinates and a randomly generated list of push cylinder lengths, and the extreme arranged samples include a list of push cylinder lengths that meet the on-site statistical results and a list of scraper arrangement coordinates that meet the extreme arrangements.

[0013] Preferably, when constructing any randomly permuted sample, the sample construction unit includes: The coordinate system construction sub-unit is used to establish a reference coordinate system with the center of the first scraper corresponding to the first hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis, and to determine the coordinates of the first scraper under the reference coordinate system. ; The random scraper arrangement coordinate generation sub-unit is used to randomly generate the positional differences between adjacent scrapers within the constraint range of adjacent scraper spacing, and then accumulates the coordinates of each randomly generated scraper to form a list of randomly generated scraper arrangement coordinates. , represented as: (twenty four); (26); In formulas (23) to (26), The positional difference between the i-th scraper and the (i-1)-th scraper. This indicates that the data is randomly generated within the constraint range of the spacing between adjacent scrapers. List the positional differences between adjacent scrapers. This represents the summation of all variables in the expression. Let the coordinates of the i-th randomly generated scraper be... A list of coordinates for the randomly generated scraper arrangement; The random cylinder length generation subunit is used to randomly generate the length of the cylinder of the i-th hydraulic support within the constraint limit, using the maximum length of the cylinder as the upper limit. And a randomly generated list of push cylinder lengths is formed according to the hydraulic support number sequence. , represented as: (28).

[0014] Preferably, when constructing samples for any extreme permutation, the sample construction unit includes: The on-site push cylinder length generation subunit is used to determine the push cylinder length at the end of each process cycle as a normal distribution, and based on the mean of the normal distribution... and variance Generate the initial pushing cylinder length of the i-th hydraulic support in the reference coordinate system. Then, based on the maximum length of the push cylinder The initial pushing cylinder length of the i-th hydraulic support Cut the material to meet physical constraints, and obtain the length of the pushing cylinder of the i-th hydraulic support that satisfies the on-site statistical results. This will generate a list of push cylinder lengths that meet the statistical results from the field. , represented as: (29); (30); (31); In formulas (29) to (31), It is a normal distribution function. This is the clipping function; The sub-unit for generating the absolute value of the position difference between adjacent scrapers is used to sample the position difference between adjacent scrapers using a triangular distribution, and to round, clip, and convert the sampling results to integer values ​​to obtain the absolute value of the position difference between adjacent scrapers within a specified range; wherein, the triangular distribution is represented as: (32); In formula (32), The positional difference between the i-th scraper and the (i-1)-th scraper. a To be the minimum value, b The maximum value, c The mode, and order , , That is, setting both the mode and the right endpoint to 1. , This represents the allowable limit for the spacing between adjacent scrapers. It is a triangular distribution function; The overall skew trend construction sub-unit is used to construct the overall skew trend: First, a global trend direction D is randomly generated, with a value of +1 or -1, corresponding to an overall increasing or decreasing trend, respectively; then, a monotonic mode variable N is introduced to randomly select strict monotonicity or perturbation monotonicity; finally, the absolute value of the position difference between adjacent scrapers is multiplied by the corresponding sign determined according to the global trend direction and the monotonic mode variable to obtain the position difference between adjacent scrapers, expressed as: (33); (34); (35); (36); In formulas (33) to (36), Let p be the function that assigns the symbol to the i-th scraper, where p is the probability of the monotonic mode variable. This represents the positional difference between the i-th scraper and the (i-1)-th scraper. The extreme scraper arrangement coordinate generation sub-unit is used to accumulate the positional differences between adjacent scrapers one by one to obtain a list of scraper arrangement coordinates that satisfy the extreme arrangement.

[0015] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.

[0016] By means of the above scheme, the beneficial effects of the present invention are as follows: by constructing and training a hydraulic support following machine adaptive intelligent decision-making model based on reinforcement learning, the trained hydraulic support following machine adaptive intelligent decision-making model can learn the complex dynamic environment of the fully mechanized mining face, and the list of support moving actions output by the hydraulic support following machine adaptive intelligent decision-making model can be adaptively adjusted with the complex dynamic environment of the fully mechanized mining face, thereby taking into account both the single-unit advance amount and the overall arrangement of the hydraulic support cluster, and thus meeting the requirements of continuous production. Attached Figure Description

[0017] Figure 1 This is a schematic diagram showing the status of each piece of equipment in the fully mechanized mining face when the automatic follow-up control of the hydraulic support ends.

[0018] Figure 2 This is a block diagram of the composition of the reinforcement learning-based hydraulic support and scraper conveyor collaborative pushing control system provided in the embodiments of the present invention.

[0019] Figure 3 This is a schematic diagram illustrating the transformation of the hydraulic support and scraper conveyor's movement control problem in this embodiment of the invention.

[0020] Figure 4 This is a schematic diagram of the overall framework of the adaptive control model for the hydraulic support and scraper conveyor based on reinforcement learning in an embodiment of the present invention.

[0021] Figure 5 This is a schematic diagram of the moving action of the hydraulic support in an embodiment of the present invention.

[0022] Figure 6 This is a schematic diagram of the pushing and sliding action of the hydraulic support in an embodiment of the present invention.

[0023] Figure 7 This is a coordinate schematic diagram of the mathematical model of hydraulic support-scraper pushing constraint in an embodiment of the present invention.

[0024] Figure 8 This is a state transition flowchart of the state transition model in this embodiment of the invention.

[0025] Figure 9 This is a schematic diagram of the Markov decision process modeling for the hydraulic support and scraper conveyor pushing control in an embodiment of the present invention.

[0026] Figure 10 This is a range analysis diagram of the hydraulic support cluster in an embodiment of the present invention.

[0027] Figure 11 This is a diagram showing the instantiation result of the mathematical model of hydraulic support-scraper pushing constraint in an embodiment of the present invention.

[0028] Figure 12 This is a diagram showing the arrangement changes of the hydraulic support cluster under different initial arrangements in the embodiments of the present invention. Detailed Implementation

[0029] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0030] The reinforcement learning-based coordinated pushing control system for hydraulic supports and scraper conveyors provided in this invention can be deployed on any computer, mobile terminal, or server with computing capabilities. Figure 2 As shown, the reinforcement learning-based coordinated pushing control system for hydraulic supports and scraper conveyors provided in this embodiment of the invention includes: The model building and training module is used to build and train a reinforcement learning-based hydraulic support adaptive intelligent decision-making model.

[0031] To illustrate the concept behind the system provided in the embodiments of the present invention, Figure 3 Taking the hydraulic support cluster movement strategy corresponding to an ideal cutting depth of 865 mm for the coal mining machine and a hydraulic support pushing cylinder length of 865 mm as an example, this paper demonstrates the arrangement state update and problem transformation approach of the hydraulic support cluster during multi-cycle support movement control. It is assumed that the coal mining machine has performed idealized coal cutting, meaning that after the coal mining machine's cutting operation, the distance between each scraper and the coal face in front is 865 mm, which is the ideal cutting depth of the coal mining machine. Figure 3 It is evident that the adjustment of the hydraulic support cluster arrangement is not an independent action of a single hydraulic support, but rather a continuous control process under the combined influence of the hydraulic support's shifting action, pushing action, and constraints on the length of the pushing cylinder, the scraper space, and the available pushing distance in front of the scraper. On one hand, the shifting action of a single hydraulic support changes the length of the corresponding pushing cylinder, thus affecting the executable range of subsequent pushing actions. On the other hand, the pushing action changes the scraper arrangement and affects the advancing space of adjacent hydraulic supports through constraints on the spacing between adjacent scrapers and the available pushing distance in front of the scraper. Therefore, the target shifting value of a certain hydraulic support in the current process cycle not only affects its own position but also further affects the scraper arrangement, the pushing cylinder margin, and the adjustable space of subsequent hydraulic supports. Therefore, the difficulty in controlling the movement of hydraulic supports and scraper conveyors is not just about setting the movement distance of a single hydraulic support, but rather about dynamically determining a reasonable hydraulic support cluster movement strategy under the constraints of on-site process and space, comprehensively considering the coupling of hydraulic support movement and pushing actions, the constraint of the spacing between adjacent scrapers, the constraint of the available pushing distance in front of the scrapers, and the coordinated optimization of arrangement deviation and effective pushing distance.

[0032] To solve the above problems, the pushing control process of the hydraulic support and scraper conveyor needs to be transformed from an engineering experience-based description into a calculable and optimizable mathematical problem. For example... Figure 3As shown, this embodiment of the invention first abstracts the on-site control process into a mathematical description composed of arrangement state, spatial constraints, on-site process, and control actions. Specifically, the scraper arrangement coordinate list, the push cylinder length list, and the hydraulic support arrangement coordinate list can be used to describe the current arrangement state of the hydraulic supports and the scraper conveyor. The list of moving actions of the hydraulic support cluster can be used to characterize control decisions. Furthermore, under the conditions of on-site process and spatial constraints, the moving actions of the hydraulic support cluster will cause continuous updates to the states of the hydraulic supports, scrapers, and push cylinders. Therefore, this problem can be transformed into a dynamic decision-making process of "current arrangement state - moving action - next arrangement state". Considering that this process has characteristics such as continuous state evolution, continuous action value taking, multi-hydraulic support coupling, and multi-objective trade-offs, this embodiment of the invention further abstracts it into a sequential decision problem under constraints. Through Markov decision process modeling and reinforcement learning strategy optimization, the mapping relationship from the current arrangement state to a reasonable moving action strategy is learned, thereby forming a hydraulic support following adaptive intelligent decision-making model.

[0033] Based on the above-mentioned problem transformation approach, the hydraulic support adaptive intelligent decision-making model constructed in this embodiment of the invention needs to adaptively determine the target value of each hydraulic support movement under the dynamic changes of the longwall face state, the constraint of the spacing between adjacent scrapers, and the requirement of coordinated arrangement of hydraulic support clusters. Its overall architecture mainly includes the construction of a mathematical model for hydraulic support-scraper movement constraints, the construction of a state transition model, and a hydraulic support adaptive intelligent decision-making model. The overall framework of the model provided by this invention is as follows: Figure 4 As shown. The specific content is as follows: In some embodiments of the present invention, the model building and training module includes: The mathematical model building unit is used to construct a mathematical model of hydraulic support-scraper pushing constraint by combining the constraint range of adjacent scraper spacing and the constraint of pushing cylinder length, so as to establish the mapping relationship between scraper arrangement coordinates, pushing cylinder length and hydraulic support arrangement coordinates.

[0034] Specifically, during the relocation, the hydraulic support uses its corresponding scraper as a fulcrum and is moved towards the coal face by pushing the hydraulic cylinder. Figure 5 As shown. During this process, the scraper position remains unchanged, the coordinates of the hydraulic support change with the retraction amount of the cylinder, and the length of the pushing cylinder decreases accordingly. Therefore, the single movement distance of the hydraulic support is limited by the current retractable length of the pushing cylinder, and the movement amount cannot exceed the current length of the pushing cylinder, i.e., the length constraint of the pushing cylinder must be met. During the pushing action, the hydraulic support uses itself as a fulcrum, and the pushing cylinder extends to drive the corresponding scraper towards the coal wall, as shown. Figure 6As shown. This process is constrained by the maximum extension length of the pushing cylinder, the spacing between adjacent scrapers, and the available pushing distance in front of the scrapers. That is, the pushing distance cannot exceed the current allowable extension length of the pushing cylinder, and the scraper spacing must meet the mechanical limit requirements. Therefore, the pushing action must simultaneously satisfy the pushing cylinder length constraint, the available pushing distance in front of the scrapers constraint, and the spacing between adjacent scrapers constraint.

[0035] Based on the above, in some embodiments of the present invention, the mathematical model construction unit includes: The coordinate system construction sub-unit is used to establish a reference coordinate system with the center of the No. 1 scraper corresponding to the No. 1 hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis.

[0036] Reference coordinate system as Figure 7 As shown. Since the numbering correspondence between the hydraulic support and the scraper remains unchanged along the length of the fully mechanized mining face, and the machine-following control mainly focuses on the displacement changes of the two in the advancing direction, this embodiment of the invention simplifies the spatial arrangement of the hydraulic support and the scraper conveyor into a one-dimensional coordinate sequence in the y-direction.

[0037] The scraper coordinate acquisition sub-unit is used to obtain the scraper arrangement coordinate list at the end of the t-th process cycle in the reference coordinate system. , represented as: (1); In formula (1), n ​​is the number of hydraulic supports in the hydraulic support cluster. Let i be the coordinates of the i-th scraper corresponding to the i-th hydraulic support at the end of the t-th process cycle. The spatial constraint determination sub-unit is used to determine the spatial constraint between the i-th scraper and its adjacent scrapers based on the adjacent scraper spacing constraint range: (2); In formula (2), This represents the allowable limit for the spacing between adjacent scrapers; The first subunit for obtaining the length of the push cylinder is used to obtain a list of push cylinder lengths at the end of the t-th process cycle in the reference coordinate system. , represented as: (3); In formula (3), Let be the length of the pushing cylinder of the i-th hydraulic support at the end of the t-th process cycle. This is the maximum length of the hydraulic cylinder. The first hydraulic support coordinate calculation subunit is used to calculate the coordinates of the i-th scraper at the end of the t-th process cycle. The length of the push cylinder of the i-th hydraulic support at the end of the t-th process cycle Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: (4).

[0038] The hydraulic support arrangement coordinate generation subunit is used to generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle based on the coordinates of each hydraulic support at the end of the t-th process cycle. , represented as: (5).

[0039] In summary, the hydraulic support-scraper pushing constraint mathematical model establishes the mapping relationship between the scraper arrangement coordinates, the pushing cylinder length, and the hydraulic support arrangement coordinates. It incorporates constraints such as the pushing cylinder length constraint, the adjacent scraper spacing constraint, and the available pushing distance in front of the scraper into the description of the support shifting and conveying processes. Based on this hydraulic support-scraper pushing constraint mathematical model, the support shifting and conveying actions of the hydraulic support cluster can be simulated, characterizing the coordinate changes of the hydraulic support and scraper conveyor under different arrangement conditions. Furthermore, given a list of support shifting actions, it can support iterative state updates in continuous process cycles, providing a mathematical foundation for the construction of a state transition model.

[0040] The state transition model construction unit is used to construct the state transition model. The state transition model takes the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t-th process cycle as the state input, and obtains the frame moving action list in the t-th process cycle of the hydraulic support cluster as the action input. Based on the hydraulic support-scraper pushing constraint mathematical model, the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t+1-th process cycle are generated as the next state.

[0041] Specifically, for the problem of controlling the movement of hydraulic supports and scraper conveyors, this embodiment of the invention uses a list of scraper arrangement coordinates and a list of moving cylinder lengths as direct state variables for state transition calculation. The list of hydraulic support arrangement coordinates is calculated from these two variables based on the moving constraint relationship. Based on this, and combined with the "support shifting-conveyor pushing" process flow of the fully mechanized mining face, a state transition model oriented towards reinforcement learning interaction is constructed, and its state transition flow is as follows: Figure 8 As shown.

[0042] In light of the above, in some embodiments of the present invention, the state transition model construction unit includes: The data acquisition subunit is used to obtain the list of scraper arrangement coordinates at the start of the t-th process cycle. The list of push cylinder lengths is as follows And obtain the list of moving actions of the hydraulic support cluster in the t-th process cycle. , is represented as; (6); In formula (6), For the moving action of the i-th hydraulic support in the t-th process cycle; Input unit, used to generate a list of scraper arrangement coordinates at the start of the t-th process cycle. List of push cylinder lengths And a list of moving actions of the hydraulic support cluster in the t-th process cycle. Input the state transition model. The state transition model calculates the list of hydraulic support arrangement coordinates at the beginning of the t-th process cycle based on the mathematical model of hydraulic support-scraper pushing constraint. , represented as: (7); In formula (7), Let be the coordinates of the i-th hydraulic support at the beginning of the t-th process cycle; The second hydraulic support coordinate calculation subunit is used to calculate the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: (8), and generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle. , represented as: (9).

[0043] in, This is a list of coordinates of the hydraulic supports after the t-th process cycle.

[0044] The second subunit for obtaining the length of the moving cylinder is used to determine the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the length of the pushing cylinder of the i-th hydraulic support after the support is moved and before the slide is pushed in the t-th process cycle. The calculation formula is: (10), and generate a list of the lengths of the pushing cylinders after the t-th process cycle shift. , represented as: (11).

[0045] Furthermore, by utilizing the coordinate list of the hydraulic supports after the shift, the length list of the pushing cylinders, and the coordinate list of the scraper arrangement, the update of the scraper arrangement coordinate list during the pushing process can be further calculated. It should be noted that in this embodiment of the invention, the pushing action is treated as a process embedded within the state transition model. The hydraulic support adaptive intelligent decision-making model only outputs the list of shifting actions for the hydraulic support cluster, while the pushing process is executed according to the commonly used on-site principle of "push to full or push no further." Specifically, the state transition model calculates the maximum allowable pushing action (allowable pushing flow) of the scraper corresponding to each hydraulic support based on the current extension length constraint of the pushing cylinder, the available pushing distance constraint in front of the scraper, and the spacing constraint between adjacent scrapers. Simultaneously, it determines the traversal order of the hydraulic supports by combining the forward and reverse operation states of the coal mining machine, and iteratively updates the scraper coordinates for each support. The calculation is repeated until all hydraulic supports and their corresponding scrapers reach the boundaries of the maximum extension constraint of the pushing cylinder, the available pushing distance constraint in front of the scraper, and the spacing constraint between adjacent scrapers, at which point the pushing action is considered complete. The scraper arrangement coordinate list obtained at this point is the scraper arrangement result after the pushing process of this cycle is completed. Therefore, during the push-pull process, the state transition model construction unit includes the following sub-units: allowable push-pull amount calculation, judgment, and update.

[0046] The allowable push amount calculation subunit is used to calculate the allowable push amount of the i-th scraper in the t-th process cycle based on the spacing constraints between adjacent scrapers. The calculation formula is: (12); In formula (12), The function is for finding the minimum value; The judgment subunit is used to determine the allowable pushing amount of the i-th scraper in the t-th process cycle. Does it meet the constraints of the maximum length of the push cylinder and the available push distance in front of the scraper? If the length of the push cylinder is after the frame is moved... With allowable push volume The sum is greater than the maximum length of the push cylinder. Alternatively, if a pushing distance B is available in front of the scraper, then the allowable pushing amount will be... Revised to or Otherwise, the allowable pushing amount remains unchanged, and is based on the allowable pushing amount of the i-th scraper in the t-th process cycle. Determine the pushing amount of scraper i in the t-th process cycle. The judgment rule is: ; (13); The update subunit is used to adjust the push amount of each scraper in the t-th process cycle. Update the length of the push cylinder of hydraulic support i at the end of process cycle t. The coordinates of the i-th scraper at the end of the t-th process cycle The calculation formula is: (15); In formulas (14) and (15), m is the total number of actions of the i-th scraper in the t-th process cycle, and d is the index of the number of actions of the i-th scraper in the t-th process cycle, which are variables.

[0047] As the conveyor pushing process iterates frame by frame, the conveyor pushing action is considered complete when all hydraulic supports and their corresponding scrapers reach the maximum length constraint of the pushing cylinder, the available pushing distance constraint in front of the scraper, and the spacing constraint boundary between adjacent scrapers. The scraper coordinate list and the pushing cylinder length list obtained at this time represent the state results after the conveyor pushing of this process cycle is completed.

[0048] The decision model construction unit is used to construct an adaptive intelligent decision model for hydraulic supports. It takes the list of scraper arrangement coordinates and the list of push cylinder lengths at the beginning of the t-th process cycle as input to generate a list of moving actions of the hydraulic support cluster in the t-th process cycle.

[0049] Specifically, based on the aforementioned state transition model, the evolution of the hydraulic support and scraper conveyor in a continuous process cycle can be described as a process where the current state and the moving action jointly determine the next state. To characterize the sequential decision-making features of the hydraulic support and scraper conveyor's moving control, this embodiment of the invention adopts a centralized single-agent modeling approach, using the hydraulic support cluster of the longwall face as the unified decision-making entity. The agent generates a list of moving actions based on the current hydraulic support-scraper arrangement, and the state transition model calculates the next state that satisfies the moving constraint based on this list, thus modeling the hydraulic support-scraper moving control problem as a Markov decision process. This modeling process includes a state space, an action space, state transition relations, and a reward function definition. The state transition relations are determined by the state transition model, and the reward function is used to evaluate the arrangement of the hydraulic support cluster and the effective advancing distance after the action is executed. A schematic diagram of the Markov decision process modeling for the moving control of the hydraulic support and scraper conveyor is shown below. Figure 9 As shown.

[0050] In some embodiments of the present invention, the decision model construction unit includes: The decision model construction subunit is used to build an adaptive intelligent decision model for hydraulic supports. , represented as: (16); In formula (16), For state space, For the action space, This is a state transition relation. For the reward function, This is the discount factor.

[0051] Considering the continuous, cyclical cumulative effect of hydraulic support cluster arrangement adjustment and propulsion, the current support movement not only affects the arrangement in this cycle but also has a sustained impact on the scraper arrangement, push cylinder length, and effective propulsion distance in subsequent cycles. Therefore, The value was set to 0.995 to enhance the hydraulic support following machine adaptive intelligent decision-making model's focus on long-term cumulative returns.

[0052] The state determination subunit is used to determine the scraper arrangement coordinate list at the start of the t-th process cycle. List of push cylinder lengths Determine the state at the start of the t-th process cycle. (17); The status input subunit is used to input the status at the start of the t-th process cycle. Input the hydraulic support following adaptive intelligent decision model, and output the list of hydraulic support cluster movement actions in the t-th process cycle. As the motion space of the hydraulic support cluster in the t-th process cycle (18).

[0053] The training unit is used to train the hydraulic support following machine adaptive intelligent decision-making model based on the reinforcement learning algorithm, so as to obtain the trained hydraulic support following machine adaptive intelligent decision-making model.

[0054] In some embodiments of the present invention, the training unit includes: The status and action input subunit is used to input the status at the start of the t-th process cycle. As the current state, the list of moving actions of the hydraulic support cluster in the t-th process cycle is input into the state transition model, which outputs the scraper arrangement coordinate list at the end of the t-th process cycle. List of push cylinder lengths and as a new state , represented as: (19).

[0055] The reward calculation subunit is used to calculate the agent's state. Next action Rewards were then given for deviations in the arrangement of hydraulic support clusters and for the effective advancing distance.

[0056] Specifically, the production requirements of fully mechanized mining faces necessitate maintaining good alignment of the hydraulic support clusters and, on this basis, increasing the effective advancing distance. Therefore, this embodiment of the invention constructs a reward function from two aspects: the alignment deviation of the hydraulic support clusters and the advancing effect.

[0057] In some embodiments of the present invention, a reward calculation subunit is used to calculate the agent's state using a reward function. Next action Subsequent rewards for deviations in the arrangement of hydraulic support clusters and for effective advancing distance The reward function is expressed as: (20); In formula (20), and These are the weighting coefficients for the ranking reward and the progress distance reward, respectively. and These are the reward functions for hydraulic support cluster arrangement and advance distance, respectively. Considering that the arrangement of the hydraulic support cluster directly affects the safety of the longwall mining face and the coordinated operation of equipment, its priority is higher than that of the advance distance; therefore, a reward function is set. To standardize reward criteria, all reward items have been normalized. The constraint is within the range [0,1].

[0058] Specifically, regarding the requirements for hydraulic support cluster arrangement, this invention uses the range of coordinates of the hydraulic support cluster arrangement to characterize the dispersion of the hydraulic support cluster arrangement, and constructs an arrangement reward function. : (twenty one); In formula (21), This is a list of scraper arrangement coordinates at the end of the t-th process cycle. Given a list of the lengths of the push cylinders at the end of the t-th process cycle, the coordinates of the hydraulic support arrangement can be calculated from these two values. ; The range indicates the arrangement deviation of the hydraulic support cluster. The smaller the range, the smaller the arrangement deviation of the hydraulic support cluster, which is more conducive to meeting the production requirements of the fully mechanized mining face. The normalization scale is used for constraints. .

[0059] In the evaluation of hydraulic support cluster arrangement, the arrangement deviation of the hydraulic supports is jointly determined by the scraper arrangement and the length of the pushing cylinder. On the one hand, there are mechanical connection constraints between adjacent scrapers, and the maximum allowable relative offset is... On the other hand, the hydraulic support and the corresponding scraper are connected by a push cylinder, the length of which is limited by the maximum length of the push cylinder. Limitations. Therefore, the maximum possible arrangement range of a hydraulic support cluster is determined by both the maximum offset of the scraper arrangement and the maximum length difference of the pushing cylinders.

[0060] like Figure 10 As shown, when the scraper continuously shifts in the same direction, and the relative shift between adjacent scrapers reaches... At that time, the scraper arrangement range reached If simultaneously the length of the corresponding pushing cylinder of the first hydraulic support reaches... When the length of the pushing cylinder corresponding to the end hydraulic support is 0, the coordinate difference between the first and last hydraulic supports reaches its maximum. At this point, the theoretical maximum arrangement range of the hydraulic support cluster is... When the scraper is continuously offset in opposite directions, the same theoretical maximum range can be obtained.

[0061] Based on the aforementioned theoretical maximum range, this invention uses the ratio of the current arrangement range of the hydraulic support cluster to this scale to characterize the degree of arrangement deviation, and constructs a normalized arrangement reward. The smaller the current arrangement range, the smaller the arrangement deviation of the hydraulic support cluster, and the higher the arrangement reward. It should be noted that actual production does not require the hydraulic support cluster range to be 0, but rather requires the arrangement deviation to be within the allowable range. According to on-site production requirements, the arrangement deviation of the hydraulic support cluster should be controlled within ±50mm.

[0062] Furthermore, in the evaluation of the hydraulic support cluster's advancement effect, the distance a single hydraulic support can move is primarily constrained by the retractable length of the current pushing cylinder. When the lengths of the pushing cylinders of each hydraulic support are inconsistent, their movable distances in the current process cycle will also differ. To measure the utilization degree of the hydraulic support cluster to the current movable capacity, this embodiment of the invention constructs an advancement distance reward function using the ratio of the sum of actual movable distances to the sum of current movable distances. : (twenty two); In formula (22), This is a list of moving actions of the hydraulic support cluster in the t-th process cycle. This is a list of the lengths of the push cylinders at the start of the t-th process cycle. The function is a summation function.

[0063] The iterative training subunit is used to iteratively train the hydraulic support following adaptive intelligent decision-making model based on reinforcement learning algorithm according to the reward until the preset termination condition is reached, so as to obtain the trained hydraulic support following adaptive intelligent decision-making model.

[0064] Based on the aforementioned Markov decision process modeling, the adaptive control of the hydraulic support and scraper conveyor can be transformed into a policy optimization problem in a continuous action space. This involves learning the mapping relationship from the current arrangement of the hydraulic supports and scraper conveyor to the list of moving actions of the hydraulic support cluster, enabling the agent to maximize the expected cumulative reward in a continuous process cycle. Considering the continuous value characteristics of the moving action quantities, and the need for state transitions to satisfy constraints such as the length of the moving cylinder, the available moving distance in front of the scraper, and the spacing between adjacent scrapers, this invention employs the TD3 (TwinDelayed Deep Deterministic Policy Gradient) algorithm, suitable for continuous action spaces, to solve this decision problem.

[0065] At the start of training, the policy network has a weak ability to judge the movement of the hydraulic support cluster. Therefore, the hydraulic support following adaptive intelligent decision-making model outputs an initial action based on the current scraper arrangement coordinate list and push cylinder length list, and adds a certain random perturbation to the initial action to increase the opportunity to explore different movement strategies. Subsequently, the state transition model simulates the movement of the hydraulic support and the scraper pushing process based on this movement action, obtaining a new scraper arrangement coordinate list, push cylinder length list, and corresponding reward. The reward value is used to evaluate the effect of this action: if the action can reduce the arrangement deviation of the hydraulic support cluster and increase the effective advance distance, a higher reward is obtained; if the action leads to a worse arrangement or insufficient advance effect, a lower reward is obtained. The "current state, movement action, reward, and next state" generated by each interaction are stored in the experience pool as subsequent training samples. During the network update phase, the TD3 algorithm extracts a batch of historical samples from the experience pool for learning. It uses two evaluation networks simultaneously to judge the long-term effect of a certain action in the current state and takes a more conservative evaluation result to reduce the policy bias caused by overestimation of value. Meanwhile, the TD3 algorithm does not immediately update the policy network every time. Instead, it updates the evaluation network multiple times first, and only updates the policy network after the evaluation results have stabilized, making the training process smoother. As training continues, the policy network gradually tends to output moving actions that yield higher rewards. In other words, the hydraulic support following adaptive intelligent decision-making model learns to provide more reasonable moving action strategies for hydraulic support clusters under conditions such as uneven arrangement of hydraulic support groups, different lengths of pushing cylinders, and limited scraper space by repeatedly comparing the results of different actions under different states. After training, the policy network with the best average reward performance is selected as the final hydraulic support following adaptive intelligent decision-making model for subsequent simulation evaluation or industrial data verification. The preset termination condition can be that the number of iterations reaches a specified number or the loss decreases to a specified loss value.

[0066] The embodiments of the present invention maintain the main structure of the TD3 algorithm, and focus on improving the experience pool sampling mechanism to address the problems of low proportion and insufficient utilization of key working condition samples during training, so as to improve sample utilization efficiency and model training stability.

[0067] In some embodiments of the present invention, considering that the coverage of different working conditions by training samples affects the learning effect of the agent's policy, the embodiments of the present invention design a hybrid sampling strategy in the sample generation stage, including random permutation sampling and extreme permutation sampling. Random permutation sampling is used to simulate general working conditions; extreme permutation sampling is used to supplement critical working condition samples near severe skew and constraint boundaries. Therefore, the model building and training module also includes a sample building unit for building multiple random permutation samples and multiple extreme permutation samples. The random permutation samples include a randomly generated list of scraper arrangement coordinates and a randomly generated list of push cylinder lengths, and the extreme permutation samples include a list of push cylinder lengths that meet the field statistical results and a list of scraper arrangement coordinates that meet the extreme permutation.

[0068] In specific implementation, when constructing any random permutation of samples, the sample construction unit includes: The coordinate system construction sub-unit is used to establish a reference coordinate system with the center of the first scraper corresponding to the first hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis, and to determine the coordinates of the first scraper under the reference coordinate system. ; The random scraper arrangement coordinate generation sub-unit is used to randomly generate the positional differences between adjacent scrapers within the constraint range of adjacent scraper spacing, and then accumulates the coordinates of each randomly generated scraper to form a list of randomly generated scraper arrangement coordinates. , represented as: (twenty three); (twenty four); (25); (26); In formulas (23) to (26), The positional difference between the i-th scraper and the (i-1)-th scraper. This indicates that the data is randomly generated within the constraint range of the spacing between adjacent scrapers. The range of spacing constraints between adjacent scrapers. List the positional differences between adjacent scrapers. This represents the summation of all variables in the expression. Let the coordinates of the i-th randomly generated scraper be... A list of coordinates for the randomly generated scraper arrangement; The random cylinder length generation subunit is used to randomly generate the length of the cylinder of the i-th hydraulic support within the constraint limit, using the maximum length of the cylinder as the upper limit. And a randomly generated list of push cylinder lengths is formed according to the hydraulic support number sequence. , represented as: (27); (28).

[0069] In practical implementation, when constructing samples for any extreme permutation, the sample construction unit includes: The on-site push cylinder length generation subunit is used to determine the push cylinder length at the end of each process cycle as a normal distribution, and based on the mean of the normal distribution... and variance Generate the initial pushing cylinder length of the i-th hydraulic support in the reference coordinate system. Then, based on the maximum length of the push cylinder The initial pushing cylinder length of the i-th hydraulic support Cut the material to meet physical constraints, and obtain the length of the pushing cylinder of the i-th hydraulic support that satisfies the on-site statistical results. This will generate a list of push cylinder lengths that meet the statistical results from the field. , represented as: (29); (30); (31); In formulas (29) to (31), It is a normal distribution function. This is the clipping function; To supplement critical operating condition samples near severe skew and constraint boundaries, this embodiment of the invention further introduces extreme arrangement sampling. This embodiment uses a triangular distribution to sample the positional difference between adjacent scrapers. The probability density of the triangular distribution increases with increasing value, making the sampling results more biased towards larger absolute values ​​of deviation, which is beneficial for generating samples close to the constraint boundaries. The specific details are as follows: The sub-unit for generating the absolute value of the position difference between adjacent scrapers is used to sample the position difference between adjacent scrapers using a triangular distribution, and to round, clip, and convert the sampling results to integer values ​​to obtain the absolute value of the position difference between adjacent scrapers within a specified range; wherein, the triangular distribution is represented as: (32); In formula (32), The positional difference between the i-th scraper and the (i-1)-th scraper.a To be the minimum value, b The maximum value, c The mode, and order , , That is, setting both the mode and the right endpoint to 1. , This represents the allowable limit for the spacing between adjacent scrapers. It is a triangular distribution function; The overall skew trend is constructed using sub-units, with the sign `sign` assigned to each sub-unit. First, a global trend direction `D` is randomly generated, taking a value of +1 or -1, corresponding to an overall increasing or decreasing trend, respectively. Then, a monotonic mode variable `N` is introduced to randomly select between strict monotonicity and perturbation monotonicity. In strict monotonicity, all deviations are assigned the same sign, ensuring continuous change of the scraper arrangement in the same direction. In perturbation monotonicity, each deviation has a 70% probability of remaining consistent with the global trend and a 30% probability of taking the opposite direction, introducing local perturbation while maintaining the overall trend to improve the diversity of extreme samples and on-site similarity. Finally, the absolute value of the position difference between adjacent scrapers is multiplied by the corresponding sign determined based on the global trend direction and the monotonic mode variable to obtain the position difference between adjacent scrapers, expressed as: (33); (34); (35); (36); In formulas (33) to (36), Let p be the function that assigns the symbol to the i-th scraper, where p is the probability of the monotonic mode variable. This represents the positional difference between the i-th scraper and the (i-1)-th scraper.

[0070] The extreme scraper arrangement coordinate generation sub-unit is used to accumulate the positional differences between adjacent scrapers one by one to obtain a list of scraper arrangement coordinates that satisfy the extreme arrangement.

[0071] Specifically, the implementation principle of this step is the same as that of formulas (25) and (26), which will not be repeated here.

[0072] By using the above-mentioned hybrid sampling method, the training samples constructed in the embodiments of the present invention not only cover general random arrangement conditions, but also supplement key samples near extreme arrangement conditions and constraint boundaries, thereby enhancing the coverage of different support-scraper arrangement states.

[0073] Furthermore, to verify the correctness and applicability of the hydraulic support-scraper pushing constraint mathematical model, this embodiment of the invention implements the model programmatically using Python, establishing an instantiated model that can simulate the movement, pushing, and state update processes of the hydraulic support cluster. Based on this, a list of scraper arrangement coordinates, a list of pushing cylinder lengths, and a list of hydraulic support cluster movement actions under typical arrangement conditions are constructed as inputs. The scraper coordinates, pushing cylinder lengths, and arrangement results of the hydraulic support cluster after movement and pushing are calculated. Subsequently, the output results are verified based on the pushing cylinder length constraints and adjacent scraper spacing constraints, validating the ability of the hydraulic support-scraper pushing constraint mathematical model to describe the coordinated propulsion process of the hydraulic support and scraper conveyor.

[0074] To conduct an instantiation verification, the key parameters and input variables of the hydraulic support-scraper pushing constraint mathematical model are first defined. The number of hydraulic supports is taken as n=94, and the maximum length of the pushing cylinder is taken as... The cutting depth of the coal mining machine is taken as 865mm, and the allowable limit of the spacing between adjacent scrapers is taken as... Based on this, a list of scraper arrangement coordinates, a list of pushing cylinder lengths, and a list of moving actions of the hydraulic support cluster are generated as inputs to the mathematical model of hydraulic support-scraper pushing constraints.

[0075] Specifically, when generating the coordinate list of the scraper arrangement, the center of the first scraper is taken as the origin of the coordinates. Subsequently, within the constraint range of adjacent scraper spacing, the positional differences between adjacent scrapers are randomly generated, and the coordinates of each scraper are accumulated to form a list of scraper arrangement coordinates. For details, please refer to formulas (23) to (26).

[0076] The generation of the list of push cylinder lengths is constrained by the maximum length of the push cylinder. For the i-th hydraulic support, the push cylinder length of the i-th hydraulic support is randomly generated within the constrained upper limit. And a randomly generated list of push cylinder lengths is formed according to the hydraulic support number sequence. Specifically, as shown in formulas (27) and (28).

[0077] The generation of the hydraulic support cluster's moving action list considers that the moving distance of a single support is limited by the retractable length of the current pushing cylinder, using the current pushing cylinder length of the i-th hydraulic support as an example. As the upper limit of the action range, randomly generate the moving action value of the i-th hydraulic support within the range. Arrange the moving action values ​​of each hydraulic support in numerical order to obtain a list of moving actions for the hydraulic support cluster. Specifically, this can be expressed as formula (37) and formula (38).

[0078] (37); (38); Through the above method, initial arrangement conditions and action inputs that satisfy basic constraints can be constructed, providing test samples for verifying the mathematical model of hydraulic support-scraper pushing constraint. In this embodiment of the invention, the generated scraper mine-clearing coordinate list, pushing cylinder length list, and hydraulic support cluster moving action list are input into the model to simulate the moving and pushing process in a complete process cycle, and the results are visualized and statistically analyzed. The visualization includes the arrangement of the hydraulic support cluster before and after moving, the scraper arrangement before and after pushing, and the change in pushing cylinder length, such as... Figure 11 As shown.

[0079] Depend on Figure 11 It can be seen that there is a certain distance difference between the initial hydraulic support cluster arrangement and the initial scraper arrangement, and the length of the pushing cylinder varies to different degrees with the hydraulic support number. After the hydraulic support cluster executes the list of moving actions, its arrangement position moves from the initial state towards the corresponding scraper direction, indicating that the hydraulic support-scraper pushing constraint mathematical model can update the hydraulic support arrangement coordinates according to the list of moving actions. Subsequently, the scrapers move as a whole towards the direction of the fully mechanized mining face under the action of the pushing action, and maintain a continuous arrangement after the pushing action, indicating that the hydraulic support-scraper pushing constraint mathematical model can complete the system state update according to the "moving support-pushing action" process sequence.

[0080] To further verify the satisfaction of the constraints by the hydraulic support-scraper pushing constraint mathematical model, this embodiment of the invention statistically analyzes the output results of the hydraulic support-scraper pushing constraint mathematical model from three aspects: the pushing cylinder length constraint, the available pushing distance constraint in front of the scraper, and the adjacent scraper spacing constraint. Regarding the pushing cylinder length, the maximum and minimum values ​​of the pushing cylinder length before and after the support shift are statistically analyzed. Regarding the scraper pushing distance, the maximum and minimum values ​​of the scraper pushing distance after the scraper shift are statistically analyzed. Regarding the adjacent scraper spacing, the maximum and minimum values ​​of the coordinate deviation between adjacent scrapers before and after the scraper shift are statistically analyzed. The adjacent scraper coordinate deviation is represented by the absolute value of the difference between the coordinates of adjacent scrapers. The statistical results are shown in Table 1.

[0081]

[0082] Table 1 shows that the maximum length of the pushing cylinder before the frame shift is 861 mm, which does not exceed the maximum length of 865 mm; the maximum length of the pushing cylinder after the slide is also 865 mm, still satisfying the pushing cylinder length constraint. Meanwhile, the maximum coordinate deviation between adjacent scrapers before and after the slide is 5 mm, which does not exceed the allowable limit for the distance between adjacent scrapers. The maximum scraper displacement after the slide is 721.0 mm, and the minimum is 655.0 mm. The results indicate that the constructed hydraulic support-scraper pushing constraint mathematical model can complete the frame shift, slide, and state update under different arrangement conditions, and ensures that the results satisfy the pushing cylinder length constraint, the available pushing distance constraint in front of the scraper, and the adjacent scraper distance constraint, verifying the correctness and constraint description capability of the hydraulic support-scraper pushing constraint mathematical model.

[0083] In this embodiment of the invention, state transition samples obtained through hybrid sampling are used for training the TD3 intelligent decision-making model, enabling the agent to learn adaptive adjustment strategies for the movement of hydraulic support clusters. The evaluation effect of the hydraulic support following adaptive intelligent decision-making model is further described below when the number of hydraulic supports is 80.

[0084] To verify the generalization performance of the hydraulic support following adaptive intelligent decision-making model based on the TD3 algorithm, this invention independently evaluates the model under conventional random conditions and typical extreme conditions after training. The conventional random conditions are used to test the control capability of the hydraulic support following adaptive intelligent decision-making model under general arrangement perturbations, while the typical extreme conditions are used to test the adaptability of the model under conditions of large initial arrangement deviations and near constraint boundaries. With a single-round decision step size of 6, the initial states of the scraper conveyor, the hydraulic support cluster, and the length of the pushing cylinder are visualized, and the arrangement changes of the hydraulic support cluster after each step are shown to characterize the dynamic control process of the hydraulic support following adaptive intelligent decision-making model under different initial arrangement conditions. The results are as follows: Figure 12 As shown. Let the arrangement of the hydraulic support cluster in the initial state and after each step of the action be denoted as follows: The embodiments of the present invention employ arrangement range. and minimum step distance of cluster Two indicators are used for quantitative evaluation. The arrangement range is used to measure the degree of arrangement deviation of the hydraulic support cluster, and the minimum advance distance of the cluster is used to characterize the effective advancement level of the support with the most unfavorable advancement in the hydraulic support cluster. Thus, the influence of the hydraulic support following machine adaptive intelligent decision-making model on the arrangement control and overall advancement effect is comprehensively evaluated. The specific values ​​are shown in Tables 2 and 3, and the mathematical definitions are shown in formulas (39) and (40).

[0085] (39); (40).

[0086]

[0087] Combination Figure 12 The results and quantitative index analysis show that the adaptive intelligent decision-making model for hydraulic supports exhibits good dynamic control effects under different initial arrangement conditions. For random arrangement conditions, the initial arrangement of the hydraulic support cluster has a certain deviation. After decision adjustment, the adaptive intelligent decision-making model can reduce the arrangement range from 186.0 mm to 29.0 mm and stably increase the minimum advance distance to 836 mm, indicating that the model can quickly improve the arrangement state of the hydraulic support group and enhance its propulsion capability under general disturbance conditions. For typical extreme conditions, although the initial arrangement deviation is larger, with the highest arrangement range reaching 493.0 mm and the lowest advance distance only 323.0 mm, after six process cycle decisions, the arrangement range of the hydraulic support group basically stabilizes within the range of 28.0–32.0 mm, and the minimum advance distance remains around 800 mm. This indicates that the adaptive intelligent decision-making model still has strong adjustment capabilities under severe skew or limited propulsion conditions. Overall, the adaptive intelligent decision-making model for hydraulic supports can effectively reduce the arrangement deviation of hydraulic support clusters, improve the overall propulsion effect, and maintain good decision-making stability and robustness under different initial arrangement conditions.

[0088] The data acquisition module is used to acquire the initial coordinates of each scraper of the scraper conveyor and the initial push cylinder length of each hydraulic support in the hydraulic support cluster at the start of the current process cycle based on the field perception system of the fully mechanized mining face.

[0089] Specifically, the field sensing system includes a push cylinder stroke sensor and a laser rangefinder sensor, which respectively collect the initial push cylinder length of each hydraulic support and the initial coordinates of each scraper.

[0090] The data processing module processes the initial coordinates of each scraper and the initial length of each hydraulic support's push cylinder to obtain a list of scraper arrangement coordinates and a list of push cylinder lengths at the start of the current process cycle. This list serves as the current state of the trained hydraulic support adaptive intelligent decision-making model.

[0091] Specifically, the processing of the initial coordinates of each scraper and the initial pushing cylinder length of each hydraulic support includes coordinate transformation, filtering, and support number matching. Coordinate transformation is the conversion from the sensor coordinate system to the reference coordinate system. Filtering includes outlier replacement and noise removal.

[0092] The status input module is used to input the current status into the trained hydraulic support following machine adaptive intelligent decision-making model, which then generates a list of moving actions.

[0093] The control module is used to control the actions of each hydraulic support in the hydraulic support cluster according to the list of moving actions.

[0094] Specifically, the list of moving actions includes the target values ​​for the moving actions of each hydraulic support. The electro-hydraulic control system of the hydraulic support controls the moving of each hydraulic support, and the moving value is the target value for the moving action of each hydraulic support.

[0095] The data acquisition module is also used to acquire the scraper arrangement coordinate list and the push cylinder length list at the start of the next process cycle, as the new current state of the trained hydraulic support adaptive intelligent decision-making model, and to feed the new current state back to the state input module until the coal mining machine finishes cutting coal.

[0096] Specifically, the initial coordinates of each scraper on the scraper conveyor and the initial length of the push cylinder of each hydraulic support in the hydraulic support cluster are obtained through the field perception system of the longwall mining face at the start of the next process cycle. After processing, a list of scraper arrangement coordinates and a list of push cylinder lengths at the start of the next process cycle are obtained. The new current state is fed back to the state input module, which then inputs the new current state into the trained hydraulic support following adaptive intelligent decision-making model. The trained model generates a list of support-moving actions, and the control module controls the actions of each hydraulic support in the hydraulic support cluster according to this list. This process repeats until the coal cutting by the mining machine ends.

[0097] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A reinforcement learning-based coordinated pushing control system for hydraulic supports and scraper conveyors, characterized in that, include: The model building and training module is used to build and train a reinforcement learning-based hydraulic support following adaptive intelligent decision-making model. The data acquisition module is used to acquire the initial coordinates of each scraper of the scraper conveyor and the initial push cylinder length of each hydraulic support in the hydraulic support cluster at the start of the current process cycle based on the field perception system of the fully mechanized mining face. The data processing module is used to process the initial coordinates of each scraper and the initial push cylinder length of each hydraulic support to obtain a list of scraper arrangement coordinates and a list of push cylinder lengths at the start of the current process cycle, which serves as the current state of the trained hydraulic support following machine adaptive intelligent decision-making model. The status input module is used to input the current status into the trained hydraulic support following machine adaptive intelligent decision-making model, which then generates a list of moving actions. The control module is used to control the actions of each hydraulic support in the hydraulic support cluster according to the list of moving actions; The data acquisition module is also used to acquire the scraper arrangement coordinate list and the push cylinder length list at the start of the next process cycle, as the new current state of the trained hydraulic support adaptive intelligent decision-making model, and to feed the new current state back to the state input module until the coal mining machine finishes cutting coal.

2. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 1, characterized in that, The model building and training module includes: The mathematical model building unit is used to combine the constraint range of the spacing between adjacent scrapers and the constraint of the length of the pushing cylinder to build a mathematical model of hydraulic support-scraper pushing constraint, so as to establish the mapping relationship between the scraper arrangement coordinates, the length of the pushing cylinder and the arrangement coordinates of the hydraulic support; The state transition model construction unit is used to construct a state transition model. The state transition model takes the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t-th process cycle as state input, and obtains the frame moving action list in the t-th process cycle of the hydraulic support cluster as action input. Based on the hydraulic support-scraper pushing constraint mathematical model, the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t+1-th process cycle are generated as the next state. The decision model construction unit is used to construct a hydraulic support following machine adaptive intelligent decision model. The hydraulic support following machine adaptive intelligent decision model takes the scraper arrangement coordinate list and the push cylinder length list at the beginning of the t-th process cycle as input to generate a list of hydraulic support cluster moving actions in the t-th process cycle. The training unit is used to train the hydraulic support following machine adaptive intelligent decision-making model based on the reinforcement learning algorithm, so as to obtain the trained hydraulic support following machine adaptive intelligent decision-making model.

3. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 2, characterized in that, The mathematical model construction unit includes: The coordinate system construction sub-unit is used to establish a reference coordinate system with the center of the No. 1 scraper corresponding to the No. 1 hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis. The scraper coordinate acquisition sub-unit is used to obtain the scraper arrangement coordinate list at the end of the t-th process cycle in the reference coordinate system. , represented as: （1）； In formula (1), n ​​is the number of hydraulic supports in the hydraulic support cluster. Let i be the coordinates of the i-th scraper corresponding to the i-th hydraulic support at the end of the t-th process cycle. The spatial constraint determination sub-unit is used to determine the spatial constraint between the i-th scraper and its adjacent scrapers based on the adjacent scraper spacing constraint range: （2）； In formula (2), This represents the allowable limit for the spacing between adjacent scrapers; The first subunit for obtaining the length of the push cylinder is used to obtain a list of push cylinder lengths at the end of the t-th process cycle in the reference coordinate system. , represented as: （3）； In formula (3), Let be the length of the pushing cylinder of the i-th hydraulic support at the end of the t-th process cycle. This is the maximum length of the hydraulic cylinder. The first hydraulic support coordinate calculation subunit is used to calculate the coordinates of the i-th scraper at the end of the t-th process cycle. The length of the push cylinder of the i-th hydraulic support at the end of the t-th process cycle Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: （4）； The hydraulic support arrangement coordinate generation subunit is used to generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle based on the coordinates of each hydraulic support at the end of the t-th process cycle. , represented as: （5）。 4. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 2, characterized in that, The state transition model construction unit includes: The data acquisition subunit is used to obtain the list of scraper arrangement coordinates at the start of the t-th process cycle. The list of push cylinder lengths is as follows And obtain the list of moving actions of the hydraulic support cluster in the t-th process cycle. , is represented as; （6）； In formula (6), For the moving action of the i-th hydraulic support in the t-th process cycle; Input unit, used to generate a list of scraper arrangement coordinates at the start of the t-th process cycle. List of push cylinder lengths And a list of moving actions of the hydraulic support cluster in the t-th process cycle. Input the state transition model. The state transition model calculates the list of hydraulic support arrangement coordinates at the beginning of the t-th process cycle based on the mathematical model of hydraulic support-scraper pushing constraint. , represented as: （7）； In formula (7), Let be the coordinates of the i-th hydraulic support at the beginning of the t-th process cycle; The second hydraulic support coordinate calculation subunit is used to calculate the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the coordinates of the i-th hydraulic support at the end of the t-th process cycle. The calculation formula is: (8), and generate a list of hydraulic support arrangement coordinates at the end of the t-th process cycle. , represented as: （9）； The second subunit for obtaining the length of the moving cylinder is used to determine the list of moving actions of the hydraulic support cluster in the t-th process cycle. Calculate the length of the pushing cylinder of the i-th hydraulic support after the support is moved and before the slide is pushed in the t-th process cycle. The calculation formula is: (10), and generate a list of the lengths of the pushing cylinders after the t-th process cycle shift. , represented as: （11）； The allowable push amount calculation subunit is used to calculate the allowable push amount of the i-th scraper in the t-th process cycle based on the spacing constraints between adjacent scrapers. The calculation formula is: （12）； In formula (12), The function is for finding the minimum value; The judgment subunit is used to determine the allowable pushing amount of the i-th scraper in the t-th process cycle. Does it meet the constraints of the maximum length of the push cylinder and the available push distance in front of the scraper? If the length of the push cylinder is after the frame is moved... With allowable push volume The sum is greater than the maximum length of the push cylinder. Alternatively, if a pushing distance B is available in front of the scraper, then the allowable pushing amount will be... Revised to or Otherwise, the allowable pushing amount remains unchanged, and is based on the allowable pushing amount of the i-th scraper in the t-th process cycle. Determine the pushing amount of scraper i in the t-th process cycle. The judgment rule is: ； （13）； The update subunit is used to adjust the push amount of each scraper in the t-th process cycle. Update the length of the push cylinder of hydraulic support i at the end of process cycle t. The coordinates of the i-th scraper at the end of the t-th process cycle The calculation formula is: （14）； （15）； In formulas (14) and (15), m is the total number of actions of the i-th scraper in the t-th process cycle, and d is the index of the number of actions of the i-th scraper in the t-th process cycle.

5. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 2, characterized in that, The decision model construction unit includes: The decision model construction subunit is used to build an adaptive intelligent decision model for hydraulic supports. , represented as: （16）； In formula (16), For state space, For the action space, This is a state transition relation. For the reward function, Discount factor; The state determination subunit is used to determine the scraper arrangement coordinate list at the start of the t-th process cycle. List of push cylinder lengths Determine the state at the start of the t-th process cycle. (17); The status input subunit is used to input the status at the start of the t-th process cycle. Input the hydraulic support following adaptive intelligent decision model, and output the list of hydraulic support cluster movement actions in the t-th process cycle. As the motion space of the hydraulic support cluster in the t-th process cycle (18).

6. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 5, characterized in that, The training unit includes: The status and action input subunit is used to input the status at the start of the t-th process cycle. As the current state, the list of moving actions of the hydraulic support cluster in the t-th process cycle is input into the state transition model, which outputs the scraper arrangement coordinate list at the end of the t-th process cycle. List of push cylinder lengths and as a new state , represented as: (19); The reward calculation subunit is used to calculate the agent's state. Next action Rewards were then given for deviations in the arrangement of hydraulic support clusters and for the effective advancing distance. The iterative training subunit is used to iteratively train the hydraulic support following adaptive intelligent decision-making model based on reinforcement learning algorithm according to the reward until the preset termination condition is reached, so as to obtain the trained hydraulic support following adaptive intelligent decision-making model.

7. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 6, characterized in that, The reward calculation subunit is used to: calculate the agent's state using a reward function. Next action Subsequent rewards for deviations in the arrangement of hydraulic support clusters and for effective advancing distance The reward function is expressed as: （20）； In formula (20), and These are the weighting coefficients for the ranking reward and the progress distance reward, respectively. and These are the reward functions for hydraulic support cluster arrangement and the reward function for advance distance, respectively. （21）； In formula (21), This is a list of scraper arrangement coordinates at the end of the t-th process cycle. This is a list of the lengths of the push cylinders at the end of the t-th process cycle. This indicates that the hydraulic support cluster arrangement is extremely poor. For normalization; （22）； In formula (22), This is a list of moving actions of the hydraulic support cluster in the t-th process cycle. This is a list of the lengths of the push cylinders at the start of the t-th process cycle. The function is a summation function.

8. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 2, characterized in that, The model building and training module also includes: The sample construction unit is used to construct multiple random permutation samples and multiple extreme permutation samples. The random permutation samples include a randomly generated list of scraper arrangement coordinates and a randomly generated list of push cylinder lengths. The extreme permutation samples include a list of push cylinder lengths that meet the field statistical results and a list of scraper arrangement coordinates that meet the extreme permutation.

9. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 8, characterized in that, When constructing any randomly permuted sample, the sample construction unit includes: The coordinate system construction sub-unit is used to establish a reference coordinate system with the center of the first scraper corresponding to the first hydraulic support as the origin, the forward running direction of the coal mining machine as the x-axis, and the advancing direction of the fully mechanized mining face as the y-axis, and to determine the coordinates of the first scraper under the reference coordinate system. ; The random scraper arrangement coordinate generation sub-unit is used to randomly generate the positional differences between adjacent scrapers within the constraint range of adjacent scraper spacing, and then accumulates the coordinates of each randomly generated scraper to form a list of randomly generated scraper arrangement coordinates. , represented as: （23）； （24）； （25）； （26）； In formulas (23) to (26), The positional difference between the i-th scraper and the (i-1)-th scraper. This indicates that the data is randomly generated within the constraint range of the spacing between adjacent scrapers. List the positional differences between adjacent scrapers. This represents the summation of all variables in the expression. Let the coordinates of the i-th randomly generated scraper be... A list of coordinates for the randomly generated scraper arrangement; The random cylinder length generation subunit is used to randomly generate the length of the cylinder of the i-th hydraulic support within the constraint limit, using the maximum length of the cylinder as the upper limit. And a randomly generated list of push cylinder lengths is formed according to the hydraulic support number sequence. , represented as: （27）； （28）。 10. The reinforcement learning-based hydraulic support and scraper conveyor coordinated pushing control system according to claim 8, characterized in that, When constructing samples for any extreme permutation, the sample construction unit includes: The on-site push cylinder length generation subunit is used to determine the push cylinder length at the end of each process cycle as a normal distribution, and based on the mean of the normal distribution... and variance Generate the initial pushing cylinder length of the i-th hydraulic support in the reference coordinate system. Then, based on the maximum length of the push cylinder The initial pushing cylinder length of the i-th hydraulic support Cut the material to meet physical constraints, and obtain the length of the pushing cylinder of the i-th hydraulic support that satisfies the on-site statistical results. This will generate a list of push cylinder lengths that meet the statistical results from the field. , represented as: （29）； （30）； （31）； In formulas (29) to (31), It is a normal distribution function. This is the clipping function; The sub-unit for generating the absolute value of the position difference between adjacent scrapers is used to sample the position difference between adjacent scrapers using a triangular distribution, and to round, clip, and convert the sampling results to integer values ​​to obtain the absolute value of the position difference between adjacent scrapers within a specified range; wherein, the triangular distribution is represented as: （32）； In formula (32), The positional difference between the i-th scraper and the (i-1)-th scraper. a To be the minimum value, b The maximum value, c The mode, and order , , That is, setting both the mode and the right endpoint to 1. , This represents the allowable limit for the spacing between adjacent scrapers. It is a triangular distribution function; The overall skew trend construction sub-unit is used to construct the overall skew trend: First, a global trend direction D is randomly generated, with a value of +1 or -1, corresponding to an overall increasing or decreasing trend, respectively; then, a monotonic mode variable N is introduced to randomly select strict monotonicity or perturbation monotonicity; finally, the absolute value of the position difference between adjacent scrapers is multiplied by the corresponding sign determined according to the global trend direction and the monotonic mode variable to obtain the position difference between adjacent scrapers, expressed as: （33）； （34）； （35）； （36）； In formulas (33) to (36), Let p be the function that assigns the symbol to the i-th scraper, where p is the probability of the monotonic mode variable. This represents the positional difference between the i-th scraper and the (i-1)-th scraper. The extreme scraper arrangement coordinate generation sub-unit is used to accumulate the positional differences between adjacent scrapers one by one to obtain a list of scraper arrangement coordinates that satisfy the extreme arrangement.

Images