A method, apparatus and machine readable storage medium for drilling assistance decision making

Abstract

The application relates to the technical field of oil and gas exploration, and discloses a drilling auxiliary decision-making method, a drilling auxiliary decision-making device and a machine readable storage medium. The drilling auxiliary decision-making method comprises the following steps: processing first real-time operation data containing first real-time working parameters and first real-time drilling data by means of a first initial decision-making model trained based on behavior cloning, so as to obtain an initial decision; after a driller determines actual operation according to the initial decision and performs actual application, second real-time operation data containing second real-time working parameters of drilling equipment and second real-time drilling data are generated, and a first intention label used for representing an engineering target of the actual operation is determined according to the second real-time operation data; and the second real-time operation data and the first intention label are input into a decision optimization model determined based on reinforcement learning, so that a drilling decision can be finally obtained efficiently and accurately.

Description

Technical Field

[0001] This application relates to the field of oil and gas exploration technology, and more specifically to a drilling auxiliary decision-making method, device and machine-readable storage medium. Background Technology

[0002] The driller is a core operator in drilling operations, typically responsible for directing on-site operations and operating equipment. Therefore, the quality of the driller's decisions directly impacts drilling efficiency, drilling quality, and operational safety.

[0003] In existing drilling operations, the working environment is often complex and highly uncertain. Traditional decision support systems usually rely on static rules, resulting in low decision-making efficiency and accuracy, which cannot meet the needs of the work site. Summary of the Invention

[0004] The purpose of this application is to provide a drilling auxiliary decision-making method to solve the problems of slow decision-making speed, weak decision-making accuracy, and inability of decisions to properly match operational requirements in the prior art.

[0005] To achieve the above objectives, the first aspect of this application provides a drilling auxiliary decision-making method, comprising:

[0006] Acquire first real-time operation data, wherein the first real-time operation data includes the first real-time operating parameters of the drilling equipment and the first real-time drilling data;

[0007] The first real-time operation data is input into the initial decision model to obtain the initial decision, wherein the initial decision model is obtained based on behavior cloning;

[0008] Acquire the second real-time operation data of the drilling equipment, wherein the second real-time operation data is obtained after the driller determines the actual operation based on the initial decision and applies the actual operation to the drilling equipment;

[0009] Based on the second real-time operation data, a first intent label is determined, wherein the first intent label is used to characterize the engineering goal of the actual operation;

[0010] Based on reinforcement learning, the second real-time task data and the first intention label are input into the decision optimization model to obtain the optimized decision.

[0011] In this embodiment of the application, the training process of the initial decision model includes:

[0012] Acquire historical operation data and corresponding historical decision information, including historical operation data such as historical operating parameters of drilling equipment and historical drilling data;

[0013] Based on historical drilling data, a second intent label is determined, which is used to characterize the motivation of historical decision-making information;

[0014] Input historical task data, corresponding historical decision information, and second intent labels into a preset neural network to obtain predictive decision information corresponding to historical task data;

[0015] Based on historical decision-making information and predictive decision-making information, and using the first loss function, the loss value corresponding to the historical operation data is calculated.

[0016] With the goal of minimizing the loss value, the preset neural network is trained iteratively until the preset first training termination condition is met, at which point the training is terminated to obtain the initial decision model.

[0017] In this embodiment of the application, the predictive decision information includes whether the driller makes a decision, the type of working parameter to be adjusted corresponding to the decision, and the adjustment value for each type of working parameter.

[0018] In this embodiment, the first loss function is obtained by weighted summation of the decision occurrence judgment loss, the decision type loss, and the decision parameter adjustment value loss, wherein the decision occurrence judgment loss is the cross-entropy loss function, the decision type loss is the multi-label cross-entropy loss function, and the decision parameter adjustment value loss is the mean squared error loss function.

[0019] In this embodiment of the application, the historical decision information includes multiple historical decisions made within a preset distance of the drilling equipment movement.

[0020] In this embodiment of the application, reinforcement learning-based second real-time job data and a first intent label are input into a decision optimization model to obtain an optimization decision, including:

[0021] The second real-time job data is used as the state space, and the first intention label is used as the state enhancement of the state space. The data is then input into the decision optimization model to determine the target action from the action space, wherein the action space includes the changing state of the target working parameters.

[0022] Based on the action space and the first intention label, the expected total reward is determined through the reward function and the second loss function;

[0023] The optimal decision is determined with the goal of maximizing the expected total reward.

[0024] In this embodiment of the application, the reward function is obtained by weighted summation of the consistency reward between the initial decision and the actual operation and the consistency reward between the actual operation and the first intention label.

[0025] In this embodiment of the application, the second loss function includes:

[0026]

[0027] in, L RL Let Q be the second loss function, representing the expected total reward that can be obtained by taking a certain action in a certain state. Let γ represent the expected value of the difference between the Q values ​​for all states s and actions a, where R is the actual reward obtained by taking an action in the current state, and γ is the discount factor. Q(s,a) is the maximum Q value that the agent takes in the next state s', and Q(s,a) is the Q value of the agent taking action a in state s.

[0028] A second aspect of this application provides a drilling auxiliary decision-making device, comprising:

[0029] The memory is configured to store instructions; and

[0030] The processor is configured to retrieve instructions from the memory and, when executing the instructions, to implement any of the methods for drilling assistance decisions.

[0031] A third aspect of this application provides a machine-readable storage medium storing instructions for causing a machine to perform any of the drilling assistance decision-making methods described in the present invention.

[0032] Through the above technical solution, this application uses a first initial decision model obtained based on behavior cloning training to process the first real-time operation data, which includes the first real-time working parameters and the first real-time drilling data, to obtain an initial decision. After the driller determines the actual operation based on the initial decision and applies it, the second real-time operation data, which includes the second real-time working parameters of the drilling equipment and the second real-time drilling data, is generated. At the same time, a first intent label is determined based on the actual operation to characterize the engineering objective of the actual operation. The second real-time operation data and the first intent label are input into a decision optimization model determined based on reinforcement learning, and finally the optimized drilling decision is obtained efficiently and accurately.

[0033] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0034] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:

[0035] Figure 1 The illustration shows a flowchart of a drilling assistance decision-making method according to an embodiment of this application;

[0036] Figure 2The diagram illustrates a training process for an initial decision model according to an embodiment of this application.

[0037] Figure 3 A schematic diagram of a drilling assistance decision apparatus according to an embodiment of this application is shown. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0039] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0040] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0041] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0042] Figure 1 The illustration schematically shows a flowchart of a drilling assistance decision-making method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides a drilling auxiliary decision-making method, which may include the following steps S110-S150.

[0043] Step S110: Obtain first real-time operation data, wherein the first real-time operation data includes the first real-time operating parameters of the drilling equipment and the first real-time drilling data.

[0044] In this embodiment of the application, the application first collects first real-time operation data, including the first real-time operating parameters of the drilling equipment and the first real-time drilling data, through automated equipment during the drilling operation, and performs preprocessing.

[0045] Specifically, in one optional implementation, the first real-time operating parameters of the drilling equipment collected may include set bit pressure, set rotation speed, set displacement, drilling fluid parameters, drill bit wear, equipment operating status, etc.; while the first real-time drilling data collected may include downhole bit pressure, downhole rotation speed, downhole displacement, formation information, well depth, geological conditions, etc. Preprocessing operations may include outlier / missing value handling, noise reduction, normalization / standardization, etc., thereby transforming the collected data into an input dataset that can be input into the initial decision model.

[0046] By collecting the above-mentioned data, the actual operation of the drilling equipment by the driller and the real-time status of the downhole working environment can be reflected, thus providing a basis for the initial decision-making model to determine the initial decision.

[0047] Step S120: Input the first real-time job data into the initial decision model to obtain the initial decision, wherein the initial decision model is obtained based on behavior cloning.

[0048] In this embodiment of the application, an initial decision model based on historical data for behavioral cloning training is used to process the first real-time data to obtain an initial decision, which is then provided to the driller for reference.

[0049] Specifically, in one alternative implementation, the initial decision model can provide the driller with adjustment decisions for parameters such as drilling equipment pressure, rotational speed, and displacement.

[0050] The initial decisions determined by the initial decision model can provide basic decision-making references and assistance to the driller, helping the driller to better achieve the potential engineering goals judged by the initial decision model at this stage.

[0051] Step S130: Obtain the second real-time operation data of the drilling equipment, wherein the second real-time operation data is obtained after the driller determines the actual operation based on the initial decision and applies the actual operation to the drilling equipment.

[0052] In this embodiment, after learning the initial decision recommended by the initial decision model, the driller can determine the actual operation to be applied to the drilling equipment based on the initial decision, the driller's engineering experience, and the actual engineering objectives. This results in second real-time operational data, including second real-time operating parameters of the drilling equipment and second real-time drilling data. The second real-time operating parameters of the drilling equipment may include set bit pressure, set rotation speed, set displacement, drilling fluid parameters, drill bit wear, and equipment operating status. The second real-time drilling data may include downhole bit pressure, downhole rotation speed, downhole displacement, formation information, well depth, and geological conditions.

[0053] Specifically, in one optional implementation, the driller can choose to adopt or not adopt the initial decision, or choose to adopt one or more parameter types recommended for adjustment in the initial decision, or choose to adopt specific parameter values ​​recommended for adjustment for one or more parameter types in the initial decision. Simultaneously, the actual operations performed by the driller on the drilling equipment will be accurately reflected in the second real-time operation data.

[0054] By analyzing and processing second-real-time operational data, including the second-real-time operating parameters of drilling equipment and second-real-time drilling data, the decision optimization model can more accurately determine the driller's decisions on actual drilling operations under different downhole environments and engineering objectives. This, in turn, can improve the processing speed and accuracy of the decision optimization model through reinforcement learning.

[0055] Step S140: Determine the first intent label based on the second real-time operation data, wherein the first intent label is used to characterize the engineering goal of the actual operation.

[0056] In this embodiment of the application, a first intent label is determined to characterize the engineering objective of the actual operation by using the second real-time operation data generated by the actual operation of the driller.

[0057] Specifically, in one optional implementation, an algorithm is used to analyze the changes in drilling pressure, rotational speed, and displacement in the second real-time operation data to identify abrupt changes in these parameters. A mutation vector is then determined based on the parameter changes, and subsequently, a first intention label is determined based on this mutation vector. One possible method is to use the CUSUM algorithm to identify the mutation point. When the algorithm determines that a parameter has undergone a mutation, it is considered that "the driller has made a decision."

[0058] By using the first intent label, the decision optimization model can more accurately understand the engineering objectives corresponding to the actual operations performed by the driller, thereby providing assistance in determining the optimal decision.

[0059] Step S150: Input the second real-time job data and the first intent label into the decision optimization model to obtain the optimization decision, wherein the decision optimization model is based on reinforcement learning.

[0060] In this embodiment, the application uses an initial decision model as a basis and a decision optimization model obtained by reinforcement learning (RL) as a basis, and uses second real-time job data and first intent label as input to determine the optimization decision.

[0061] By inputting the second real-time operation data, which reflects the current actual drilling operation status, and the first intention label, which reflects the driller's actual operation and the target, into the decision optimization model, the determined optimized decision can be more in line with the actual operation requirements compared with the initial decision.

[0062] Through the above technical solution, this application obtains an initial decision based on an initial decision model obtained using behavior cloning, based on the first real-time operation data including the first real-time working parameters of the drilling equipment and the first real-time drilling data; then, based on the second real-time operation data generated after the driller determines the actual operation and applies it, a first intent label for representing the engineering objective of the actual operation is determined; based on the second real-time operation data and the first intent label, and with the help of a decision optimization model based on reinforcement learning, an optimized decision that is more in line with the actual operation requirements, more adaptive, and more intelligent is finally determined more efficiently.

[0063] Figure 2 The diagram illustrates a training process for an initial decision model according to an embodiment of this application. Figure 2 As shown in the figure, this application embodiment provides a training process for an initial decision model, which may include the following steps S210-S250.

[0064] Step S210: Obtain historical operation data and corresponding historical decision information, wherein the historical operation data includes the historical operating parameters of the drilling equipment and historical drilling data.

[0065] In this embodiment, various types of data generated during historical operations can be collected and preprocessed to convert them into training datasets. Behavioral cloning is then used to train a pre-defined neural network. Historical operation data includes records of historical drilling equipment parameters such as set drilling pressure, set rotation speed, set displacement, drilling fluid parameters, drill bit wear, and equipment operating status. Historical drilling data records downhole drilling pressure, downhole rotation speed, downhole displacement, formation information, well depth, and geological conditions. Simultaneously, the driller's parameter adjustments in the historical operation data are mapped to historical decisions and labeled with engineering objectives, such as "avoiding equipment wear" or "increasing mechanical drilling speed."

[0066] In this embodiment of the application, the historical decision information includes multiple historical decisions made within a preset distance of the drilling equipment movement.

[0067] Specifically, in historical drilling data, due to formation heterogeneity, hydraulic system delays, and automatic control compensation, successive changes in multiple historical parameters within a short period may represent the same historical decision made by the driller; while the drilling process is still ongoing and the equipment is still moving. Therefore, in this embodiment, a preset distance for the movement of the drilling equipment is set to accurately determine the driller's historical decision; one possible value is 3 meters.

[0068] Step S220: Determine a second intent label based on historical drilling data, wherein the second intent label is used to characterize the motivation of historical decision information.

[0069] In this embodiment, an algorithm is used to analyze changes in downhole drilling pressure, downhole rotation speed, and downhole displacement in historical drilling data to identify abrupt changes in these parameters. A mutation vector is then determined based on the parameter changes, and subsequently, a second intent label is determined based on this mutation vector. One possible method is to use the CUSUM algorithm to identify the mutation point. When the algorithm detects a mutation in any parameter, it is considered that "the driller has made a decision."

[0070] Step S230: Input the historical task data, the corresponding historical decision information, and the second intent label into the preset neural network to obtain the prediction decision information corresponding to the historical task data.

[0071] In this embodiment of the application, in the imitation learning, behavioral cloning uses supervised learning to enable a preset neural network to imitate the decision-making behavior of the driller. Specifically, it can learn the operating patterns by analyzing the driller's decisions (such as adjusting drilling pressure, rotation speed, and displacement) by inputting historical operation data and corresponding historical decision information; the training objective can be to determine whether the driller has made a decision in the next moment, what decision the driller has made, and what the parameter changes are corresponding to the decision.

[0072] In one possible embodiment, the predictive decision information includes whether the driller makes a decision, the type of working parameter to be adjusted corresponding to the decision, and the adjustment value for each type of working parameter.

[0073] The first step is a binary classification problem, which processes historical drilling data to output a probability value between 0 and 1, representing the probability that the driller will make a decision adjustment at the next moment (which can be the next meter). When the probability is greater than a set value, it is considered that an operation is required. In one embodiment, the set value can be 0.8.

[0074] The second step is a multi-label classification problem, sharing some feature extraction layers with the first step. Since actual drilling operations may require adjusting the parameters of multiple drilling rigs simultaneously, three non-mutually exclusive independent sigmoid output layers are used to determine whether the parameters of each drilling rig have been adjusted.

[0075] The third step is a multi-objective regression problem, which also shares the feature extraction layer. Given that the decision parameter types have been determined, this step outputs the parameter changes of the drilling equipment. This step can have a gating mechanism; that is, if the second step determines that a parameter has not been adjusted, the rate of change of that parameter is forced to zero in the third step.

[0076] For both the second and third steps, physical constraints based on drilling engineering experience are introduced to prevent safety issues arising from the parameter adjustment methods. For example, when the formation data indicates a "salt-gypsum layer," reducing the discharge rate is strictly prohibited.

[0077] Therefore, by inputting historical task data, corresponding historical decision information, and second intent tags, predictive decision information corresponding to the historical task data can be obtained.

[0078] Step S240: Based on historical decision information and predicted decision information, calculate the loss value corresponding to the historical operation data based on the first loss function.

[0079] In the embodiments of this application, it is understood that the predicted decision information during the training process may not necessarily match the historical decision information. Therefore, after each training session, the predicted decision information is compared with the historical decision information, and the accuracy of the model is evaluated using a first loss function.

[0080] In this embodiment of the application, the first loss function is obtained by weighted summation of decision occurrence judgment loss, decision type loss and decision parameter adjustment value loss, wherein the decision occurrence judgment loss is the binary crossentropy loss function, the decision type loss is the multi-label cross entropy loss function, and the decision parameter adjustment value loss is the mean squared error loss function (MSE).

[0081] Step S250: With the goal of minimizing the loss value, iteratively train the preset neural network until the preset first training termination condition is met, then terminate the training to obtain the initial decision model.

[0082] In this embodiment, the training process is validated and tested with the goal of minimizing the loss value. The training parameters and algorithm structure are adjusted iteratively until a preset first training termination condition is met, at which point training terminates, resulting in an initial decision model. The first training termination condition can be that the loss value is less than a preset loss value threshold, or that the number of training iterations reaches a preset training iteration threshold.

[0083] Through the above technical solution, this application uses historical operation data and corresponding historical decision information to perform behavioral cloning training on a preset neural network, imitates the driller's behavior under supervised learning, and then uses the first loss function for verification and evaluation, finally obtaining an initial decision model with high accuracy.

[0084] In one optional implementation, step S150 may include the following steps S151-S153.

[0085] Step S151: Using the second real-time job data as the state space and the first intention label as the state enhancement of the state space, input it into the decision optimization model to determine the target action from the action space, wherein the action space includes the changing state of the target working parameters.

[0086] Those skilled in the art will understand that in reinforcement learning, a state is a description of an agent's interaction with its environment, while an action is an operation that the agent can perform in a specific state. In the embodiments of this application, the second real-time job data is first preprocessed, including handling outliers / missing values, noise reduction, normalization / standardization, etc., and then modeled and quantized to serve as input to the state space; at the same time, a first intent label is introduced to represent the engineering goal behind the actual operation, as a state enhancement of the state space, and is also input into the decision optimization model.

[0087] The decision optimization model is based on the initial decision model and uses reinforcement learning to determine the target action in the action space based on the input state space. The action space includes the changing states of the target working parameters.

[0088] Step S152: Based on the action space and the first intention label, determine the expected total reward using the reward function and the second loss function.

[0089] In one possible embodiment of this application, the action space includes, but is not limited to, the following aspects: keeping the drilling pressure constant, increasing, or decreasing the drilling bit rotation speed constant, increasing, or decreasing the mud pump displacement constant, increasing, or decreasing the drilling equipment displacement. Within the action space and in conjunction with a first intent label, the expected total reward is calculated using a reward function and a second loss function.

[0090] In this embodiment, an innovative dual-track reward function design is proposed. The reward function is obtained by weighted summation of the consistency reward between the initial decision and the actual operation, and the consistency reward between the actual operation and the first intention label. Specifically, the consistency reward between the initial decision and the actual operation represents the consistency between the initial decision obtained from the first real-time operation data through the initial decision model and the actual operation applied to the drilling equipment after the driller references the initial strategy and the actual engineering goal; consistency is assigned a value of 1, and inconsistency is assigned a value of 0. The consistency reward between the actual operation and the first intention label represents the consistency between the driller's actual operation and the first intention label obtained from the second real-time data; consistency is assigned a value of 1, and inconsistency is assigned a value of 0. This dual-track reward function design encourages the reinforcement learning model to learn more about the driller's decision-making tendencies and enhances the efficiency of optimizing decisions to achieve the final engineering goal.

[0091] It is understood that the weights of the consistency reward between the initial decision and the actual operation, and the consistency reward between the actual operation and the first intention label, can be set according to actual needs, and this application embodiment does not limit this. Illustratively, the consistency reward between the initial decision and the actual operation can be 0.3, and the consistency reward between the actual operation and the first intention label can be 0.7.

[0092] By setting the reward function, the model can balance various parameters and objectives in the task during training, thereby improving its adaptability and optimizing the overall efficiency and safety of the task.

[0093] In this embodiment of the application, the second loss function includes:

[0094]

[0095] in, L RL Let Q be the second loss function, representing the expected total reward that can be obtained by taking a certain action in a certain state. Let γ represent the expected value of the difference between the Q values ​​for all states s and actions a, where R is the actual reward obtained by taking an action in the current state, and γ is the discount factor. Q(s,a) is the maximum Q value that the agent takes in the next state s', and Q(s,a) is the Q value of the agent taking action a in state s.

[0096] Through the second loss function, the model can continuously improve its prediction accuracy, thereby gradually improving the accuracy and adaptability of the optimized decision.

[0097] Step S153: Determine the optimal decision with the goal of maximizing the expected total reward.

[0098] In this embodiment, the decision optimization model is updated by calculating the temporal difference (TD) error to determine the error between the expected total reward that the decision optimization model can obtain by taking a certain action in a certain state and the actual reward obtained, thereby determining the optimal decision.

[0099] According to the above technical solution, by simultaneously inputting the second real-time operation data and the first intention label into the reinforcement learning-based decision optimization model, and with the help of the reward function and the second loss function, a highly intelligent and adaptable decision optimization model is finally formed through training. This model can provide drillers with accurate, adaptable and interpretable decision support.

[0100] Figure 3 A schematic diagram of a drilling assistance decision-making apparatus according to an embodiment of this application is shown. Figure 3 As shown, this application embodiment provides a controller 300, which may include:

[0101] Memory 310 is configured to store instructions; and

[0102] Processor 320 is configured to retrieve instructions from memory 310 and, when executing the instructions, to implement the aforementioned method for drilling auxiliary decision-making.

[0103] This application also provides a machine-readable storage medium storing instructions for causing a machine to perform the above-described drilling auxiliary decision-making method.

[0104] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0105] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0106] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0107] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0108] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0109] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0110] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0111] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0112] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A drilling-aided decision-making method, characterized in that, include: Acquire first real-time operation data, wherein the first real-time operation data includes first real-time operating parameters of the drilling equipment and first real-time drilling data; The first real-time operation data is input into the initial decision model to obtain an initial decision, wherein the initial decision model is obtained based on behavior cloning; Acquire second real-time operation data of the drilling equipment, wherein the second real-time operation data is obtained after the driller determines the actual operation based on the initial decision and applies the actual operation to the drilling equipment; Based on the second real-time operation data, a first intent tag is determined, wherein the first intent tag is used to characterize the engineering goal of the actual operation; Based on reinforcement learning, the second real-time job data and the first intent label are input into the decision optimization model to obtain an optimized decision; The training process of the initial decision model includes: Acquire historical operation data and corresponding historical decision information, wherein the historical operation data includes the historical operating parameters of the drilling equipment and historical drilling data; Based on the historical drilling data, a second intent label is determined, wherein the second intent label is used to characterize the motivation of the historical decision information; The historical task data, the corresponding historical decision information, and the second intent label are input into a preset neural network to obtain the predictive decision information corresponding to the historical task data. Based on the historical decision information and the predicted decision information, and using the first loss function, the loss value corresponding to the historical operation data is calculated. With the goal of minimizing the loss value, the preset neural network is trained iteratively until a preset first training termination condition is met, at which point the training is terminated to obtain the initial decision model.

2. The method according to claim 1, characterized in that, The predictive decision information includes whether the driller makes a decision, the type of working parameter that needs to be adjusted corresponding to the decision, and the adjustment value for each type of working parameter.

3. The method according to claim 1, characterized in that, The first loss function is obtained by weighted summation of decision occurrence judgment loss, decision type loss, and decision parameter adjustment value loss, wherein the decision occurrence judgment loss is a cross-entropy loss function, the decision type loss is a multi-label cross-entropy loss function, and the decision parameter adjustment value loss is a mean squared error loss function.

4. The method according to claim 1, characterized in that, It also includes the historical decision information, which includes multiple historical decisions made within a preset distance of the drilling equipment's movement.

5. The method according to claim 1, characterized in that, The step of using reinforcement learning to input the second real-time job data and the first intent label into a decision optimization model to obtain an optimization decision includes: The second real-time job data is used as the state space, and the first intent label is used as the state enhancement of the state space. The data is then input into the decision optimization model to determine the target action from the action space, wherein the action space includes the changing states of the target working parameters. Based on the action space and the first intent label, the expected total reward is determined using a reward function and a second loss function; The optimization decision is determined with the objective of maximizing the expected total reward.

6. The method according to claim 5, characterized in that, The reward function is obtained by weighted summation of the consistency reward between the initial decision and the actual operation, and the consistency reward between the actual operation and the first intent label.

7. The method according to claim 5, characterized in that, The second loss function includes: in, L RL Let Q be the second loss function, and let Q represent the expected total reward that can be obtained by taking a certain action in a certain state. Let represent the expected difference between the Q values ​​of all states s and actions a, R is the actual reward obtained by taking an action in the current state, γ is the discount factor, maxa'Q(s',a') is the maximum Q value that the agent can take in the next state s', and Q(s,a) is the Q value that the agent can take in state s by taking action a.

8. A drilling auxiliary decision-making device, characterized in that, include: The memory is configured to store instructions; as well as A processor is configured to retrieve the instructions from the memory and, when executing the instructions, to implement the drilling assistance decision-making method according to any one of claims 1 to 7.

9. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform the drilling auxiliary decision-making method according to any one of claims 1 to 7.

Images