A data lake informatization scene construction sequencing method

Abstract

The application discloses a data lake informatization scene construction sequencing method, which comprises a basic scene and influence factors, determines each influence factor as a positive parameter or a negative parameter, carries out expert scoring on the influence factors, adopts a weighted average scoring method, normalizes the proportion weight according to the job level of the experts, carries out multiplication summation on the weight and the corresponding score, and obtains a basic scene expert weighted average scoring table; according to the basic scene expert weighted average scoring table, the weight of each influence factor is determined by adopting an entropy weight method model, data solving is carried out by combining a TOPSIS model and the entropy weight method model, and all basic scene scoring results are obtained. The application can avoid the cost increase phenomenon caused by neglecting basic scene construction in the prior art, promote the relatively simple scene construction technology, and avoid the "data island" phenomenon caused by the relatively simple scene construction technology, and avoid the low business matching degree phenomenon caused by blindly imitating peer enterprises.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a sorting method for the construction of a data lake information scenario. Background Technology

[0002] Currently, in the process of building information scenarios for data lakes, the following methods are commonly used to determine the construction order: (1) Prioritize the scenarios with the greatest business urgency based on the urgency of the needs proposed by the business departments. This method often leads to resources being tilted towards high-attention projects, while neglecting the construction of basic and supportive scenarios, resulting in a loose data architecture and increased integration costs; (2) Assess the implementation difficulty based on the technical difficulty assessment by the technical team, and prioritize projects with relatively simple technology and short cycles to quickly demonstrate results. This method is prone to creating "data silos" scenarios, lacking overall data system planning, and in the long run, it will increase the difficulty of system integration and governance; (3) Imitate the construction order of common data application scenarios of companies in the same industry. Since different companies have different data foundations, organizational structures and strategic directions, simply applying the same method may result in low fit between the scenario and its own business, and an unsatisfactory return on investment. Summary of the Invention

[0003] The purpose of this application is to provide a sorting method for the construction of data lake information scenarios, so as to solve the problems mentioned in the background technology.

[0004] The present invention solves its problems through the following technical solution: A method for prioritizing the construction of data lake information scenarios includes the following steps: S1. Statistical data: All basic scenarios that need to be built for the data lake, and summarize the influencing factors that need to be considered for the current basic scenarios. Determine whether each influencing factor is a positive parameter or a negative parameter. The higher the value of the positive parameter, the better, and the higher the value of the negative parameter, the worse. S2. Using the influencing factors collected in step S1, experts score the basic scenario construction in their own field. When facing multiple experts, a weighted average scoring method is used. The weights are normalized according to the experts' job levels, and the weights are multiplied and summed with the corresponding scores to obtain the basic scenario expert weighted average scoring table. S3. Based on the basic scenario expert weighted average scoring table obtained in step S2, the weight of each influencing factor is determined using the entropy weight method model. The data is solved by combining the TOPSIS model with the entropy weight method model. In the TOPSIS model, the influencing factors are either positive or negative parameters, which has been determined in step S1. All basic scenario scoring results are obtained.

[0005] Optionally, in step S2, the weighted average score assigned by the experts includes the following steps: S21: Normalize the weighting of experts by their job level, and sum the weights by multiplying them by their corresponding scores. The normalized weighting equation is as follows: (1) Where, S i =10-v i v i T represents the rank of the i-th expert. i This represents the normalized percentage weight of the i-th expert. S22: Multiply and sum the weights of each expert and their scores. The formula is as follows: (2) Among them, E i G represents the weighted score of the i-th expert. i Let Q represent the initial score of the i-th expert, and let Q = E1 + E2 + ... + E n .

[0006] Optionally, in step S3, the entropy weight method model is used to explore the weights of influencing factors. Based on the entropy weight method model, the comprehensive evaluation model TOPSIS is used for further analysis to obtain the final ranking results. The overall formula derivation process is as follows: S31: Define information entropy. Suppose x represents one possible outcome of event X, and p(x) represents the probability of this outcome occurring. Define I(x) = -ln(p(x)). Since p(x) is between 0 and 1, I(x) is greater than 0. If the possible outcomes of the events are x1, x2, ..., x... n Then the information entropy of event X is defined as: (3) S32: Standardization process: ,in The standardized value of the j-th influencing factor for the i-th unit after dimensionless processing; This represents the original value of the j-th influencing factor for the i-th unit. S33: Define the difference coefficient d j =1-H(X), where d j The larger the value, the greater the information utility provided by the influencing factor, and the more important its role in the evaluation. S34: Normalize the difference coefficients to obtain the weight of each influencing factor: (4) S35: Constructing the scoring formula in the TOPSIS model: (5) in Represents the value of an object, with max and min corresponding to the maximum and minimum values ​​in the object, respectively.

[0007] S36: Assume there are n objects to be evaluated, and m standardized matrices of influencing factors, where... x : (6) The maximum value is defined as: (7) The minimum value is defined as: (8) Define the distance between the i-th (i=1,2,...,n) evaluation object and the maximum value as: (9) Define the distance between the i-th (i=1,2,...,n) evaluation object and the minimum value as: (10) Therefore, the unnormalized score of the i-th (i=1,2,...,n) evaluation object can be calculated: (11) After summarizing the scores of all basic scenarios, they are sorted according to their scores, with the basic scenarios with higher scores being the priority scenarios for construction.

[0008] Optionally, the basic scenarios include single-well production characteristic analysis, single-well production capacity and production allocation analysis, single-well measure effect analysis, well group production characteristic analysis, well group water invasion analysis, well group reservoir connectivity analysis, well group reserve utilization degree analysis, gas field production characteristic analysis, gas field connectivity analysis, gas field dynamic reserves and potential evaluation, gas field development effect evaluation, gas field development index prediction, recoverable reserves and recovery rate prediction, gas well VFP calculation, gas well critical fluid carrying capacity calculation, gas well production string erosion risk analysis, gas well production prediction, subsea pipeline network flow analysis, and drainage gas production technology. Design and effect evaluation, unblocking process design and effect evaluation, subdivided layer process design and effect evaluation, water control and plugging process design and effect evaluation, perforation process design and effect evaluation, sand control process design and effect evaluation, geological characteristic analysis of adjustment area, reservoir characteristic analysis of adjustment area, three-dimensional geological modeling of adjustment area, development status analysis of adjustment area, production history fitting of adjustment area, residual gas distribution law analysis of adjustment area, well location design for adjustment well research, reserve utilization of adjustment area, numerical simulation analysis of adjustment area, recovery rate analysis of adjustment area, adjustment well scheme recommendation, and shallow gas and geological oil spill risk analysis.

[0009] Optionally, the influencing factors include business value, complexity, data dependency, and inter-scenario dependency, wherein business value and inter-scenario dependency are positive parameters, and complexity and data dependency are negative parameters.

[0010] In summary, the technical effects and advantages of this invention are as follows: This invention provides a scientific, systematic, and quantifiable method for prioritizing data lake information scenario construction. It avoids the increased costs caused by neglecting fundamental scenario construction, the "data silos" phenomenon resulting from promoting relatively simple scenario construction, and the low business fit caused by blindly imitating competitors—all problems inherent in traditional technologies. This method has a rigorous theoretical foundation and practical project application basis, and can effectively solve the information transformation problems faced by petroleum companies. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a data lake information scenario construction sorting method according to an embodiment of the present invention; Figure 2 This is a basic scene score graph obtained by solving the model in S3 according to an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0014] The following is an overview of the subject matter described in detail herein. This overview is not intended to limit the scope of the claims.

[0015] This embodiment aims to overcome the problems in existing technologies, such as increased costs due to neglecting the construction of basic scenarios, the "data silo" phenomenon caused by promoting the construction of relatively simple scenarios, and low business fit due to blindly imitating peer companies. It provides a scientific, systematic, and quantifiable method for prioritizing the construction of data lake information scenarios.

[0016] This embodiment proposes a ranking method for the construction of data lake information scenarios. More specifically, it involves a ranking method for the construction of data lake information scenarios based on human experience (expert scoring), entropy weight method, and TOPSIS model. Figure 1 As shown, it includes the following steps: S1. Statistical data: All basic scenarios that need to be built for the data lake, and summarize the influencing factors that need to be considered for the current basic scenarios. Determine whether each influencing factor is a positive parameter or a negative parameter. The higher the value of the positive parameter, the better, and the higher the value of the negative parameter, the worse. Optionally, the basic scenarios selected in step S1 cover the most widely used technologies in the entire development direction. These are determined jointly by the production and research departments and include the following 36 basic scenarios: single-well production characteristic analysis, single-well production capacity and production allocation analysis, single-well measure effectiveness analysis, well group production characteristic analysis, well group water invasion analysis, well group reservoir connectivity analysis, well group reserve utilization analysis, gas field production characteristic analysis, gas field connectivity analysis, gas field dynamic reserves and potential evaluation, gas field development effect evaluation, gas field development index prediction, recoverable reserves and recovery rate prediction, gas well VFP calculation, gas well critical fluid carrying capacity calculation, and gas well production tubing erosion. Risk analysis, gas well production prediction, subsea pipeline network flow analysis, drainage and gas production process design and effect evaluation, unblocking process design and effect evaluation, subdivided layer process design and effect evaluation, water control and plugging process design and effect evaluation, perforation process design and effect evaluation, sand control process design and effect evaluation, geological characteristic analysis of adjustment zone, reservoir characteristic analysis of adjustment zone, 3D geological modeling of adjustment zone, development status analysis of adjustment zone, production history fitting of adjustment zone, residual gas distribution law analysis of adjustment zone, well location design for adjustment well research, reserve utilization of adjustment zone, numerical simulation analysis of adjustment zone, recovery rate analysis of adjustment zone, adjustment well scheme recommendation, shallow gas and geological oil spill risk analysis.

[0017] Optionally, the influencing factors include business value, complexity, data dependency, and inter-scenario dependency, wherein business value and inter-scenario dependency are positive parameters, and complexity and data dependency are negative parameters.

[0018] S2. Using the influencing factors collected in step S1, experts score the basic scenario construction in their own field. When facing multiple experts, a weighted average scoring method is used. The weights are normalized according to the experts' job levels, and the weights are multiplied and summed with the corresponding scores to obtain the basic scenario expert weighted average scoring table. In step S2, the weighted average score assigned by the experts includes the following steps: S21: Normalize the weighting of experts by their job level, and sum the weights by multiplying them by their corresponding scores. The normalized weighting equation is as follows: (1) Where, S i =10-v i v i T represents the job title of the i-th expert (since the job titles are in a decreasing sequence, a forward conversion is required). i This represents the normalized percentage weight of the i-th expert. S22: Multiply and sum the weights of each expert and their scores. The formula is as follows: (2) Among them, E i G represents the weighted score of the i-th expert. i Let Q represent the initial score of the i-th expert, and let Q = E1 + E2 + ... + E n .

[0019] The corresponding score table is obtained by calculating the weighted average score in S2 (see Table 1).

[0020] Table 1. Weighted Average Score for Basic Scenarios by Experts

[0021] S3. Based on the basic scenario expert weighted average scoring table obtained in step S2, the weight of each influencing factor is determined using the entropy weight method model to avoid human subjective influence. The data is solved by combining the TOPSIS model with the entropy weight method model. In the TOPSIS model, the influencing factors are either positive or negative parameters, which have been determined in step S1. All basic scenario scoring results are obtained (the scoring order is the construction order).

[0022] In step S3, the entropy weight method model is used to explore the weights of influencing factors. Based on the entropy weight method model, the comprehensive evaluation model TOPSIS is used for further analysis to obtain the final ranking results. The overall formula derivation process is as follows: S31: Define information entropy. Suppose x represents one possible outcome of event X, and p(x) represents the probability of this outcome occurring. Define I(x) = -ln(p(x)). Since p(x) is between 0 and 1, I(x) is greater than 0. If the possible outcomes of the events are x1, x2, ..., x... n Then the information entropy of event X is defined as: (3) S32: Standardization process: ,in The standardized value of the j-th influencing factor for the i-th unit after dimensionless processing; This represents the original value of the j-th influencing factor for the i-th unit; here, "unit" refers to different content within a scenario, and "influencing factors" refers to four factors including business value and complexity. ij After processing the standardized data, in W j The system calculates weights, which are the proportions of the four categories in the final score table, and are used to multiply the indicators in the TOPSIS model.

[0023] S33: Define the difference coefficient d j =1-H(X), where d j The larger the value, the greater the information utility provided by the influencing factor, and the more important its role in the evaluation. S34: Normalize the difference coefficients to obtain the weight of each influencing factor: (4) S35: Constructing the scoring formula in the TOPSIS model: (5) in Represents the value of an object, with max and min corresponding to the maximum and minimum values ​​in the object, respectively.

[0024] S36: Assume there are n objects to be evaluated, and m standardized matrices of influencing factors, where... x : (6) The maximum value is defined as: (7) The minimum value is defined as: (8) Define the distance between the i-th (i=1,2,...,n) evaluation object and the maximum value as: (9) Define the distance between the i-th (i=1,2,...,n) evaluation object and the minimum value as: (10) Therefore, the unnormalized score of the i-th (i=1,2,...,n) evaluation object can be calculated: (11) Furthermore, after summarizing the scores of all basic scenarios, they are sorted according to their scores, with the basic scenarios with higher scores being the priority scenarios for construction.

[0025] The weights of different influencing factors are determined using the entropy weight method model (see Table 2). Note that this analysis is based on data volume analysis and can be adjusted according to the requirements of oil and gas field development and production in actual practice.

[0026] Table 2. Weighting of Different Influencing Factors in Basic Scenarios

[0027] The TOPSIS model was used to score different basic scenarios, resulting in a graph of basic scenario scores (see...). Figure 2 The top five priority construction scenarios are: dynamic reserves and potential evaluation of gas fields, single-well production characteristic analysis, single-well production capacity and production allocation analysis, residual gas distribution pattern analysis in adjustment zones, and production history fitting in adjustment zones.

[0028] This embodiment provides a data lake information scenario construction method that comprehensively considers multiple influencing factors. It possesses a certain degree of scientific rigor, systematicity, and feasibility, avoiding the blind, siloed, and inefficient prioritization problems present in existing technologies. This method has a rigorous theoretical foundation and has already been applied and validated in enterprise data lake construction projects.

[0029] Finally, it should be noted that: the accompanying drawings of the embodiments disclosed in this invention only involve structures related to the embodiments disclosed in this invention, and other structures can refer to general designs. In the absence of conflict, the same embodiment and different embodiments of this invention can be combined with each other.

[0030] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for prioritizing the construction of data lake information scenarios, characterized in that, Includes the following steps: S1. Statistical data: All basic scenarios that need to be built for the data lake, and summarize the influencing factors that need to be considered for the current basic scenarios. Determine whether each influencing factor is a positive parameter or a negative parameter. The higher the value of the positive parameter, the better, and the higher the value of the negative parameter, the worse. S2. Using the influencing factors collected in step S1, experts score the basic scenario construction in their own field. When facing multiple experts, a weighted average scoring method is used. The weights are normalized according to the experts' job levels, and the weights are multiplied and summed with the corresponding scores to obtain the basic scenario expert weighted average scoring table. S3. Based on the basic scenario expert weighted average scoring table obtained in step S2, the weight of each influencing factor is determined using the entropy weight method model. The data is solved by combining the TOPSIS model with the entropy weight method model. In the TOPSIS model, the influencing factors are either positive or negative parameters, which has been determined in step S1. All basic scenario scoring results are obtained.

2. The data lake information scenario construction ranking method according to claim 1, characterized in that, In step S2, the weighted average score assigned by the experts includes the following steps: S21: Normalize the weighting of experts by their job level, and sum the weights by multiplying them by their corresponding scores. The normalized weighting equation is as follows: （1） Where, S i =10-v i v i T represents the rank of the i-th expert. i This represents the normalized percentage weight of the i-th expert. S22: Multiply and sum the weights of each expert and their scores. The formula is as follows: （2） Among them, E i G represents the weighted score of the i-th expert. i Let Q represent the initial score of the i-th expert, and let Q = E1 + E2 + ... + E n .

3. The data lake information scenario construction ranking method according to claim 1, characterized in that, In step S3, the entropy weight method model is used to explore the weights of influencing factors. Based on the entropy weight method model, the comprehensive evaluation model TOPSIS is used for further analysis to obtain the final ranking results. The overall formula derivation process is as follows: S31: Define information entropy. Suppose x represents one possible outcome of event X, and p(x) represents the probability of this outcome occurring. Define I(x) = -ln(p(x)). Since p(x) is between 0 and 1, I(x) is greater than 0. If the possible outcomes of the events are x1, x2, ..., x... n Then the information entropy of event X is defined as: （3） S32: Standardization process: ,in The standardized value of the j-th influencing factor for the i-th unit after dimensionless processing; This represents the original value of the j-th influencing factor for the i-th unit. S33: Define the difference coefficient d j =1-H(X), where d j The larger the value, the greater the information utility provided by the influencing factor, and the more important its role in the evaluation. S34: Normalize the difference coefficients to obtain the weight of each influencing factor: （4） S35: Constructing the scoring formula in the TOPSIS model: （5） in These represent object values, with max and min corresponding to the maximum and minimum values ​​in the object, respectively. S36: Assume there are n objects to be evaluated and j standardized matrices of influencing factors, where... x : （6） The maximum value is defined as: （7） The minimum value is defined as: （8） Define the distance between the i-th (i=1,2,...,n) evaluation object and the maximum value as: （9） Define the distance between the i-th (i=1,2,...,n) evaluation object and the minimum value as: （10） Therefore, the unnormalized score of the i-th (i=1,2,...,n) evaluation object can be calculated: （11） After summarizing the scores of all basic scenarios, they are sorted according to their scores, with the basic scenarios with higher scores being the priority scenarios for construction.

4. The data lake information scenario construction ranking method according to claim 1, characterized in that, The basic scenarios include single-well production characteristic analysis, single-well production capacity and allocation analysis, single-well measure effectiveness analysis, well group production characteristic analysis, well group water invasion analysis, well group reservoir connectivity analysis, well group reserve utilization analysis, gas field production characteristic analysis, gas field connectivity analysis, gas field dynamic reserves and potential evaluation, gas field development effect evaluation, gas field development index prediction, recoverable reserves and recovery rate prediction, gas well VFP calculation, gas well critical fluid carrying capacity calculation, gas well production string erosion risk analysis, gas well production prediction, subsea pipeline network flow analysis, and drainage gas production process design. This includes evaluation of effects, design and evaluation of unblocking processes, design and evaluation of subdivided stratigraphic processes, design and evaluation of water control processes, design and evaluation of perforation processes, design and evaluation of sand control processes, analysis of geological characteristics of the adjustment zone, analysis of reservoir characteristics of the adjustment zone, 3D geological modeling of the adjustment zone, analysis of the current development status of the adjustment zone, fitting of production history of the adjustment zone, analysis of the distribution law of residual gas in the adjustment zone, well location design for adjustment wells, utilization of reserves in the adjustment zone, numerical simulation analysis of the adjustment zone, recovery rate analysis of the adjustment zone, recommendation of adjustment well schemes, and analysis of shallow gas and geological oil spill risks.

5. The data lake information scenario construction ranking method according to claim 1, characterized in that, The influencing factors include business value, complexity, data dependency, and inter-scenario dependency, with business value and inter-scenario dependency being positive parameters, and complexity and data dependency being negative parameters.

Images