Data acquisition and pushing method, pushing device and data connector

Abstract

The invention provides a data collecting and pushing method and device and a data connector, and the method comprises the steps: obtaining a target service request, and determining a first target position and a second target position according to the target service request; a target first interface is determined according to the first target position, target original data are collected from a data lake through the target first interface, the target first interface is a first interface corresponding to the target original data, and the first interface is used for collecting the original data from the data lake; the target original data is stored to a second target position, a target second interface is determined according to the second target position, the target original data is sent to target software through the target second interface, and the target software is used for operating the target original data to achieve the target service request. The method solves the problems that in the prior art, data of engineering such as well drilling is complex, collection and pushing depend on labor, and efficiency is low.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular, to a method for collecting and pushing data, a pushing device, a computer-readable storage medium, and a data connector. Background Art

[0002] Through the preliminary research requirements for oilfields and the investigation and research of relevant scientific research and information literature, a feasibility analysis and prospect have been made on the application of new technologies such as digitalization and intelligentization in the oil and gas field. It is considered that the digital and intelligent technologies in the oil and gas field represent the future development direction of oilfield technologies. In the early stage, the compilation of drilling engineering designs, plans, etc. mainly relied on stand-alone software, and the data input method was to perform simulation calculations. Through the set model conditions, different working conditions and different input parameters were simulated to predict indicators and obtain optimized solutions. Due to the characteristics of difficult data collection and manual data input, there are obvious defects such as low work efficiency. Summary of the Invention

[0003] The main object of the present application is to provide a method for collecting and pushing data, a pushing device, a computer-readable storage medium, and a data connector, so as to at least solve the problem that the data in projects such as drilling in the prior art is relatively complex, and the collection and pushing rely on manual labor with low efficiency.

[0004] To achieve the above object, according to one aspect of the present application, a method for collecting and pushing data is provided, including: obtaining a target service request, and determining a first target location and a second target location according to the target service request, where the first target location is the storage location of target original data involved in implementing the target service request in a data lake, and the second target location is the storage location of the target original data in a cloud platform; determining a target first interface according to the first target location, and collecting the target original data from the data lake through the target first interface, where the target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake; storing the target original data in the second target location, and determining a target second interface according to the second target location, and sending the target original data to a target software through the target second interface, where the target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

[0005] Optionally, determining a first target location and a second target location according to the target service request includes: determining a target service type according to the target service request, where the target service type is the service type of the target service request; querying a first mapping relationship according to the target service type to obtain the target original data, where the first mapping relationship is the mapping relationship between the service type and the original data; querying a second mapping relationship according to the target original data to obtain the first target location, and querying a third mapping relationship according to the target original data to obtain the second target location, where the second mapping relationship is the mapping relationship between the original data and the storage location of the original data in the data lake, and the third mapping relationship is the mapping relationship between the original data and the storage location of the original data in the cloud platform.

[0006] Optionally, before obtaining the target service request, the method further includes: obtaining target account information, where the target account information includes the account information currently logged in to the cloud platform and password information; retrieving a first database according to the account information to determine whether there is an account in the first database that is the same as the account information, where the first database is used to store the accounts that have been authenticated on the cloud platform; in the case where there is an account in the first database that is the same as the account information, retrieving a second database according to the password information to determine whether the password corresponding to the account information in the second database is the same as the password information, where the second database is used to store the password corresponding to the account; in the case where the password corresponding to the account in the second database is the same as the password information, determining that the target service request is allowed to be submitted.

[0007] Optionally, determining a target first interface according to the first target location and collecting the target original data from the data lake through the target first interface includes: querying a fourth mapping relationship according to the first target location to obtain the target first interface, where the fourth mapping relationship is the mapping relationship between the storage location of the target original data in the data lake and the first interface; determining a target data type according to the target original data, where the target data type is the data type of the target original data; querying a fifth mapping relationship according to the target data type to obtain a target tool, where the target tool is a data collection tool for collecting the original data of the target data type, and the fifth mapping relationship is the mapping relationship between the target data type and the data collection tool; controlling the target tool to obtain the target original data by grabbing the data at the first target location of the data lake through the target first interface.

[0008] Optionally, storing the target original data in the second target location includes: querying a sixth mapping relationship according to the target data type to obtain a target model type, where the target model type is the structural type of the data model when the target original data is stored in the second target location, and the sixth mapping relationship is the mapping relationship between the data type and the structural type of the data model; converting the target original data according to the target data type to obtain a target data model and storing it in the second target location.

[0009] Optionally, determining a target second interface according to the second target location and sending the target original data to a target software through the target second interface includes: querying a seventh mapping relationship according to the target data type to obtain the target second interface, where the seventh mapping relationship is the mapping relationship between the data type and the second interface; querying an eighth mapping relationship according to the target second interface to obtain the target software, where the eighth mapping relationship is the mapping relationship between the second interface and the target software; sending the target original data to the target software through the target second interface.

[0010] Optionally, after sending the target original data to the target software through the target second interface, the method includes: receiving target derived data through the target second interface, where the target derived data is obtained by operating on the target original data by the target software; storing the target derived data in a corresponding partition of a third target location, and the target derived data in different partitions of the third target location is displayed in different areas of a display screen.

[0011] According to another aspect of the present application, there is provided a data acquisition and push device, the device includes: a first acquisition unit, configured to acquire a target service request and determine a first target location and a second target location according to the target service request, where the first target location is the storage location of target original data involved in implementing the target service request in a data lake, and the second target location is the storage location of the target original data in a cloud platform; a first determination unit, configured to determine a target first interface according to the first target location and collect the target original data from the data lake through the target first interface, where the target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake; a first storage unit, configured to store the target original data in the second target location and determine a target second interface according to the second target location, and send the target original data to a target software through the target second interface, where the target software is used to operate on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

[0012] According to another aspect of the present application, there is provided a computer-readable storage medium, which includes a stored program. When the program runs, it controls the device where the computer-readable storage medium is located to execute any one of the above methods.

[0013] According to yet another aspect of the present application, there is provided a data connector, including: one or more processors, a memory, and one or more programs. Wherein, the one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include those for executing any one of the above methods.

[0014] Applying the technical solution of the present application in the above data collection and pushing method, first, a target service request is obtained, and a first target location and a second target location are determined according to the target service request. The first target location is the storage location of the target original data involved in realizing the target service request in the data lake, and the second target location is the storage location of the target original data in the cloud platform; then, a target first interface is determined according to the first target location, and the target original data is collected from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake; after that, the target original data is stored to the second target location, and a target second interface is determined according to the second target location, and the target original data is sent to the target software through the target second interface. The target software is used to perform operations on the target original data to realize the target service request, and the second interface corresponds to the target software one by one. The present application sets up a data connector, that is, a data collaboration platform, and configures the preset interfaces and tools in the platform. When a service request is received, data is fetched from the data lake through the corresponding interface and stored in the corresponding location of the data connector. On this basis, the corresponding location is used as a data source to build a mapping with professional software. After the data is stored in the corresponding location, it is automatically sent to the corresponding target software for operation, and then the calculation result is received and displayed through the corresponding interface, realizing automatic data collection, pushing and operation, and solving the problem that the data of engineering such as drilling in the prior art is relatively complex, and the collection and pushing rely on manual work with low efficiency. Description of the Drawings

[0015] Figure 1 Shows the hardware structure block diagram of a mobile terminal for a data collection and pushing method provided in an embodiment of the present application;

[0016] Figure 2 Shows the flowchart of a data collection and pushing method provided in an embodiment of the present application;

[0017] Figure 3 The block diagram of a data acquisition and push device provided according to an embodiment of the present application is shown.

[0018] Among them, the above-mentioned drawings include the following reference numerals:

[0019] 102, processor; 104, memory; 106, transmission device; 108, input / output device. Detailed implementation manners

[0020] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0021] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data used can be interchanged under appropriate circumstances for the embodiments of the present application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0023] As introduced in the background art, in the prior art, the data operation and analysis related to oil fields rely on stand-alone software, and the input and acquisition of data rely on manual operations, resulting in low efficiency. To solve the problems that the data in engineering such as drilling in the prior art is relatively complex and the collection and push rely on manual operations with low efficiency, the embodiments of the present application provide a data collection and push method, a push device, a computer-readable storage medium, and a data connector.

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention.

[0025] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal for a data collection and push method according to an embodiment of the present invention. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1 only one is shown in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 1 shown in the figure, or have a different configuration from Figure 1 shown in the figure.

[0026] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the device information display method in the embodiments of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above-mentioned method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] In this embodiment, a method for collecting and pushing data running on a mobile terminal, a computer terminal, or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0028] Figure 2 It is a flowchart of the method for collecting and pushing data according to an embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0029] Step S201, obtain a target service request, and determine a first target location and a second target location according to the above target service request. The above first target location is the storage location of the target original data involved in realizing the above target service request in the data lake, and the above second target location is the storage location of the above target original data in the cloud platform;

[0030] Specifically, to improve the data collection rate of relevant parameters in the oil field, the present application sets up data interaction between the data bank and the database of the professional computing software through the collaboration platform, replacing manual data collection and data input. Specifically, during the data collection process, when receiving a data operation and analysis request from the staff to the collaboration platform, that is, when obtaining the above target service request, the required original data is determined according to the type of the service request, and then the storage location of the above original data in the above data bank, that is, the data lake, is determined to obtain the above first target location. On the other hand, to improve the data input efficiency of the professional software, the present application sets up a mapping relationship between the database of the collaboration platform and the database of the corresponding professional software. Data stored in different storage locations of the above collaboration platform can be directly extracted by the corresponding software, that is, the above second target location.

[0031] Step S202, determine a target first interface according to the above first target location, and collect the above target original data from the data lake through the above target first interface. The above target first interface is the first interface corresponding to the above target original data, and the above first interface is used to collect original data from the data lake;

[0032] Specifically, since the data formats corresponding to the transmitted data are different, it is necessary to call data interfaces that support different data types. To improve the data transmission rate, the present application sets different types of the above original data corresponding to different dedicated interfaces. Therefore, after determining the storage location of the above original data, the corresponding dedicated interface can be called according to the storage location to obtain the above first interface.

[0033] Step S203: Store the above-mentioned target original data at the above-mentioned second target location, determine the target second interface according to the above-mentioned second target location, and send the above-mentioned target original data to the target software through the above-mentioned target second interface. The above-mentioned target software is used to perform operations on the above-mentioned target original data to implement the above-mentioned target service request, and the above-mentioned second interface corresponds to the above-mentioned target software one by one.

[0034] Specifically, to improve the rate of pushing the above-mentioned original data to the above-mentioned professional software, this application sets up partitions in the database of the above-mentioned collaborative platform. Each corresponding partition corresponds to a different above-mentioned target software, and a corresponding interface is configured. Then, after determining the above-mentioned second target location, the dedicated data interface for the corresponding professional software to call data can be determined to obtain the above-mentioned second interface.

[0035] Through this embodiment, first, obtain the target service request, and determine the first target location and the second target location according to the above-mentioned target service request. The above-mentioned first target location is the storage location of the target original data involved in implementing the above-mentioned target service request in the data lake, and the above-mentioned second target location is the storage location of the above-mentioned target original data in the cloud platform. Then, determine the target first interface according to the above-mentioned first target location, and collect the above-mentioned target original data from the above-mentioned data lake through the above-mentioned target first interface. The above-mentioned target first interface is the first interface corresponding to the above-mentioned target original data, and the above-mentioned first interface is used to collect original data from the above-mentioned data lake. After that, store the above-mentioned target original data at the above-mentioned second target location, determine the target second interface according to the above-mentioned second target location, and send the above-mentioned target original data to the target software through the above-mentioned target second interface. The above-mentioned target software is used to perform operations on the above-mentioned target original data to implement the above-mentioned target service request, and the above-mentioned second interface corresponds to the above-mentioned target software one by one. This application sets up a data connector, that is, a data collaboration platform, and configures the preset interfaces and tools in the platform. When receiving a service request, grab data from the data lake through the corresponding interface and store it at the corresponding location of the data connector. On this basis, map the corresponding location as the data source to the professional software. After the data is stored at the corresponding location, it is automatically sent to the corresponding target software for operation, and then receive and display the calculation results through the corresponding interface, realizing automatic data collection, pushing and operation, and solving the problem that the data of drilling and other projects in the prior art is relatively complex, and the collection and pushing rely on manual work with low efficiency.

[0036] To determine the above-mentioned first target location and the above-mentioned second target location, in an optional implementation manner, the above-mentioned step S201 includes:

[0037] Step S2011: Determine the target service type according to the above-mentioned target service request. The above-mentioned target service type is the service type of the above-mentioned target service request.

[0038] Specifically, according to the above data operation request proposed by industrial personnel, the corresponding request type is determined to obtain the above target service type.

[0039] Step S2012, query the first mapping relationship according to the above target service type to obtain the above target original data, where the first mapping relationship is the mapping relationship between the above service type and the above original data;

[0040] Specifically, to improve the efficiency of data collection, the present application stores the mapping relationship between the original data required for data operation of each of the above target service request types, that is, the above target original data and the corresponding target service request type, in the above collaborative platform to obtain the first mapping relationship. When receiving the above target service request, the above target original data can be determined by querying the first mapping relationship according to the target service type of the above target service request.

[0041] Step S2013, query the second mapping relationship according to the above target original data to obtain the above first target location, and query the third mapping relationship according to the above target original data to obtain the above second target location. The second mapping relationship is the mapping relationship between the above original data and the storage location of the above original data in the above data lake, and the third mapping relationship is the mapping relationship between the above original data and the storage location of the above original data in the above cloud platform.

[0042] Specifically, to improve the efficiency of data collection, the present application sets and stores in the collaborative platform the correspondence between the storage location of the above target original data in the data lake and the above target original data to obtain the second mapping relationship, and stores in the above collaborative platform the correspondence between the storage location of the above target original data in the collaborative platform and the above target original data to obtain the third mapping relationship. Then, after determining the required above target original data, by querying the corresponding mapping relationship, the above first target location and the above second target location can be obtained.

[0043] To ensure data security, in an optional implementation manner, before obtaining the target service request, the above method further includes:

[0044] Step S301, obtain target account information, where the target account information includes the account information currently logged in to the above cloud platform and password information;

[0045] Specifically, to ensure the security of data for oilfield monitoring, the present application is set to allocate accounts to staff, and when logging in to the corresponding account, it is allowed to submit data operation requests through the above account in the above collaborative platform.

[0046] Step S302: Retrieve the first database based on the above account information to determine whether there is an account in the first database that is the same as the above account information. The first database is used to store the above accounts that have been authenticated on the above cloud platform.

[0047] Specifically, obtain the above account information of the account currently logging in to the above collaborative platform, and match it with the accounts of the users who have been authenticated on the cloud platform according to the above account information to determine whether the current account has a record in the above collaborative platform.

[0048] Step S303: In the case where there is an account in the first database that is the same as the above account information, retrieve the second database according to the above password information to determine whether the password corresponding to the above account information in the second database is the same as the above password information. The second database is used to store the above password corresponding to the above account.

[0049] Specifically, in the case where it is determined that the above account information has been authenticated in the above collaborative platform, match the input above password information with the above password information recorded in the above collaborative platform for security authentication.

[0050] Step S304: In the case where the password corresponding to the above account in the second database is the same as the above password information, determine that the above target service request is allowed to be submitted.

[0051] Specifically, in the case where the input above password information is the same as the above password information recorded in the above collaborative platform, determine that the authentication is passed, and then open the permission for the staff to submit a data operation request.

[0052] In order to determine the interface for data collection and obtain the above raw data through this interface, in an alternative embodiment, the above step S202 includes:

[0053] Step S2021: Query the fourth mapping relationship based on the above first target location to obtain the above target first interface. The fourth mapping relationship is the mapping relationship between the storage location of the above target raw data in the above data lake and the above first interface.

[0054] Specifically, in this application, the corresponding relationship between the storage location of the above target raw data in the data lake and the data interface used for data transmission, that is, the above fourth mapping relationship, is stored in the above collaborative platform. Therefore, when performing data collection, when the above first target location is determined, query the above fourth mapping relationship according to the above first target location to obtain the above first interface for data collection, and obtain the target first interface.

[0055] Step S2022: Determine the target data type according to the above-mentioned target original data, where the above-mentioned target data type is the data type of the above-mentioned target original data;

[0056] Specifically, the data scraping tools corresponding to data of different data types are different. Therefore, this application is configured to, after determining the above-mentioned target original data, determine the corresponding data type according to the above-mentioned target original data to obtain the above-mentioned target data type.

[0057] Step S2023: Query the fifth mapping relationship according to the above-mentioned target data type to obtain the target tool, where the above-mentioned target tool is the data collection tool for collecting the above-mentioned original data of the above-mentioned target data type, and the above-mentioned fifth mapping relationship is the mapping relationship between the above-mentioned target data type and the above-mentioned data collection tool;

[0058] Specifically, this application is configured to store the corresponding relationship between the above-mentioned target data type and the corresponding scraping tool, that is, the above-mentioned fifth mapping relationship, in the above-mentioned collaborative platform. Then, by querying the above-mentioned fifth mapping relationship according to the above-mentioned target data type, the data collection tool for scraping the above-mentioned target original data from the above-mentioned first target location, that is, the above-mentioned target tool, can be obtained.

[0059] Step S2024: Control the above-mentioned target tool to obtain the above-mentioned target original data by scraping the data at the first target location of the above-mentioned data lake through the above-mentioned target first interface.

[0060] Specifically, in the case of determining the above-mentioned target tool, connect the above-mentioned target first interface and the above-mentioned target tool, access the above-mentioned first target location through the above-mentioned target first interface, and scrape the above-mentioned target original data through the target tool.

[0061] To facilitate data interaction between the above-mentioned target software and the above-mentioned collaborative platform, in an optional implementation manner, the above-mentioned step S203 includes:

[0062] Step S2031: Query the sixth mapping relationship according to the above-mentioned target data type to obtain the target model type, where the above-mentioned target model type is the structural type of the data model when the above-mentioned target original data is stored at the above-mentioned second target location, and the above-mentioned sixth mapping relationship is the mapping relationship between the above-mentioned data type and the above-mentioned structural type of the above-mentioned data model;

[0063] Specifically, corresponding to different above-mentioned target software, the data structures for operation are different. Therefore, this application is configured to store the corresponding relationship between the above-mentioned target software and the data structure of the data for operation in the above-mentioned collaborative platform to obtain the above-mentioned sixth mapping relationship. Then, since the above-mentioned target software, the above-mentioned data type, and the above-mentioned second target location are in one-to-one correspondence, according to the above-mentioned target data type, the data structure of the data, that is, the above-mentioned target model type, can be queried by querying the above-mentioned sixth mapping relationship.

[0064] Step S2032: Convert the above-mentioned target original data according to the above-mentioned target data type to obtain a target data model and store it in the above-mentioned second target location.

[0065] Specifically, perform data conversion on the collected above-mentioned target original data to map the above-mentioned target original data into the corresponding data model, that is, obtain the above-mentioned target data model, and then store the above-mentioned target data model in the above-mentioned second target location.

[0066] In order to transmit the above-mentioned target data model to the above-mentioned target software, in an optional implementation manner, the above-mentioned step S203 further includes:

[0067] Step S2033: Query the seventh mapping relationship according to the above-mentioned target data type to obtain the above-mentioned target second interface, where the seventh mapping relationship is the mapping relationship between the above-mentioned data type and the above-mentioned second interface;

[0068] Specifically, the above-mentioned data models stored corresponding to different target data types are different, and the corresponding data protocols are different. Therefore, data transmission needs to be performed through different interfaces. This application sets the corresponding relationship between the above-mentioned target data model and the above-mentioned second interface in the above-mentioned system collaboration platform, that is, the above-mentioned seventh mapping relationship. Then, query the above-mentioned seventh mapping relationship according to the above-mentioned target data type to obtain the second interface for transmitting the above-mentioned target data model, and obtain the above-mentioned target second interface.

[0069] Step S2034: Query the eighth mapping relationship according to the above-mentioned target second interface to obtain the above-mentioned target software, where the eighth mapping relationship is the mapping relationship between the above-mentioned second interface and the above-mentioned target software;

[0070] Specifically, the above-mentioned second interface and the above-mentioned target software are in one-to-one correspondence. This application sets the corresponding relationship between the above-mentioned second interface and the above-mentioned target software in the above-mentioned collaboration platform to obtain the above-mentioned eighth mapping relationship. Then, query the above-mentioned eighth mapping relationship according to the above-mentioned target second interface to obtain the data transmission target of the above-mentioned target second interface, and obtain the above-mentioned target software.

[0071] Step S2035: Send the above-mentioned target original data to the above-mentioned target software through the above-mentioned target second interface.

[0072] Specifically, transmit the above-mentioned target original data converted into the above-mentioned target data model to the above-mentioned target software through the above-mentioned target second interface for data operation.

[0073] In order to facilitate the staff to perform data analysis, in an optional implementation manner, after sending the above-mentioned target original data to the target software through the above-mentioned target second interface, the method includes:

[0074] Step S401: Receive target derived data through the above-mentioned target second interface. The above-mentioned target derived data is obtained by operating the above-mentioned target original data through the above-mentioned target software;

[0075] Specifically, after transmitting the above-mentioned target original data to the above-mentioned target software, receive the operation result returned by the above-mentioned target software through the above-mentioned target second interface to obtain the above-mentioned target derived data.

[0076] Step S402: Store the above-mentioned target derived data in the corresponding partition at the third target location. The above-mentioned target derived data in different partitions at the above-mentioned third target location is displayed in different areas on the display screen.

[0077] Specifically, after the above-mentioned collaborative platform receives the above-mentioned target derived data, store the above-mentioned target derived data in the corresponding partition in the preset area and display it in the corresponding area on the display screen for the staff to view.

[0078] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0079] The embodiment of the present application also provides a data acquisition and push device. It should be noted that the data acquisition and push device in the embodiment of the present application can be used to execute the data acquisition and push method provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiment and the preferred implementation manner, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0080] The following introduces the data acquisition and push device provided in the embodiment of the present application.

[0081] Figure 3 is the structural block diagram of the data acquisition and push device according to the embodiment of the present application. As Figure 3 shown, the device includes:

[0082] The first acquisition unit 10 is used to acquire a target service request and determine a first target location and a second target location according to the above-mentioned target service request. The above-mentioned first target location is the storage location of the target original data involved in implementing the above-mentioned target service request in the data lake, and the above-mentioned second target location is the storage location of the above-mentioned target original data in the cloud platform;

[0083] Specifically, to improve the data acquisition rate of relevant parameters in the oilfield, this application sets up data interaction between the data bank and the database of professional operation software through a collaborative platform, replacing manual data acquisition and data input. Specifically, during the data acquisition process, when receiving a data operation and analysis request from a staff member to the collaborative platform, that is, when obtaining the above-mentioned target service request, the original data required is determined according to the type of the service request, and then the storage location of the above-mentioned original data in the above-mentioned data bank, that is, the data lake, is determined to obtain the above-mentioned first target location. On the other hand, to improve the data input efficiency of professional software, this application sets up a mapping relationship between the database of the collaborative platform and the database of the corresponding professional software, and data stored in different storage locations of the above-mentioned collaborative platform can be directly extracted by the corresponding software, that is, the above-mentioned second target location.

[0084] The first determination unit 20 is configured to determine a target first interface according to the above-mentioned first target location, and collect the above-mentioned target original data from the above-mentioned data lake through the above-mentioned target first interface. The above-mentioned target first interface is the first interface corresponding to the above-mentioned target original data, and the above-mentioned first interface is used to collect original data from the above-mentioned data lake.

[0085] Specifically, since the data formats corresponding to different transmitted data are different, it is necessary to call data interfaces that support different data types. To improve the data transmission rate, this application sets different dedicated interfaces for different types of the above-mentioned original data. Therefore, after determining the storage location of the above-mentioned original data, the corresponding dedicated interface can be called according to the storage location to obtain the above-mentioned first interface.

[0086] The first storage unit 30 is configured to store the above-mentioned target original data in the above-mentioned second target location, and determine a target second interface according to the above-mentioned second target location, and send the above-mentioned target original data to the target software through the above-mentioned target second interface. The above-mentioned target software is used to perform operations on the above-mentioned target original data to implement the above-mentioned target service request, and the above-mentioned second interface corresponds to the above-mentioned target software one by one.

[0087] Specifically, to improve the rate of pushing the above-mentioned original data to the above-mentioned professional software, this application sets up partitions in the database of the above-mentioned collaborative platform. Each corresponding partition corresponds to a different above-mentioned target software, and a corresponding interface is configured. Then, after determining the above-mentioned second target location, the dedicated data interface for the corresponding professional software to call data can be determined to obtain the above-mentioned second interface.

[0088] Through this embodiment, the first acquisition unit acquires a target service request, and determines a first target location and a second target location according to the target service request. The first target location is the storage location of the target original data involved in implementing the target service request in the data lake, and the second target location is the storage location of the target original data in the cloud platform. The first determination unit determines a target first interface according to the first target location, and acquires the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to acquire the original data from the data lake. The first storage unit stores the target original data in the second target location, and determines a target second interface according to the second target location, and sends the target original data to the target software through the target second interface. The target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one. This application sets up a data connector, that is, a data collaboration platform, and configures the preset interfaces and tools in the platform. When a service request is received, data is fetched from the data lake through the corresponding interface and stored in the corresponding location of the data connector. On this basis, the corresponding location is used as the data source to build a mapping with the professional software. After the data is stored in the corresponding location, it is automatically sent to the corresponding target software for operation, and then the calculation result is received and displayed through the corresponding interface, realizing the automatic acquisition and push operation of data, and solving the problem that the data of engineering such as drilling in the prior art is relatively complex, and the acquisition and push rely on manual work with low efficiency.

[0089] In order to determine the first target location and the second target location, in an alternative embodiment, the first acquisition unit includes:

[0090] A first determination module, configured to determine a target service type according to the target service request, where the target service type is the service type of the target service request;

[0091] Specifically, according to the data operation request proposed by industrial personnel, the corresponding request type is determined to obtain the target service type.

[0092] A first query module, configured to query a first mapping relationship according to the target service type to obtain the target original data, where the first mapping relationship is the mapping relationship between the service type and the original data;

[0093] Specifically, to improve the efficiency of data collection, the present application stores the mapping relationship between the original data required for data operation on each of the above target service request types, that is, the above target original data and the corresponding target service request type, in the above collaborative platform to obtain the above first mapping relationship. When receiving the above target service request, the above target original data can be determined by querying the above first mapping relationship according to the target service type of the above target service request.

[0094] A second query module, configured to query a second mapping relationship according to the above target original data to obtain the above first target location, and query a third mapping relationship according to the above target original data to obtain the above second target location. The above second mapping relationship is the mapping relationship between the above original data and the storage location of the above original data in the above data lake, and the above third mapping relationship is the mapping relationship between the above original data and the storage location of the above original data in the above cloud platform.

[0095] Specifically, to improve the efficiency of data collection, the present application sets to store the correspondence between the storage location of the above target original data in the data lake and the above target original data in the collaborative platform to obtain the above second mapping relationship, and stores the correspondence between the storage location of the above target original data in the collaborative platform and the above target original data in the above collaborative platform to obtain the above third mapping relationship. Then, after determining the required above target original data, querying the corresponding mapping relationship can obtain the above first target location and the above second target location.

[0096] To ensure data security, in an optional implementation manner, the above device further includes:

[0097] A second acquisition unit, configured to acquire target account information before acquiring a target service request. The above target account information includes the account information currently logged in to the above cloud platform and password information;

[0098] Specifically, to ensure the security of oilfield monitoring data, the present application is configured to assign accounts to staff, and allow data operation requests to be submitted through the above accounts on the above collaborative platform when logging in to the corresponding accounts.

[0099] A first retrieval unit, configured to retrieve a first database according to the above account information to determine whether there is an account in the above first database that is the same as the above account information. The above first database is used to store the above accounts that have been authenticated on the above cloud platform;

[0100] Specifically, acquire the above account information of the account currently logged in to the above collaborative platform, and match the account information of the users who have been authenticated on the cloud platform according to the above account information to determine whether the current account has a record in the above collaborative platform.

[0101] A second retrieval unit, configured to retrieve a second database according to the password information and determine whether the password corresponding to the account information in the second database is consistent with the password information in the case that there is an account in the first database that is consistent with the account information, where the second database is used to store the password corresponding to the account;

[0102] Specifically, in the case that it is determined that the account information has been authenticated in the collaboration platform, security authentication is performed by matching the input password information with the password information recorded for the account information in the collaboration platform.

[0103] A second determination unit, configured to determine that the target service request is allowed to be submitted in the case that the password corresponding to the account in the second database is consistent with the password information.

[0104] Specifically, in the case that the input password information is consistent with the password information recorded in the collaboration platform, it is determined that the authentication is passed, and then the permission to submit a data operation request is opened for the staff.

[0105] In order to determine an interface for data collection and obtain the original data through this interface, in an optional implementation manner, the first determination unit includes:

[0106] A third query module, configured to query a fourth mapping relationship according to the first target location to obtain the target first interface, where the fourth mapping relationship is the mapping relationship between the storage location of the target original data in the data lake and the first interface;

[0107] Specifically, in this application, the corresponding relationship between the storage location of the target original data in the data lake and the data interface used for data transmission, that is, the fourth mapping relationship, is stored in the collaboration platform. Therefore, when performing data collection, the first interface for data collection, that is, the target first interface, is obtained by querying the fourth mapping relationship according to the first target location in the case that the first target location is determined.

[0108] A second determination module, configured to determine a target data type according to the target original data, where the target data type is the data type of the target original data;

[0109] Specifically, the data scraping tools corresponding to data of different data types are different. Therefore, in this application, after the target original data is determined, the corresponding data type is determined according to the target original data to obtain the target data type.

[0110] A fourth query module, configured to query a fifth mapping relationship according to the above-mentioned target data type to obtain a target tool, where the target tool is a data collection tool for collecting the above-mentioned raw data of the above-mentioned target data type, and the fifth mapping relationship is a mapping relationship between the above-mentioned target data type and the above-mentioned data collection tool;

[0111] Specifically, this application sets the corresponding relationship between the above-mentioned target data type and the corresponding scraping tool in the above-mentioned collaborative platform, that is, the above-mentioned fifth mapping relationship. Then, by querying the above-mentioned fifth mapping relationship according to the above-mentioned target data type, a data collection tool for scraping the above-mentioned target raw data from the above-mentioned first target location can be obtained, that is, the above-mentioned target tool.

[0112] A collection module, configured to control the above-mentioned target tool to obtain the above-mentioned target raw data by scraping data from a first target location of the above-mentioned data lake through the above-mentioned target first interface.

[0113] Specifically, in the case of determining the above-mentioned target tool, connect the above-mentioned target first interface and the above-mentioned target tool, access the above-mentioned first target location through the above-mentioned target first interface, and scrape the above-mentioned target raw data through the target tool.

[0114] To facilitate data interaction between the above-mentioned target software and the above-mentioned collaborative platform, in an optional implementation manner, the above-mentioned first storage unit includes:

[0115] A fifth query module, configured to query a sixth mapping relationship according to the above-mentioned target data type to obtain a target model type, where the target model type is the structural type of the data model when the above-mentioned target raw data is stored in the above-mentioned second target location, and the sixth mapping relationship is a mapping relationship between the above-mentioned data type and the above-mentioned structural type of the above-mentioned data model;

[0116] Specifically, corresponding to different above-mentioned target software, the data structures for operation are different. Therefore, this application stores the corresponding relationship between the above-mentioned target software and the data structure of the data for operation in the above-mentioned collaborative platform to obtain the above-mentioned sixth mapping relationship. Then, since the above-mentioned target software, the above-mentioned data type, and the above-mentioned second target location are in one-to-one correspondence, according to the above-mentioned target data type, the data structure of the data, that is, the above-mentioned target model type, can be queried from the above-mentioned sixth mapping relationship.

[0117] A conversion module, configured to convert the above-mentioned target raw data according to the above-mentioned target data type to obtain a target data model and store it in the above-mentioned second target location.

[0118] Specifically, perform data conversion on the collected above-mentioned target raw data to map the above-mentioned target raw data into the corresponding data model, that is, obtain the above-mentioned target data model, and then store the above-mentioned target data model in the above-mentioned second target location.

[0119] In an alternative embodiment, in order to transmit the above-mentioned target data model to the above-mentioned target software, the above-mentioned first storage unit further includes:

[0120] A sixth query module, configured to query a seventh mapping relationship according to the above-mentioned target data type to obtain the above-mentioned target second interface, where the seventh mapping relationship is the mapping relationship between the above-mentioned data type and the above-mentioned second interface;

[0121] Specifically, the above-mentioned data models stored corresponding to different target data types are different, and the corresponding data protocols are different. Therefore, data transmission needs to be performed through different interfaces. In this application, the corresponding relationship between the above-mentioned target data model and the above-mentioned second interface, that is, the above-mentioned seventh mapping relationship, is stored in the above-mentioned collaborative platform. Furthermore, the seventh mapping relationship is queried according to the above-mentioned target data type to obtain the second interface for transmitting the above-mentioned target data model, and the above-mentioned target second interface is obtained.

[0122] A seventh query module, configured to query an eighth mapping relationship according to the above-mentioned target second interface to obtain the above-mentioned target software, where the eighth mapping relationship is the mapping relationship between the above-mentioned second interface and the above-mentioned target software;

[0123] Specifically, the above-mentioned second interface corresponds to the above-mentioned target software one by one. In this application, the corresponding relationship between the above-mentioned second interface and the above-mentioned target software is stored in the above-mentioned collaborative platform to obtain the above-mentioned eighth mapping relationship. Furthermore, the eighth mapping relationship is queried according to the above-mentioned target second interface to obtain the data transmission target of the above-mentioned target second interface, and the above-mentioned target software is obtained.

[0124] A sending module, configured to send the above-mentioned target original data to the above-mentioned target software through the above-mentioned target second interface.

[0125] Specifically, the above-mentioned target original data converted into the above-mentioned target data model is transmitted to the above-mentioned target software through the above-mentioned target second interface for data operation.

[0126] In an alternative embodiment, in order to facilitate data analysis by staff, the above-mentioned device includes:

[0127] A receiving unit, configured to receive target derived data through the above-mentioned target second interface after sending the above-mentioned target original data to the target software through the above-mentioned target second interface, where the target derived data is obtained by operating the above-mentioned target original data by the above-mentioned target software;

[0128] Specifically, after transmitting the above-mentioned target original data to the above-mentioned target software, the operation result returned by the above-mentioned target software is received through the above-mentioned target second interface to obtain the above-mentioned target derived data.

[0129] A second storage unit, configured to store the above-mentioned target derived data in a corresponding partition at a third target location, and the above-mentioned target derived data in different partitions of the above-mentioned third target location is displayed in different areas of a display screen.

[0130] Specifically, after receiving the above-mentioned target derived data, the above-mentioned collaborative platform stores the above-mentioned target derived data in a corresponding partition in a preset area and displays it in a corresponding area of the display screen for staff to view.

[0131] The above-mentioned data acquisition and push device includes a processor and a memory. The above-mentioned first acquisition unit, the above-mentioned first determination unit, the above-mentioned first storage unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to implement corresponding functions. The above-mentioned modules are all located in the same processor; alternatively, the above-mentioned each module is located in different processors in any combined form.

[0132] The processor contains a kernel, and the kernel retrieves corresponding program units from the memory. One or more kernels can be set, and the data acquisition and push efficiency can be improved by adjusting kernel parameters.

[0133] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in forms such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0134] An embodiment of the present invention provides a computer-readable storage medium, and the above-mentioned computer-readable storage medium includes a stored program, wherein when the above-mentioned program runs, it controls a device where the above-mentioned computer-readable storage medium is located to execute the above-mentioned data acquisition and push method.

[0135] Specifically, the data acquisition and push method includes:

[0136] Step S201, obtain a target service request, and determine a first target location and a second target location according to the above-mentioned target service request, where the above-mentioned first target location is a storage location of target original data involved in implementing the above-mentioned target service request in a data lake, and the above-mentioned second target location is a storage location of the above-mentioned target original data in a cloud platform;

[0137] Specifically, to improve the data acquisition rate of relevant parameters in the oilfield, this application sets up data interaction between the data bank and the database of professional computing software through a collaborative platform, replacing manual data acquisition and data input. Specifically, during the data acquisition process, when receiving a data operation and analysis request from the staff to the collaborative platform, that is, when obtaining the above-mentioned target service request, the required original data is determined according to the type of the service request, and then the storage location of the above-mentioned original data in the above-mentioned data bank, that is, the data lake, is determined to obtain the above-mentioned first target location. On the other hand, to improve the data input efficiency of professional software, this application sets up a mapping relationship between the database of the collaborative platform and the database of the corresponding professional software. Data stored in different storage locations of the above-mentioned collaborative platform can be directly extracted by the corresponding software, that is, the above-mentioned second target location.

[0138] Step S202: Determine the target first interface according to the above-mentioned first target location, and collect the above-mentioned target original data from the data lake through the above-mentioned target first interface. The above-mentioned target first interface is the first interface corresponding to the above-mentioned target original data, and the first interface is used to collect original data from the data lake.

[0139] Specifically, since the data formats corresponding to different transmitted data are different, it is necessary to call data interfaces that support different data types. To improve the data transmission rate, this application sets different types of the above-mentioned original data to correspond to different dedicated interfaces. Therefore, after determining the storage location of the above-mentioned original data, the corresponding dedicated interface can be called according to the storage location to obtain the above-mentioned first interface.

[0140] Step S203: Store the above-mentioned target original data in the above-mentioned second target location, and determine the target second interface according to the above-mentioned second target location. Send the above-mentioned target original data to the target software through the above-mentioned target second interface. The above-mentioned target software is used to perform operations on the above-mentioned target original data to implement the above-mentioned target service request. The above-mentioned second interface corresponds to the above-mentioned target software one by one.

[0141] Specifically, to improve the rate of pushing the above-mentioned original data to the above-mentioned professional software, this application sets up partitions in the database of the above-mentioned collaborative platform. Each corresponding partition corresponds to a different above-mentioned target software, and a corresponding interface is configured. Then, after determining the above-mentioned second target location, the dedicated data interface for the corresponding professional software to call data can be determined to obtain the above-mentioned second interface.

[0142] An embodiment of the present invention provides a processor, and the above-mentioned processor is used to run a program. Among them, when the above-mentioned program runs, it executes the above-mentioned data acquisition and pushing method.

[0143] Specifically, the data acquisition and pushing method includes:

[0144] Step S201: Obtain a target service request, and determine a first target location and a second target location according to the target service request. The first target location is the storage location of target original data involved in implementing the target service request in the data lake, and the second target location is the storage location of the target original data in the cloud platform.

[0145] Specifically, to improve the data collection rate of relevant parameters of the oilfield, this application is configured to connect the data bank and the database of the professional operation software through a collaborative platform for data interaction, replacing manual data collection and data input. Specifically, during data collection, when receiving a data operation and analysis request from a staff member to the collaborative platform, that is, when obtaining the target service request, the required original data is determined according to the type of the service request, and then the storage location of the original data in the data bank, that is, the data lake, is determined to obtain the first target location. On the other hand, to improve the data input efficiency of the professional software, this application is configured to establish a mapping relationship between the database of the collaborative platform and the database of the corresponding professional software. Data stored in different storage locations of the collaborative platform can be directly extracted by corresponding software, that is, the second target location.

[0146] Step S202: Determine a target first interface according to the first target location, and collect the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake.

[0147] Specifically, since the data formats corresponding to different transmitted data are different, it is necessary to call data interfaces supporting different data types. To improve the data transmission rate, this application is configured that different types of the original data correspond to different dedicated interfaces. Therefore, after determining the storage location of the original data, the corresponding dedicated interface can be called according to the storage location to obtain the first interface.

[0148] Step S203: Store the target original data in the second target location, and determine a target second interface according to the second target location. Send the target original data to a target software through the target second interface. The target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

[0149] Specifically, to improve the rate of pushing the above-mentioned original data to the above-mentioned professional software, this application sets up partitions in the database of the above-mentioned collaborative platform. Each corresponding partition corresponds to a different one of the above-mentioned target software, and a corresponding interface is configured. Then, after determining the above-mentioned second target position, the dedicated data interface for the corresponding professional software to call data can be determined to obtain the above-mentioned second interface.

[0150] An embodiment of the present invention provides a data connector. The data connector includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements at least the following steps:

[0151] Step S201: Obtain a target service request, and determine a first target position and a second target position according to the target service request. The first target position is the storage position of the target original data involved in implementing the target service request in the data lake, and the second target position is the storage position of the target original data in the cloud platform;

[0152] Step S202: Determine a target first interface according to the first target position, and collect the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake;

[0153] Step S203: Store the target original data in the second target position, and determine a target second interface according to the second target position. Send the target original data to the target software through the target second interface. The target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

[0154] This application also provides a computer program product. When executed on a data processing device, it is adapted to execute a program initialized with at least the following method steps:

[0155] Step S201: Obtain a target service request, and determine a first target position and a second target position according to the target service request. The first target position is the storage position of the target original data involved in implementing the target service request in the data lake, and the second target position is the storage position of the target original data in the cloud platform;

[0156] Step S202: Determine a target first interface according to the first target position, and collect the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake;

[0157] Step S203: Store the above-mentioned target original data at the above-mentioned second target location, determine the target second interface according to the above-mentioned second target location, and send the above-mentioned target original data to the target software through the above-mentioned target second interface. The above-mentioned target software is used to perform operations on the above-mentioned target original data to implement the above-mentioned target service request, and the above-mentioned second interface corresponds to the above-mentioned target software one by one.

[0158] Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described herein can be executed in a different order, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module to implement. In this way, the present invention is not limited to any specific combination of hardware and software.

[0159] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program codes.

[0160] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processors of general-purpose computers, special-purpose computers, embedded processors, or other programmable data processing devices to generate a machine, such that the instructions executed by the processors of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows or multiple flows and / or blocks Figure 1 one or more of the blocks or multiple blocks.

[0161] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions in Figure 1 one or more of the flows or multiple flows and / or blocksFigure 1 The functions specified in one or more boxes.

[0162] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the steps of the functions specified in one Figure 1 one process or more processes and / or boxes Figure 1 step of the functions specified in one or more boxes.

[0163] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0164] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0165] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can store information by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0166] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the element.

[0167] As can be seen from the above description, the above embodiments of the present application achieve the following technical effects:

[0168] 1), In the data collection and push method of the present application, first, a target service request is obtained, and a first target location and a second target location are determined according to the above target service request. The first target location is the storage location of the target original data involved in implementing the above target service request in the data lake, and the second target location is the storage location of the above target original data in the cloud platform; then, a target first interface is determined according to the above first target location, and the target original data is collected from the data lake through the above target first interface. The target first interface is the first interface corresponding to the above target original data, and the first interface is used to collect original data from the data lake; after that, the above target original data is stored in the above second target location, and a target second interface is determined according to the above second target location, and the above target original data is sent to the target software through the above target second interface. The target software is used to perform operations on the above target original data to implement the above target service request, and the second interface corresponds to the above target software one by one. The present application sets up a data connector, that is, a data collaboration platform, and configures the preset interfaces and tools in the platform. When a service request is received, data is fetched from the data lake through the corresponding interface and stored in the corresponding location of the data connector. On this basis, the corresponding location is used as a data source to build a mapping with professional software. After the data is stored in the corresponding location, it is automatically sent to the corresponding target software for operation, and then the calculation result is received and displayed through the corresponding interface, realizing automatic data collection, push operation, and solving the problem that the data of projects such as drilling in the prior art is relatively complex, and the collection and push rely on manual work with low efficiency.

[0169] 2) The data acquisition and push device of the present application. The first acquisition unit acquires a target service request and determines a first target location and a second target location according to the target service request. The first target location is the storage location of the target original data involved in implementing the target service request in the data lake, and the second target location is the storage location of the target original data in the cloud platform. The first determination unit determines a target first interface according to the first target location, and acquires the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to acquire original data from the data lake. The first storage unit stores the target original data in the second target location, and determines a target second interface according to the second target location, and sends the target original data to a target software through the target second interface. The target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one. The present application sets up a data connector, that is, a data collaboration platform, and configures the preset interfaces and tools in the platform. When a service request is received, data is fetched from the data lake through the corresponding interface and stored in the corresponding location of the data connector. On this basis, the corresponding location is used as a data source to establish a mapping with professional software. After the data is stored in the corresponding location, it is automatically sent to the corresponding target software for operation, and then the calculation result is received and displayed through the corresponding interface, realizing automatic data acquisition, push operation, and solving the problem that the data of engineering such as drilling in the prior art is relatively complex, and the acquisition and push rely on manual work with low efficiency.

[0170] The foregoing are only the preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for data collection and push, characterized in that, Including: Obtain a target service request, and determine a first target location and a second target location according to the target service request. The first target location is the storage location of target original data involved in implementing the target service request in a data lake, and the second target location is the storage location of the target original data in a cloud platform. Determine a target first interface according to the first target location, and collect the target original data from the data lake through the target first interface. The target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake. Store the target original data in the second target location, and determine a target second interface according to the second target location. Send the target original data to a target software through the target second interface. The target software is used to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

2. The method according to claim 1, wherein Determining the first target location and the second target location according to the target service request includes: Determine a target service type according to the target service request. The target service type is the service type of the target service request. Query a first mapping relationship according to the target service type to obtain the target original data. The first mapping relationship is the mapping relationship between the service type and the original data. Query a second mapping relationship according to the target original data to obtain the first target location, and query a third mapping relationship according to the target original data to obtain the second target location. The second mapping relationship is the mapping relationship between the original data and the storage location of the original data in the data lake, and the third mapping relationship is the mapping relationship between the original data and the storage location of the original data in the cloud platform.

3. The method according to claim 1, wherein Before obtaining the target service request, the method further includes: Obtain target account information, where the target account information includes the account information currently logged in to the cloud platform and password information. Retrieve a first database according to the account information to determine whether there is an account in the first database that is the same as the account information. The first database is used to store the accounts that have been authenticated on the cloud platform. In the case where there is an account in the first database that is the same as the account information, retrieve a second database according to the password information to determine whether the password corresponding to the account information in the second database is the same as the password information. The second database is used to store the password corresponding to the account. In the case where the password corresponding to the account in the second database is the same as the password information, determine that the target service request is allowed to be submitted.

4. The method according to claim 1, characterized in that Determining the target first interface according to the first target location and collecting the target original data from the data lake through the target first interface includes: Query a fourth mapping relationship according to the first target location to obtain the target first interface. The fourth mapping relationship is the mapping relationship between the storage location of the target original data in the data lake and the first interface. Determine the target data type according to the target original data, where the target data type is the data type of the target original data; Query the fifth mapping relationship according to the target data type to obtain the target tool, where the target tool is a data acquisition tool for acquiring the original data of the target data type, and the fifth mapping relationship is the mapping relationship between the target data type and the data acquisition tool; Control the target tool to obtain the target original data by grabbing data at the first target location of the data lake through the target first interface.

5. The method according to claim 4, wherein Storing the target original data in the second target location includes: Query the sixth mapping relationship according to the target data type to obtain the target model type, where the target model type is the structural type of the data model when the target original data is stored in the second target location, and the sixth mapping relationship is the mapping relationship between the data type and the structural type of the data model; Convert the target original data according to the target data type to obtain a target data model and store it in the second target location.

6. The method according to claim 4, characterized in that Determine the target second interface according to the second target location, and send the target original data to the target software through the target second interface, including: Query the seventh mapping relationship according to the target data type to obtain the target second interface, where the seventh mapping relationship is the mapping relationship between the data type and the second interface; Query the eighth mapping relationship according to the target second interface to obtain the target software, where the eighth mapping relationship is the mapping relationship between the second interface and the target software; Send the target original data to the target software through the target second interface.

7. The method according to claim 1, characterized in that, After sending the target original data to the target software through the target second interface, the method includes: Receive target derived data through the target second interface, where the target derived data is obtained by the target software operating on the target original data; Store the target derived data in the corresponding partition of the third target location, and the target derived data in different partitions of the third target location is displayed in different areas of the display screen.

8. A data acquisition and push device, characterized in that, The device includes: A first acquisition unit, configured to acquire a target service request, and determine a first target location and a second target location according to the target service request, where the first target location is the storage location of the target original data involved in implementing the target service request in the data lake, and the second target location is the storage location of the target original data in the cloud platform; A first determination unit, configured to determine a target first interface according to the first target location, and collect the target original data from the data lake through the target first interface, where the target first interface is the first interface corresponding to the target original data, and the first interface is used to collect original data from the data lake; A first storage unit, configured to store the target original data to the second target location, determine a target second interface according to the second target location, and send the target original data to a target software through the target second interface, where the target software is configured to perform operations on the target original data to implement the target service request, and the second interface corresponds to the target software one by one.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program runs, it controls the device where the computer-readable storage medium is located to execute the method according to any one of claims 1 to 7.

10. A data connector, characterized in that, Comprising: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include those for executing the method according to any one of claims 1 to 7.

Images