Data acquisition method and device

Abstract

The disclosed embodiment discloses a data acquisition method and device, which includes storing relational data and non-relational data in a relational database and a non-relational database in the form of tables; if the fields of relational data and non-relational data belong to the same object, the fields of relational data and non-relational data are mapped to the object to obtain graphical data; after receiving a query request input by a user through an interface, at least one query field in the request is determined; a matching field that matches the query field is determined from the graphical data; the classification of the data mapped by the matching field is determined, and data is obtained from the relational database and / or the non-relational database according to the classification result. The relational database and the non-relational database are integrated, and data is obtained on demand through a unified interactive interface, which solves the problem in the related art that data cannot be obtained on demand during the front-end and back-end interaction process.

Description

Technical Field

[0001] The present disclosure relates to the technical field of data processing, and in particular to a data acquisition method and device. Background Art

[0002] In the interaction process between the front-end and back-end, the mainstream interaction method is RESTful. For different types of front-ends (for example, mobile terminals, third-party interfaces, etc.), if the RESTful interface type is used for interaction, it cannot meet the needs of different front-ends to obtain data on demand. Summary of the invention

[0003] The main purpose of the present invention is to provide a data acquisition method and device.

[0004] In order to achieve the above-mentioned purpose, according to the first aspect of the present disclosure, a data acquisition method is provided, comprising: storing relational data and non-relational data in a relational database and a non-relational database in the form of tables, respectively, wherein each table contains fields for representing data; if the fields of the relational data and the fields of the non-relational data belong to the same object, mapping the fields of the relational data and the fields of the non-relational data to the object to obtain graphical data; upon receiving a query request input by a user through an interface, determining at least one query field in the request; determining a matching field that matches the query field from the graphical data; determining the classification of the data mapped by the matching field, and acquiring data from the relational database and / or the non-relational database based on the classification result.

[0005] Optionally, the method further includes: classifying the data into relational data and non-relational data according to business requirements.

[0006] Optionally, after receiving a query request input by a user through an interface, determining at least one query field in the request includes: monitoring whether a GraphQL code for obtaining data is entered in a test page; if it is detected that the GraphQL code is entered, obtaining the query field in the code.

[0007] According to a second aspect of the present disclosure, a data acquisition device is provided, comprising: a storage unit, configured to store relational data and non-relational data in a relational database and a non-relational database in the form of tables, respectively, wherein each table contains fields for representing data; a graph data creation unit, configured to map the fields of relational data and the fields of non-relational data to the same object if the fields of relational data and the fields of non-relational data belong to the object to obtain graph data; a request response unit, configured to determine at least one query field in the request after receiving a query request input by a user through an interface; a matching unit, configured to determine a matching field that matches the query field from the graph data; and a data acquisition unit, configured to determine the classification of the data mapped by the matching field, and acquire data from the relational database and / or the non-relational database according to the classification result.

[0008] Optionally, the device further includes: a data classification unit configured to classify data into relational data and non-relational data according to business requirements.

[0009] Optionally, after receiving a query request input by a user through an interface, determining at least one query field in the request includes: monitoring whether a GraphQL code for obtaining data is entered in a test page; if it is detected that the GraphQL code is entered, obtaining the query field in the code.

[0010] Optionally, the device further comprises: a page display unit configured to display the query field and the queried data in the test page.

[0011] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, storing computer instructions, wherein the computer instructions are used to implement the data acquisition method described in any one of the implementations of the first aspect.

[0012] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the data acquisition method described in any one implementation of the first aspect.

[0013] In the data acquisition method and device of the disclosed embodiment, relational data and non-relational data are stored in a relational database and a non-relational database in the form of tables, respectively, wherein each table contains fields for representing data; if the fields of relational data and non-relational data belong to the same object, the fields of relational data and non-relational data are mapped to the object to obtain graphical data; upon receiving a query request input by a user through an interface, at least one query field in the request is determined; a matching field matching the query field is determined from the graphical data; the classification of the data mapped by the matching field is determined, and data is obtained from the relational database and / or the non-relational database according to the classification result. By integrating the relational database and the non-relational database and realizing on-demand data acquisition through a unified interactive interface, the problem in the related art that data cannot be obtained on-demand during the front-end and back-end interaction process is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the specific embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 is a flow chart of a data acquisition method according to an embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present disclosure, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present disclosure.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0020] The front end in this implementation can be a variety of different types of front ends, such as third-party interfaces, mobile terminals, etc. If the restful interaction method is adopted, the data transmission method is fixed for different front-end transmission requirements. For example, when requesting data, the third-party interface needs to transmit data corresponding to n fields at one time, and the mobile terminal needs to transmit data corresponding to m fields at one time; when m and n are different, two different restful interfaces need to be written for the two front ends to meet the transmission requirements of the front ends; or data is directly transmitted according to the only current interface, which leads to the shortcomings of insufficient data acquisition, excessive acquisition, or waterfall acquisition.

[0021] According to an embodiment of the present disclosure, a data acquisition method is provided, such as Figure 1 As shown, the method includes the following steps 101 to 102:

[0022] Step 101: storing relational data and non-relational data in a relational database and a non-relational database in the form of tables, respectively, wherein each table includes fields for representing data.

[0023] In this embodiment, in the process of acquiring data, if the types of data are different, it is necessary to implement it through hard coding in the process of acquiring data (different types of data are usually stored in different libraries, and cross-library queries require clear knowledge of which library the required data is stored in. Therefore, there will be hard coding). Therefore, this embodiment can pre-classify business data (business data involved in the interaction process) into relational data and non-relational data, and store relational data in a relational database and non-relational data in a non-relational database. The data in the database can be stored in the form of tables, and each data can correspond to one or more fields.

[0024] Step 102: If the fields of relational data and the fields of non-relational data belong to the same object, the fields of relational data and the fields of non-relational data are mapped to the object to obtain graph data.

[0025] In this embodiment, an object may have multiple fields, and these fields may be either relational data or non-relational data, so fields corresponding to all types of data may be mapped to the same object, and through this mapping, a graph data may be obtained. This graph data contains the relationship between various data.

[0026] Step 103: After receiving a query request input by the user through the interface, at least one query field in the request is determined.

[0027] In this embodiment, the code for acquiring data can be obtained from a preset interface, and the fields in the code can be determined. In this way, the purpose of determining the acquired data on demand can be achieved.

[0028] Furthermore, the interface may be a front-end interface or an interface for simulating a front-end to send a request for testing.

[0029] As an optional implementation method of this embodiment, after receiving a query request input by a user through an interface, determining at least one query field in the request includes: monitoring whether a GraphQL code for obtaining data is entered in a test page; if it is detected that the GraphQL code is entered, obtaining the query field in the code.

[0030] In this optional implementation, since the current mainstream interaction method between the front-end and the back-end adopts RESTful, and the RESTful request method has shortcomings such as insufficient acquisition, excessive acquisition, and waterfall network requests, this application adopts GraphQL as the interaction protocol between the front-end and the back-end.

[0031] The test interface may be an interface for simulating front-end (such as web or mobile) requests, through which it is possible to test whether data can be requested.

[0032] Different types of front-ends can use GraphQL to obtain data on demand, solving the shortcomings of insufficient acquisition, excessive acquisition, and waterfall network requests.

[0033] In this embodiment, a unified interface is provided to the outside through GraphQL, and the required data is acquired by the front end on demand, which solves the shortcomings of insufficient acquisition, excessive acquisition, and waterfall.

[0034] Step 104: Determine a matching field that matches the query field from the graph data.

[0035] In this embodiment, after the field of the requested data is determined, a matching field identical to the field can be determined from the graph data, and the data can be acquired through the field.

[0036] Step 105: Determine the classification of the data mapped by the matching field, and acquire data from a relational database and / or a non-relational database according to the classification result.

[0037] In this embodiment, after determining the matching field, the type of data corresponding to the request field can be determined. If the data type is relational data, the data is retrieved from the relational database. If the data type is non-relational data, the non-relational data is retrieved from the non-relational database.

[0038] As an optional implementation of this embodiment, data is classified into relational data and non-relational data according to business requirements.

[0039] In this optional implementation, if only GraphQL is used as the front-end and back-end interaction protocol, the interaction effect is not ideal when targeting relational databases. That is, when using GraphQL to obtain cross-database resources, the back-end coders need to hard-code, which causes technical problems such as inflexibility and poor scalability. Therefore, by classifying the data and storing it in different databases, and establishing an association between the two at the same time, for example, storing relational data and non-relational data separately, storing relational data in MySQL and non-relational data in ES, and establishing an association between the two to obtain graph data, it is possible to achieve flexible interaction between both non-relational and relational data.

[0040] For example, data with fixed data structure or type that does not change frequently can be classified as "relational data", and data with non-fixed data structure or type that changes frequently can be classified as "non-relational data". For example, the categories of student performance data include "mathematics, Chinese, physics, biology, and chemistry", and the categories do not change frequently, so they can be regarded as "relational data"; and the hot pot soup base data can be classified as "non-relational data" because the ingredients of each special hot pot soup base are different (some have pepper, some have star anise, some have sesame, etc.).

[0041] As an optional implementation manner of this embodiment, the method further includes: displaying the queried data in a test page.

[0042] In this optional implementation, the test page may include a code input area, a data display area, and a request trigger option. The GraphQL code may be entered in the code input area, and the request trigger selection may be triggered to realize the data request. Then, through steps 101 to 102 of this embodiment, the data query step may be realized. After the queried data is fed back to the front end, the data may be displayed in the data display area of ​​the front end.

[0043] This embodiment uses GraphQL+MySQL (relational database)+ES (non-relational database) to integrate relational databases with non-relational databases, and uses GraphQL to provide a unified interface, making the front-end and back-end separation more thorough. It improves the flexibility of front-end and back-end data interaction and meets the needs of different front-ends to obtain data on demand.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] According to an embodiment of the present disclosure, there is also provided a device for implementing the above-mentioned data acquisition method, the device comprising: a storage unit, configured to store relational data and non-relational data in a relational database and a non-relational database in the form of tables, respectively, wherein each table contains fields for representing data; a graphical data establishment unit, configured to map the fields of relational data and the fields of non-relational data to the same object if the fields of relational data and the fields of non-relational data belong to the same object, so as to obtain graphical data; a request response unit, configured to determine at least one query field in the request after receiving a query request input by a user through an interface; a matching unit, configured to determine a matching field that matches the query field from the graphical data; a data acquisition unit, configured to determine the classification of the data mapped by the matching field, and acquire data from the relational database and / or the non-relational database according to the classification result.

[0046] As an optional implementation of this embodiment, the device further includes: a data classification unit configured to classify data into relational data and non-relational data according to business requirements.

[0047] As an optional implementation method of this embodiment, after receiving a query request input by a user through an interface, determining at least one query field in the request includes: monitoring whether a GraphQL code for obtaining data is entered in a test page; if it is detected that the GraphQL code is entered, obtaining the query field in the code.

[0048] As an optional implementation of this embodiment, the device further includes: a page display unit configured to display the query field and the queried data in the test page.

[0049] The present disclosure provides an electronic device, such as Figure 2 As shown, the electronic device includes one or more processors 21 and a memory 22. Figure 2 A processor 21 is taken as an example.

[0050] The controller may further include: an input device 23 and an output device 24 .

[0051] The processor 21, the memory 22, the input device 23 and the output device 24 may be connected via a bus or other means. Figure 2 The example of connecting through bus is taken in the following.

[0052] The processor 21 may be a central processing unit (CPU). The processor 21 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. A general-purpose processor may be a microprocessor or the processor may be any conventional processor.

[0053] The memory 22 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the control method in the embodiment of the present disclosure. The processor 21 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 22, that is, the method of implementing the above method embodiment.

[0054] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of a processing device operated by the server, etc. In addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 22 may optionally include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to a network connection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0055] The input device 23 can receive input digital or character information, and generate key signal input related to user settings and function control of the processing device of the server. The output device 24 can include display devices such as a display screen.

[0056] One or more modules are stored in the memory 22, and when executed by one or more processors 21, the execution is as follows: Figure 1 The method shown.

[0057] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment method can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned motor control methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0058] Although the embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data acquisition method, It is characterized in that include: The relational data and the non-relational data are stored in a relational database and a non-relational database in the form of tables, respectively, wherein each table contains fields for representing data; If the fields of relational data and the fields of non-relational data belong to the same object, the fields of relational data and the fields of non-relational data are mapped to the object to obtain graph data; After receiving a query request input by a user through an interface, determining at least one query field in the request; Determining a matching field that matches the query field from the graphical data; The classification of the data mapped by the matching field is determined, and according to the classification result, the data is obtained from a relational database and / or a non-relational database.

2. The data acquisition method according to claim 1, It is characterized in that The method further comprises: According to business needs, data is classified into relational data and non-relational data.

3. The data acquisition method according to claim 1, It is characterized in that After receiving a query request input by a user through an interface, determining at least one query field in the request includes: Monitor whether the GraphQL code used to obtain data is entered in the test page; If the GraphQL code is detected to be input, the query field in the code is obtained.

4. The data acquisition method according to claim 3, It is characterized in that After acquiring data from the relational database and / or the non-relational database, the method further includes: Display the queried data on the test page.

5. A data acquisition device, It is characterized in that include: A storage unit configured to store relational data and non-relational data in a relational database and a non-relational database in the form of tables, respectively, wherein each table includes fields for representing data; A graph data creation unit is configured to map the fields of the relational data and the fields of the non-relational data to the same object to obtain graph data if the fields of the relational data and the fields of the non-relational data belong to the same object; a request response unit, configured to, upon receiving a query request input by a user through an interface, determine at least one query field in the request; a matching unit configured to determine a matching field matching the query field from the graph data; The data acquisition unit is configured to determine the classification of the data mapped by the matching field, and acquire data from a relational database and / or a non-relational database according to the classification result.

6. The data acquisition device according to claim 5, It is characterized in that The device also includes: The data classification unit is configured to classify data into relational data and non-relational data according to business requirements.

7. The data acquisition device according to claim 5, It is characterized in that After receiving a query request input by a user through an interface, determining at least one query field in the request includes: Monitor whether the GraphQL code used to obtain data is entered in the test page; If the GraphQL code is detected to be input, the query field in the code is obtained.

8. The data acquisition device according to claim 7, It is characterized in that The device also includes: The page display unit is configured to display the query field and the queried data on the test page.

9. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the data acquisition method according to any one of claims 1 to 4.

10. An electronic device, It is characterized in that include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the data acquisition method described in any one of claims 1-4.

Images