A data processing method and device, electronic equipment and storage medium

Abstract

The application relates to the technical field of big data, and discloses a data processing method and device, electronic equipment and a storage medium. The method comprises the following steps: obtaining a data processing request of a data requester; wherein the data processing request comprises a requested data index; obtaining target data associated with the requested data index from existing data of a data collection platform, and feeding back the target data to the data requester; wherein the existing data is obtained by processing to-be-processed data provided by a data provider according to a data type and a data processing rule provided by the data provider. The technical scheme solves the problem that different data collection platforms can only collect specific types of data and do not have high availability, improves the universality of the data collection platform, and further provides a new idea for data processing.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of big data, and in particular to a data processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] With the advent of the era of big data, a large amount of data emerges every moment, so data analysis is particularly important. Data analysis is the process of organizing data collection, data analysis and processing for a purpose. The premise of data analysis is data collection and data processing. In the data collection process, the data sources may be different, and the data formats may also be different.

[0003] In the prior art, when collecting data from different upstream systems at the same time, different data collection platforms need to be deployed. However, due to the large differences in the technical architecture of different data collection platforms, different data collection platforms can only collect specific types of data and do not have high availability. SUMMARY

[0004] The present application provides a data processing method and device, electronic equipment and storage medium to improve the high availability of the data collection platform and improve user experience.

[0005] In a first aspect, the embodiments of the present application provide a data processing method, comprising:

[0006] obtaining a data processing request of a data requester; wherein the data processing request comprises a request data index;

[0007] obtaining target data associated with the request data index from existing data of a data collection platform, and feeding back the target data to the data requester; wherein the existing data is obtained by processing to-be-processed data provided by a data provider according to a data type and a data processing rule provided by the data provider.

[0008] In a second aspect, the embodiments of the present application further provide a data processing device, comprising:

[0009] a request obtaining module configured to obtain a data processing request of a data requester; wherein the data processing request comprises a request data index;

[0010] a target data obtaining module configured to obtain target data associated with the request data index from existing data of a data collection platform, and feed back the target data to the data requester; wherein the existing data is obtained by processing to-be-processed data provided by a data provider according to a data type and a data processing rule provided by the data provider.

[0011] Thirdly, embodiments of this application also provide an electronic device, including:

[0012] One or more processors;

[0013] Memory, used to store one or more programs;

[0014] When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method provided in any embodiment of this application.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the data processing method provided in any embodiment of this application.

[0016] The technical solution of this application embodiment involves obtaining a data processing request, including requested data metrics, sent by a data requester. Then, it retrieves the target data associated with the requested data metrics from existing data on the data acquisition platform and returns the target data to the data requester. The existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider. Compared with existing technologies, this technical solution obtains different data from different data providers through a data acquisition platform, solving the problem that different data acquisition platforms can only collect specific types of data and lack high availability. This improves the versatility of the data acquisition platform and provides a new approach to data processing. Attached Figure Description

[0017] Figure 1 A flowchart of a data processing method provided in Embodiment 1 of this application;

[0018] Figure 2 A flowchart of a data processing method provided in Embodiment 2 of this application;

[0019] Figure 3 A flowchart of a data processing method provided in Embodiment 3 of this application;

[0020] Figure 4 A flowchart of a data processing method provided in Embodiment 4 of this application;

[0021] Figure 5 A flowchart of a data processing method provided in Embodiment 5 of this application;

[0022] Figure 6 This is a schematic diagram of the structure of a data processing device provided in Embodiment Six of this application;

[0023] Figure 7This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of this application. Detailed Implementation

[0024] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0025] Example 1

[0026] Figure 1 This is a flowchart illustrating a data processing method provided in Embodiment 1 of this application. This embodiment is applicable to data processing scenarios. The method can be executed by a data processing device, which can be implemented in software or hardware and integrated into an electronic device that performs data processing functions. The data processing method provided in this embodiment is applied to a data acquisition platform.

[0027] like Figure 1 As shown, the method may specifically include:

[0028] S110. Obtain the data processing request from the data requester; wherein, the data processing request includes requested data metrics.

[0029] In this context, a data requester refers to the party that needs to acquire, use, or query data; a data processing request refers to the request sent by the data requester to the data acquisition platform when it has the need to acquire, use, or query data. Furthermore, different data requesters send different data processing requests to the data acquisition platform depending on their specific needs. For example, if a data requester needs to query data, it can send a data query request to the data acquisition platform; conversely, if a data requester needs to acquire data, it can send a data acquisition request to the data acquisition platform.

[0030] Optionally, the data processing request may include a requested data indicator; the so-called data request indicator is used to uniquely identify the data to be requested, and may be a data ID.

[0031] In this embodiment, the data acquisition platform can obtain the data processing request sent by the data requester.

[0032] S120. Obtain the target data associated with the requested data indicator from the existing data of the data acquisition platform, and feed back the target data to the data requester; wherein, the existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider.

[0033] In this embodiment, the data acquisition platform refers to a platform that collects and processes data and provides it to data requesters. Furthermore, the data acquisition platform can communicate with different data providers through application programming interfaces (APIs) to obtain data from them. A data provider is a party that provides data to the data acquisition platform through an API, such as a public cloud, private cloud, the internet, or business system logs. It should be noted that the API can be a WebService interface, a Representational State Transfer (REST) ​​interface, or an Elasticsearch (ES) cluster. WebService is a fundamental component for building distributed internet systems; it is an application that provides an API that can be invoked via the web. REST interfaces offer a flexible way to expose system resources through REST-based APIs, providing data formatted in a standardized manner for different types of applications. An Elasticsearch cluster is an open-source, distributed, RESTful full-text search engine built on Lucene, and also a distributed document database where every field is indexable and searchable, allowing for the storage, searching, and analysis of large amounts of data in a very short time.

[0034] Furthermore, the data acquisition platform stores data, with each data point linked to a specific data metric. Optionally, the data in the data acquisition platform, i.e., the existing data, is obtained by processing the data to be processed provided by the data provider according to the data type and processing rules specified by the data provider. The data to be processed refers to the source data provided by the data provider to the data acquisition platform; different data providers provide different data types and processing rules for the data to be processed.

[0035] In this embodiment, as an optional approach, the existing data in the data acquisition platform can be determined in the following way: the data acquisition platform can periodically obtain data from the data provider through the API interface, process the data to be processed based on the data type and data processing rules provided by the data provider, and store the processed data in the data acquisition platform.

[0036] Furthermore, as another optional approach in this embodiment, the existing data in the data acquisition platform can also be determined in the following way: the data provider can also push data to be processed to the data acquisition platform at regular intervals (e.g., at the end of each day), and then the data acquisition platform processes the data to be processed based on the data type and data processing rules provided by the data provider, and stores the processed data in the data acquisition platform.

[0037] Specifically, after receiving a data processing request from a data requester, the system searches the existing data in the data acquisition platform based on the requested data metrics in the request to obtain the target data associated with the requested metrics, and then returns the target data to the data requester. The target data refers to the data that the data requester needs to obtain from the data acquisition platform.

[0038] It should be noted that with existing technologies, when users have the need to collect data from different upstream systems (i.e., data providers) simultaneously, different data collection platforms need to be deployed, which is costly. At the same time, due to the significant differences in the technical architecture of different data collection platforms, different data collection platforms can only collect specific types of data and do not have high availability.

[0039] This embodiment provides a universal data acquisition platform that can interact with different data providers through an application programming interface (API) to acquire different types of data. After processing the data using data processing rules, the data is stored within the data acquisition platform (e.g., in the platform's database). Therefore, when users have a need to acquire data from different data providers simultaneously, there is no need to deploy multiple data acquisition platforms, reducing costs and simplifying the data acquisition process for users.

[0040] The technical solution provided in this application involves acquiring a data processing request, including requested data metrics, sent by a data requester. Then, it retrieves the target data associated with the requested data metrics from existing data on the data acquisition platform and returns the target data to the data requester. The existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider. Compared with existing technologies, this technical solution, by acquiring different data from different data providers through a data acquisition platform, solves the problem that different data acquisition platforms can only collect specific types of data and lack high availability, thus improving the versatility of the data acquisition platform and providing a new approach to data processing.

[0041] Example 2

[0042] Figure 2This is a flowchart of a data processing method provided in Embodiment 2 of this application; based on the above embodiment, the method of "obtaining target data associated with the requested data indicator from the existing data of the data acquisition platform and feeding back the target data to the data requester" is further optimized to provide an optional implementation scheme.

[0043] like Figure 2 As shown, the method may specifically include:

[0044] S210. Obtain the data processing request from the data requester; wherein, the data processing request includes requested data metrics.

[0045] S220. Obtain the update time of the target data associated with the requested data indicator from the existing data of the data acquisition platform.

[0046] The update time refers to the update time of existing data in the data acquisition platform. It should be noted that the data acquisition platform updates the existing data periodically, and each update records the update time in a designated location associated with the existing data within the data acquisition platform. Different existing data may have different update times.

[0047] In this embodiment, the update time of the target data can be obtained from a specified location of the target data associated with the requested data indicator from the existing data of the data acquisition platform, based on the requested data indicator.

[0048] S230. Based on the update time of the target data, determine whether the target data needs to be updated. If yes, execute S240; otherwise, execute S250.

[0049] In this embodiment, it can be determined whether the target data needs to be updated based on the target data's update time and a preset update time. The preset update time is set by those skilled in the art based on actual circumstances. Specifically, if the target data's update time is a multiple of the preset update time, then the target data is determined to need updating.

[0050] Optionally, the time difference between the current time and the update time of the target data can be calculated, and then, based on the time difference and a preset update period, it can be determined whether the target data needs to be updated. The preset update period is set by those skilled in the art based on the data conditions. Specifically, if the time difference is greater than the preset update period, it is determined that the target data needs to be updated.

[0051] S240. Update the target data and send the updated target data back to the data requester.

[0052] It should be noted that while providing the updated target data to the data requester, the updated target data is also stored in the data acquisition platform.

[0053] S250: Obtain the target data associated with the requested data indicator from the existing data of the data acquisition platform, and return the target data to the data requester.

[0054] The technical solution in this embodiment obtains the update time of the target data associated with the requested data indicator from the existing data of the data acquisition platform, and then determines whether the target data needs to be updated based on the update time. By introducing update time, this technical solution ensures that the data requested by the data requester is receiving the latest data, thereby improving the user experience.

[0055] Example 3

[0056] Figure 3 This is a flowchart of a data processing method provided in Embodiment 3 of this application; based on the above embodiments, the "updating of target data" is further optimized to provide an optional implementation scheme.

[0057] like Figure 3 As shown, the method may specifically include:

[0058] S310. Obtain the data processing request from the data requester; wherein, the data processing request includes requested data metrics.

[0059] S320. Obtain the update time of the target data associated with the requested data indicator from the existing data of the data acquisition platform.

[0060] S330. Determine whether the target data needs to be updated based on the update time of the target data; if yes, execute S340; if no, execute S370.

[0061] S340. Determine the target data provider for the target data.

[0062] In this embodiment, when the data acquisition platform obtains data to be processed from the data provider, in addition to processing the data based on the data type and data processing rules provided by the data provider and storing the processed data in the data acquisition platform, it can also store the data provider identifier of the data to be processed in the data platform. The data provider identifier is used to uniquely identify the data provider and can be a string of numbers. Therefore, the target data provider is the one from whom the target data is obtained from the data acquisition platform.

[0063] Furthermore, when the data acquisition platform obtains the data to be processed from the data provider, it does not store the data provider's identifier in the data acquisition platform. As an optional approach in this embodiment, the data provider can also be parsed based on machine learning technology, according to the data type of the target data and the data processing rules.

[0064] S350. Obtain the target data to be updated from the target data provider.

[0065] In this embodiment, the data to be updated refers to the data that needs to update the target data. The data to be updated is obtained from the target data provider via an API interface. For example, the data to be updated can be in JSON or Extensible Markup Language (XML) format. JSON or XML format data is a multi-level data format, with node data in each level.

[0066] S360. Using the data processing rules associated with the data type of the target data provided by the data provider, process the data to be updated to obtain the updated target data, and then send the updated target data back to the data requester.

[0067] For example, it is determined whether the data type of the target data is a specified type; if so, new data is obtained from the data to be updated and used as the updated target data. The specified type is pre-defined by those skilled in the art. Specifically, if the data type of the target data is JSON or XML, then the data type of the data to be updated is also JSON or XML. Since the hierarchical structure of the data to be updated is clear, node data can be obtained from the specified level of the data to be updated as new data, and then this new data is used as the updated target data.

[0068] S370. Obtain the target data associated with the requested data indicator from the existing data of the data acquisition platform, and return the target data to the data requester.

[0069] The technical solution of this embodiment involves identifying the target data provider, obtaining the data to be updated from the target data provider, and then processing the data to be updated using the data processing rules associated with the data type of the target data provided by the data provider to obtain the updated target data. This technical solution, by obtaining the data to be updated, makes the updating of the target data more accurate and standardized.

[0070] Example 4

[0071] Figure 4 This is a flowchart of a data processing method provided in Embodiment 4 of this application; based on the above embodiments, the "feedback of target data to the data requester" is further optimized to provide an optional implementation scheme.

[0072] like Figure 4 As shown, the method may specifically include:

[0073] S410. Obtain the data processing request from the data requester; wherein, the data processing request includes requesting data metrics.

[0074] S420: Obtain the target data associated with the requested data indicator from the existing data of the data acquisition platform.

[0075] The existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider.

[0076] S430. Based on preset display rules, process the target data to obtain the target display data.

[0077] Among them, the preset display rules are used to process the target data and are set by those skilled in the art according to actual needs.

[0078] In this embodiment, the target data is processed by at least one of the following methods: date formatting, data scaling, data accumulation, data anonymization, and data sorting, to obtain the target display data.

[0079] For example, when the target data is a timestamp, the target data will be formatted according to the configured date display format. For example, the configured date display format could be: yyyyMMdd, yyyy-MM-dd, or HH:mm:ss.

[0080] For example, when the target data is numerical, it can be scaled. For instance, when calculating the conversion rate of a customer at a branch or displaying exchange rates, if the target data is 0.5, it can be scaled up by 100 times to become 50% and used as the target display data.

[0081] For example, when the target data consists of at least two numbers, the target data can be summed. For instance, if the data requester wants to query the annual income of a certain financial product, the target data is the monthly income of that financial product. The monthly income of each month is added together, and the result is used as the target data to be displayed.

[0082] For example, when the target data is sensitive data, such as name, ID number, or mobile phone number, the target data can be anonymized. Specifically, only part of the information can be displayed, and the rest can be replaced with *.

[0083] For example, when there are multiple target data, the target data can also be sorted according to the configured sorting rules.

[0084] For example, when the target data consists of at least two data points, calculations can be performed according to a configured formula, and the calculated result can be used as the target display data. For instance, when calculating the year-on-year and month-on-month growth rates of a sales point's revenue, the calculation can be performed according to a configured formula, and the calculated result can be used as the target display data.

[0085] S440: Provide the target display data to the data requester.

[0086] The technical solution of this embodiment processes the target data based on preset display rules to obtain the target display data, and then feeds the target data back to the data requester. This technical solution, by introducing preset display rules to process the target data, not only meets the needs of the data requester but also ensures the security of the data information.

[0087] Example 5

[0088] Figure 5 This is a flowchart of a data processing method provided in Embodiment 5 of this application; based on the above embodiments, the data provider and the requested data indicators are further verified, providing an optional implementation scheme.

[0089] like Figure 5 As shown, the method may specifically include:

[0090] S510, Obtain the data processing request from the data requester; wherein, the data processing request includes requested data metrics.

[0091] In this embodiment, the data processing request may also include the identity identifier of the data requester. The identity identifier may be the account name, account ID, etc. of the data request method, used to uniquely identify the data requester. It may also be a unique identifier assigned to the data requester by the data collection platform when the data requester registers on the data collection platform.

[0092] S520: Verify the requester's authorization and the legality of the requested data metrics.

[0093] To ensure the authenticity of the data requester, as an optional method in this embodiment, the requester's request permissions can be verified based on the data requester's identity identifier and the identity identifiers registered with the data collection platform. Specifically, the data requester's identity identifier is used as an index to search among the identity identifiers registered with the data collection platform. If the identity is found, the data requester has the right to request the data.

[0094] To ensure the legitimacy of the requested data metrics, as an optional approach in this embodiment, the data metrics can be queried from the existing data of the data acquisition platform, and the legitimacy of the requested data metrics can be determined based on the query results. Specifically, the requested data metrics can be used as an index to query the metrics in the existing data of the data acquisition platform; if found, the requested data metrics are deemed legitimate.

[0095] S530. If the permission verification and the legality verification are passed, the target data associated with the requested data indicator is obtained from the existing data of the data collection platform, and the target data is fed back to the data requester; wherein, the existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider.

[0096] The technical solution in this embodiment ensures the authenticity of the data requester and the legality of the requested data indicators by verifying the requester's permission and the legality of the data.

[0097] Example 6

[0098] Figure 6 This is a schematic diagram of a data processing device provided in Embodiment Six of this application; this embodiment is applicable to data processing situations, and the device can be implemented by software / hardware and integrated into an electronic device that carries data processing functions.

[0099] like Figure 6 As shown, the device may include a request acquisition module 610 and a target data acquisition module 620, wherein,

[0100] The request acquisition module 610 is used to acquire the data processing request from the data requester; wherein, the data processing request includes requested data indicators;

[0101] The target data acquisition module 620 is used to acquire the target data associated with the requested data indicators from the existing data of the data acquisition platform and to feed back the target data to the data requester; wherein, the existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider.

[0102] The technical solution provided in this application involves acquiring a data processing request, including requested data metrics, sent by a data requester. Then, it retrieves the target data associated with the requested data metrics from existing data on the data acquisition platform and returns the target data to the data requester. The existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider. Compared with existing technologies, this technical solution, by acquiring different data from different data providers through a data acquisition platform, solves the problem that different data acquisition platforms can only collect specific types of data and lack high availability, thus improving the versatility of the data acquisition platform and providing a new approach to data processing.

[0103] Furthermore, the target data acquisition module 620 includes an update time determination submodule, an update judgment submodule, and a target data update submodule, wherein,

[0104] The update time determination submodule is used to obtain the update time of the target data associated with the requested data indicator from the existing data of the data acquisition platform;

[0105] The update judgment submodule is used to determine whether the target data needs to be updated based on the update time of the target data;

[0106] The target data update submodule is used to update the target data if the condition is met, and then send the target data back to the data requester.

[0107] Furthermore, the update judgment submodule includes a time difference calculation unit and an update judgment unit, wherein,

[0108] The time difference calculation unit is used to calculate the time difference between the current time and the update time of the target data;

[0109] The update judgment unit is used to determine whether the target data needs to be updated based on the time difference and the preset update cycle.

[0110] Furthermore, the update judgment unit is specifically used for:

[0111] If the time difference is greater than the preset update cycle, the target data needs to be updated.

[0112] Furthermore, the target data update submodule includes a target data provider determination unit, a data acquisition unit to be updated unit, and a target data update unit, wherein,

[0113] The target data provider determination unit is used to determine the target data provider of the target data.

[0114] The data acquisition unit is used to acquire the data to be updated from the target data provider.

[0115] The target data update unit is used to process the data to be updated using the data processing rules associated with the data type of the target data provided by the data provider, so as to obtain the updated target data.

[0116] Furthermore, the target data update unit includes a specified type determination subunit and a target data update subunit, wherein,

[0117] The specified type determination sub-unit is used to determine whether the data type of the target data is a specified type;

[0118] The target data update sub-unit is used to obtain new data from the data to be updated if the condition is met, and use it as the updated target data.

[0119] Furthermore, the target data acquisition module 620 also includes a target display data acquisition unit and a target data feedback unit, wherein,

[0120] The target display data acquisition unit is used to process target data based on preset display rules to obtain target display data;

[0121] The target data feedback unit is used to provide target display data to the data requester.

[0122] Furthermore, the target data display unit is specifically used for:

[0123] Perform at least one of the following processing steps on the target data: date formatting, data scaling, data accumulation, data anonymization, and data sorting, to obtain the target display data.

[0124] Furthermore, the device also includes a verification module, which is specifically used for:

[0125] Verify the requester's permissions and the legality of the requested data metrics.

[0126] Furthermore, the verification module includes an authentication unit, which is specifically used for:

[0127] Verify the requester's authorization based on the data requester's identity and the identity registered with the data collection platform.

[0128] Furthermore, the verification module also includes a legality verification module, which comprises an indicator query unit and a legality determination unit.

[0129] The indicator query unit is used to query existing data from the data collection platform based on the requested data indicator.

[0130] The legality determination unit is used to determine the legality of the requested data indicators based on the query results.

[0131] Furthermore, the target data acquisition module 620 also includes a target data storage unit, which is specifically used for:

[0132] The updated target data is stored in the data acquisition platform.

[0133] The above-described data processing apparatus can execute the data processing method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects for executing the method.

[0134] Example 7

[0135] Figure 7 This is a schematic diagram of the structure of an electronic device provided in Embodiment 7 of this application. Figure 7 A block diagram of an exemplary device suitable for implementing embodiments of the present application is shown. Figure 7 The device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0136] like Figure 7 As shown, the electronic device 12 is represented in the form of a general-purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0137] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0138] Electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0139] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 7 Not shown; usually referred to as a "hard drive"). Although Figure 7As not shown, disk drives for reading and writing to removable non-volatile disks (e.g., "floppy disks") and optical disc drives for reading and writing to removable non-volatile optical discs (e.g., CD-ROMs, DVD-ROMs, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0140] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of this application.

[0141] Electronic device 12 can also communicate with one or more external devices 14 (e.g., keyboard, pointing device, display 24, etc.), and with one or more devices that enable a user to interact with electronic device 12, and / or with any device that enables electronic device 12 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 22. Furthermore, electronic device 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of electronic device 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0142] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the data processing method provided in the embodiments of this application.

[0143] Example 8

[0144] This application also provides a computer-readable storage medium storing a computer program (or computer-executable instructions) thereon. When executed by a processor, the program is used to perform the data processing method provided in this application embodiment, the method comprising:

[0145] Obtain the data processing request from the data requester; wherein, the data processing request includes the requested data metrics;

[0146] The system retrieves the target data associated with the requested data metrics from the existing data on the data acquisition platform and feeds the target data back to the data requester. The existing data is obtained by processing the data to be processed provided by the data provider according to the data type and data processing rules provided by the data provider.

[0147] The computer storage medium in this application embodiment can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0148] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0149] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0150] Computer program code for performing the operations of the embodiments of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although the embodiments of this application have been described in detail through the above examples, the embodiments of this application are not limited to the above embodiments. More other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A data processing method, characterized in that, include: Obtain the data processing request from the data requester; wherein, the data processing request includes requested data metrics; The update time of the target data associated with the requested data indicator is obtained from the existing data of the data acquisition platform; wherein, the existing data is obtained and stored by the data acquisition platform processing the data to be processed provided by the data provider at regular intervals according to the data type and data processing rules provided by the data provider; the data to be processed refers to the source data provided by the data provider to the data acquisition platform; the update time refers to the update time of the existing data in the data acquisition platform; different existing data have different update times. Calculate the time difference between the current time and the update time of the target data; Based on the time difference and the preset update cycle, determine whether the target data needs to be updated; If so, then determine the target data provider of the target data; Obtain the data to be updated from the target data provider; The data to be updated is processed using the data processing rules associated with the data type of the target data provided by the data provider to obtain the updated target data, and the updated target data is then fed back to the data requester. If not, then obtain the target data associated with the requested data indicator from the existing data of the data acquisition platform, and feed back the target data to the data requester; Specifically, the data processing rules associated with the data type of the target data provided by the data provider are used to process the data to be updated to obtain the updated target data, including: Determine whether the data type of the target data is the specified type; If so, new data is obtained from the data to be updated and used as the target data after the update.

2. The method according to claim 1, characterized in that, Determining whether the target data needs to be updated based on the time difference and the preset update cycle includes: If the time difference is greater than the preset update cycle, it is determined that the target data needs to be updated.

3. The method according to claim 1, characterized in that, Feedback of the target data to the data requester includes: Based on preset display rules, the target data is processed to obtain the target display data; The target display data is fed back to the data requester.

4. The method according to claim 3, characterized in that, Based on preset display rules, the target data is processed to obtain the target display data, including: The target data is processed by at least one of the following methods: date formatting, data scaling, data accumulation, data anonymization, and data sorting, to obtain the target display data.

5. The method according to claim 1, characterized in that, After obtaining the data processing request from the data requester, the process also includes: Verify the request permissions of the data requester and verify the legality of the requested data metrics.

6. The method according to claim 5, characterized in that, Verifying the request permissions of the data requester includes: The request permissions of the data requester are verified based on the identity identifier of the data requester and the identity identifier registered by the data collection platform.

7. The method according to claim 5, characterized in that, Verifying the validity of the requested data metrics includes: Based on the requested data metrics, query from the existing data on the data collection platform; Based on the query results, the requested data metrics were determined to be valid.

8. The method according to claim 1, characterized in that, Also includes: The updated target data is stored in the data acquisition platform.

9. A data processing apparatus, characterized in that, include: The request acquisition module is used to acquire the data processing request from the data requester; wherein, the data processing request includes requested data indicators; The target data acquisition module is used to acquire the target data associated with the requested data indicator from the existing data of the data acquisition platform, and to feed back the target data to the data requester; wherein, the existing data is obtained and stored by the data acquisition platform processing the data to be processed provided by the data provider at regular intervals according to the data type and data processing rules provided by the data provider; the data to be processed refers to the source data provided by the data provider to the data acquisition platform. The target data acquisition module includes: The update time determination submodule is used to obtain the update time of the target data associated with the requested data indicator from the existing data in the data acquisition platform; the update time refers to the update time of the existing data in the data acquisition platform; different existing data have different update times; The update judgment submodule is used to determine whether the target data needs to be updated based on the update time of the target data; The target data update submodule is used to update the target data if the data is true and to send the target data back to the data requester; otherwise, it retrieves the target data associated with the requested data indicator from the existing data of the data acquisition platform and sends the target data back to the data requester. The target data update submodule includes: The target data provider determination unit is used to determine the target data provider of the target data. The data to be updated acquisition unit is used to acquire the data to be updated of the target data from the target data provider; The target data update unit is used to process the data to be updated using the data processing rules associated with the data type of the target data provided by the data provider, so as to obtain the updated target data. The target data update unit includes: The specified type determination subunit is used to determine whether the data type of the target data is a specified type; The target data update subunit is used to obtain new data from the data to be updated if the condition is met, and use it as the updated target data. The update determination submodule includes: A time difference calculation unit is used to calculate the time difference between the current time and the update time of the target data; An update judgment unit is used to determine whether the target data needs to be updated based on the time difference and a preset update cycle.

10. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the data processing method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the data processing method as described in any one of claims 1-8.

Images