Data pushing method, device and equipment, and storage medium
By using the second target data to correct the first target data in the message queue during the data push process, the problem of inconsistency between the data obtained by the business and the data source was solved, and the accuracy and real-time performance of data push were achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING YOUZHUJU NETWORK TECH CO LTD
- Filing Date
- 2023-06-01
- Publication Date
- 2026-06-23
AI Technical Summary
In scenarios involving massive data queries from multiple business units, inconsistencies exist between the data obtained by the business units and the data source, leading to inaccurate data delivery.
By obtaining the first target data from the data source and pushing it to the message queue, and then using the second target data to correct the first target data in the message queue, we can ensure that the data read by the business side is consistent with the data source.
It improved the accuracy of data push, ensured that the data obtained by the business side was consistent with the data source, and enhanced the real-time performance and accuracy of the data.
Smart Images

Figure CN116756418B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of data processing technology, and in particular to a data push method, apparatus, device, and storage medium. Background Technology
[0002] In scenarios involving massive data queries from multiple business stakeholders, after subscribing to data in a specific dimension, businesses expect to obtain accurate datasets with ultra-low latency. Related technologies push data from a data source to users; however, when the data in the data source is updated, inconsistencies arise between the data obtained by the business and the data in the storage engine. Summary of the Invention
[0003] This disclosure provides a data push method, apparatus, device, and storage medium that can ensure the data pushed to the business party is consistent with the data in the data source, thereby improving the accuracy of data push.
[0004] In a first aspect, embodiments of this disclosure provide a data push method, comprising: obtaining first target data from a data source based on a data subscription request from a business party; pushing the first target data to a message queue of the business party; obtaining second target data from the data source based on the data subscription request; and correcting the first target data in the message queue based on the second target data, so that the business party reads the corrected first target data from the message queue.
[0005] Secondly, embodiments of this disclosure also provide a data push device, comprising: a first target data acquisition module, configured to acquire first target data from a data source based on a data subscription request from a business party; a data push module, configured to push the first target data to a message queue of the business party; a second target data acquisition module, configured to acquire second target data from the data source based on the data subscription request; and a correction module, configured to correct the first target data in the message queue based on the second target data, so that the business party reads the corrected first target data from the message queue.
[0006] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:
[0007] One or more processors;
[0008] Storage device for storing one or more programs.
[0009] When the one or more programs are executed by the one or more processors, the one or more processors implement the data push method as described in the embodiments of this disclosure.
[0010] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the data push method as described in embodiments of this disclosure.
[0011] In this embodiment, a first target data is obtained from a data source based on a data subscription request from a business party; the first target data is pushed to the business party's message queue; a second target data is obtained from the data source based on the data subscription request; and the first target data in the message queue is corrected based on the second target data, so that the business party can read the corrected first target data from the message queue. This embodiment, by correcting the first target data in the message queue using the second target data, ensures that the data pushed to the business party is consistent with the data in the data source, improving the accuracy of data push. Attached Figure Description
[0012] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0013] Figure 1 This is a schematic diagram of a data push method provided in an embodiment of the present disclosure;
[0014] Figure 2 This is a schematic diagram of another data push method provided in an embodiment of the present disclosure;
[0015] Figure 3 This is a schematic diagram of another data push method provided in an embodiment of the present disclosure;
[0016] Figure 4 This is a schematic diagram of the structure of a data push device provided in an embodiment of the present disclosure;
[0017] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation
[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0019] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0020] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.
[0025] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.
[0026] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.
[0027] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.
[0028] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.
[0029] Figure 1 This is a schematic diagram of a data push method provided in an embodiment of the present disclosure. The embodiments of the present disclosure are applicable to the situation where data is pushed to a business party based on the business party's data subscription request. The method can be executed by a data push device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, such as a mobile terminal, a PC, or a server.
[0030] like Figure 1 As shown, the method includes:
[0031] S110: Obtain the first target data from the data source based on the business party's data subscription request.
[0032] In this embodiment, the first target data can be understood as data subscribed to by the business party or data pushed to the business party. This embodiment does not limit the specific first target data; for example, it could be advertising stream data. In this embodiment, the business party can send a data subscription request to the data center platform, and the data center platform extracts the corresponding first target data from the data source based on the data subscription request. In this embodiment, there can be multiple business parties, and each business party uses the same data push method. The data center platform can be understood as a server. The data source can be understood as a database, and the data source can include massive amounts of data.
[0033] Optionally, obtaining the first target data from the data source based on the business party's data subscription request includes: obtaining the first target data of at least one dimension every first set time interval based on the business party's data subscription request; and aggregating the first target data of at least one dimension to obtain the aggregated first target data.
[0034] In this embodiment, based on the dimension identifier carried in the data subscription request, one or more corresponding first target data for each dimension can be retrieved from the data source at first predetermined intervals. The first target data for multiple dimensions is then aggregated to obtain the aggregated first target data. The first predetermined interval can be several minutes, and this embodiment does not limit this; for example, it could be one minute, two minutes, etc. The first target data for multiple dimensions can be understood as multiple first target data of different categories. This embodiment does not limit the aggregation method; for example, the Flink window aggregation function can be used to aggregate the first target data for multiple dimensions.
[0035] In this embodiment, by acquiring first target data from multiple dimensions and aggregating the first target data from multiple dimensions, the first target data can be accurately obtained, thereby making the data pushed to the business side more accurate.
[0036] S120. Push the first target data to the business's message queue.
[0037] The message queue can be used as a container to cache the first target data during data transmission. In this embodiment, the first target data can be pushed to the business's message queue for caching.
[0038] Optionally, the first target data can be pushed to the business's message queue, including: pushing the aggregated first target data to the business's message queue.
[0039] In this embodiment, the aggregated first target data is pushed to the business party's message queue, so that the business party can read the accurate first target data from the message queue. Furthermore, by setting up the message queue, it is possible to avoid transmitting a large amount of redundant data to the business party, thereby improving the data transmission speed and enabling the business party to obtain the first target data in a timely and accurate manner.
[0040] Optionally, before pushing the first target data to the business's message queue, the method further includes: preprocessing the first target data based on the set data processing rules.
[0041] The setting of data processing rules can be understood as rules for preprocessing the first target data. This embodiment does not limit the specific data processing rules and they can be set according to actual circumstances. Setting data processing rules may include at least one of the following: permission verification rules and data cleaning rules. The permission verification rules may include permission verification for business parties to access the first target data, and the data cleaning rules may include rules for handling missing values, duplicate values, outliers, and useless values.
[0042] In this embodiment, the permission verification rules can be used to verify whether the business party has the permission to access the first target data. Furthermore, the first target data can be cleaned using data cleaning rules. Specifically, permission verification can be performed on the business party's permissions to access multiple fields included in the first target data. If the permission verification for any field fails, that field can be discarded or deleted. The first target data that passes the permission verification is pushed to the business party's message queue, or the first target data that passes the permission verification is cleaned, and the cleaned first target data is pushed to the business party's message queue.
[0043] In this embodiment, the first target data is preprocessed using permission verification rules and data cleaning rules to make the first target data more accurate.
[0044] Optionally, before pushing the first target data to the business's message queue, the method further includes: filtering the first target data based on set business rules to obtain the filtered first target data; correspondingly, pushing the first target data to the business's message queue includes: pushing the filtered first target data to the business's message queue.
[0045] In this embodiment, the set business rules can be understood as the conditions for data push. This embodiment does not limit the specific set business rules and can be set according to actual conditions. For example, the set business rules could be that when the order quantity in the first target data reaches a set quantity, or when the sales amount in the first target data reaches a set sales amount, the corresponding first target data will be pushed to the business's message queue. In this embodiment, the first target can be filtered based on the set business rules. When the first target data meets the set business rules, the first target data that meets the set rules can be obtained, that is, the filtered first target data can be obtained, and the filtered first target data can be pushed to the business's message queue. When the first target data does not meet the set business rules, the data push operation is not performed.
[0046] In this embodiment, by setting business rules to filter the first target data and pushing the filtered first target data to the business's message queue, the first target data pushed to the message queue can be more accurate, thereby making the data read by the business from the message queue more accurate.
[0047] S130. Obtain the second target data from the data source based on the data subscription request.
[0048] It should be noted that the data subscription request can carry at least one dimension identifier, which can be used to retrieve the first or second target data for at least one dimension from the data source. The second target data can be understood as the same data as the first target data, or the updated first target data.
[0049] Optionally, obtaining the second target data from the data source based on the data subscription request includes: obtaining the second target data from the data source every second set time interval during the period from when the first target data enters the message queue to when it is read out of the message queue.
[0050] The second set duration can be the same as or different from the first set duration. For example, the second set duration can be one minute, two minutes, etc. In this embodiment, during the period from when the first target data enters the message queue to when it is read from the message queue, second target data of at least one dimension can be obtained from the data source every second set duration based on at least one dimension identifier in the data subscription request. In this embodiment, since the data in the data source may change during the period from when the first target data enters the message queue to when it is read from the message queue, the data obtained by the business from the message queue may be inconsistent with the data in the data source. Therefore, during the period from when the first target data enters the message queue to when it is read from the message queue, second target data can be obtained from the data source every second set duration based on the data subscription request to update the first target data, ensuring that the data obtained by the business is up-to-date, that is, ensuring the real-time performance of the business data.
[0051] S140. Correct the first target data in the message queue based on the second target data, so that the business party can read the corrected first target data from the message queue.
[0052] In this embodiment, the second target data can be compared with the first target data to obtain the comparison result, and the first target data can be corrected based on the comparison result.
[0053] Optionally, the first target data in the message queue is corrected based on the second target data, including: comparing the second target data with the first target data; if the second target data is the same as the first target data, then the first target data in the message queue remains unchanged; if the second target data is different from the first target data, then the second target data replaces the first target data as the new first target data in the message queue.
[0054] In this embodiment, the second target data can be processed. The processing operations can be the same as those for processing the first target data, including aggregation, preprocessing, and filtering. The processed second target data can be compared with the first target data. If the second target data is the same as the first target data, the first target data in the message queue remains unchanged. If the second target data is different from the first target data, it indicates that the data in the data source has been updated. In this case, the first target data is replaced with the second target data, and the second target data becomes the new first target data in the message queue.
[0055] In this embodiment, by comparing the second target data with the first target data, if the second target data is different from the first target data, the first target data is replaced with the second target data as the new first target data in the message queue. This effectively corrects the first target data and ensures that the first target data obtained by the business party is consistent with the data in the data source.
[0056] The technical solution of this disclosure involves obtaining first target data from a data source based on a data subscription request from a business party; pushing the first target data to the business party's message queue; obtaining second target data from the data source based on the data subscription request; and correcting the first target data in the message queue based on the second target data, so that the business party can read the corrected first target data from the message queue. This disclosure, by correcting the first target data in the message queue using the second target data, ensures that the data pushed to the business party is consistent with the data in the data source, improving the accuracy of data push.
[0057] Figure 2 This is a schematic diagram of another data push method provided in an embodiment of this disclosure. Figure 2 As shown, multiple business parties can send data subscription requests to the data center platform. The data center platform can then retrieve first target data from the data source across multiple dimensions based on the corresponding data subscription requests, aggregate the first target data across these dimensions, and obtain aggregated first target data. This aggregated first target data is then pushed to the message queue of each business party. During the period from when the first target data enters the message queue to when it is read from the message queue, a scheduled task manager retrieves second target data from the data source every second set interval based on the data subscription requests. A calibration task manager is then used to calibrate the first target data in the message queue based on the second target data, ensuring that the data pushed to the business party is consistent with the data in the data source.
[0058] Figure 3 This is a schematic flowchart of another data push method provided in an embodiment of this disclosure. The specific steps are as follows:
[0059] S310: Based on the business party's data subscription request, obtain the first target data from multiple dimensions from the data source.
[0060] S320. Aggregate the first target data from multiple dimensions to obtain the aggregated first target data.
[0061] S330. Based on the set data processing rules, preprocess the aggregated first target data to obtain the preprocessed aggregated first target data.
[0062] S340. Based on the set business rules, filter the preprocessed aggregated first target data to obtain the filtered first target data.
[0063] S350: Push the filtered first target data to the business's message queue.
[0064] S360: Obtain second target data from the data source based on a data subscription request.
[0065] S370. Correct the first target data in the message queue based on the second target data.
[0066] In this embodiment, by setting data processing rules and business rules to process the first target data, the first target data pushed to the message queue of the business party is more accurate.
[0067] Figure 4 This is a schematic diagram of a data push device structure provided in an embodiment of the present disclosure, as shown below. Figure 4 As shown, the device includes: a first target data acquisition module 410, a data push module 420, a second target data acquisition module 430, and a correction module 440.
[0068] The first target data acquisition module 410 is used to acquire the first target data from the data source based on the data subscription request of the business party;
[0069] Data push module 420 is used to push the first target data to the message queue of the business party;
[0070] The second target data acquisition module 430 is used to acquire second target data from the data source based on the data subscription request;
[0071] The correction module 440 is used to correct the first target data in the message queue based on the second target data, so that the business party can read the corrected first target data from the message queue.
[0072] The technical solution of this disclosure involves a first target data acquisition module acquiring first target data from a data source based on a data subscription request from a business party; a data push module pushing the first target data to the business party's message queue; a second target data acquisition module acquiring second target data from the data source based on the data subscription request; and a correction module correcting the first target data in the message queue based on the second target data, so that the business party can read the corrected first target data from the message queue. This embodiment, by correcting the first target data in the message queue using the second target data, ensures that the data pushed to the business party is consistent with the data in the data source, improving the accuracy of data push.
[0073] Optionally, the first target data acquisition module is specifically used to: acquire first target data of at least one dimension every first set time interval based on the data subscription request of the business party; and aggregate the first target data of at least one dimension to obtain aggregated first target data.
[0074] Optionally, the data push module is specifically used to push the aggregated first target data to the message queue of the business party.
[0075] Optionally, the above-mentioned device further includes a preprocessing module, which is used to: preprocess the first target data based on a set data processing rule; wherein the set data processing rule includes at least one of the following: permission verification rule and data cleaning rule.
[0076] Optionally, the above device further includes a filtering module, which is used to: filter the first target data based on set business rules to obtain the filtered first target data;
[0077] Optionally, the data push module is also used to push the filtered first target data to the message queue of the business party.
[0078] Optionally, the second target data acquisition module is specifically used to: acquire second target data from the data source based on the data subscription request every second set time interval during the period from when the first target data enters the message queue to when it is read out of the message queue.
[0079] Optionally, the correction module is specifically used to: compare the second target data with the first target data; if the second target data is the same as the first target data, then keep the first target data in the message queue unchanged; if the second target data is different from the first target data, then replace the first target data with the second target data as the new first target data in the message queue.
[0080] The data push device provided in this disclosure can execute the data push method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the method execution.
[0081] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.
[0082] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Reference is made below. Figure 5 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 5 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0083] like Figure 5 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.
[0084] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 5An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0085] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.
[0086] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0087] The electronic device provided in this embodiment and the data push method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.
[0088] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the data push method provided in the above embodiments.
[0089] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0090] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0091] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0092] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: obtain first target data from a data source based on a data subscription request from a business party; push the first target data to a message queue of the business party; obtain second target data from the data source based on the data subscription request; and correct the first target data in the message queue based on the second target data, so that the business party reads the corrected first target data from the message queue.
[0093] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0095] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".
[0096] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0097] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0098] According to one or more embodiments of the present disclosure, a data push method is disclosed, comprising: obtaining first target data from a data source based on a data subscription request from a business party; pushing the first target data to a message queue of the business party; obtaining second target data from the data source based on the data subscription request; and correcting the first target data in the message queue based on the second target data, so that the business party reads the corrected first target data from the message queue.
[0099] Furthermore, obtaining the first target data from the data source based on the business party's data subscription request includes: obtaining the first target data of at least one dimension every first set time interval based on the business party's data subscription request; and aggregating the first target data of the at least one dimension to obtain the aggregated first target data.
[0100] Furthermore, pushing the first target data to the message queue of the business party includes: pushing the aggregated first target data to the message queue of the business party.
[0101] Furthermore, before pushing the first target data to the message queue of the business party, the method further includes: preprocessing the first target data based on the set data processing rules; wherein the set data processing rules include at least one of the following: permission verification rules and data cleaning rules.
[0102] Furthermore, before pushing the first target data to the message queue of the business party, the method further includes: filtering the first target data based on set business rules to obtain filtered first target data; correspondingly, pushing the first target data to the message queue of the business party includes: pushing the filtered first target data to the message queue of the business party.
[0103] Furthermore, obtaining the second target data from the data source based on the data subscription request includes: obtaining the second target data from the data source every second predetermined time interval during the period from when the first target data enters the message queue to when it is read out of the message queue.
[0104] Further, the correction of the first target data in the message queue based on the second target data includes: comparing the second target data with the first target data; if the second target data is the same as the first target data, then keeping the first target data in the message queue unchanged; if the second target data is different from the first target data, then replacing the first target data with the second target data as the new first target data in the message queue.
[0105] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0106] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0107] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A data push method, characterized in that, include: The business side obtains the first target data from the data source based on the data subscription request; The first target data is pushed to the message queue of the business party; Based on the data subscription request, second target data is obtained from the data source, wherein the second target data is the same as the first target data, or an updated version of the first target data; The first target data in the message queue is corrected based on the second target data, so that the business party can read the corrected first target data from the message queue; The step of obtaining the second target data from the data source based on the data subscription request includes: During the period from when the first target data enters the message queue to when it is read out of the message queue, a second target data is obtained from the data source every second set interval based on the data subscription request, so as to update the first target data with the second target data.
2. The method according to claim 1, characterized in that, Based on the business's data subscription request, the first target data is obtained from the data source, including: Based on the data subscription request from the business party, at least one dimension of the first target data is retrieved every first set time interval; The first target data of at least one dimension is aggregated to obtain aggregated first target data.
3. The method according to claim 2, characterized in that, Pushing the first target data to the message queue of the business party includes: The aggregated first target data is pushed to the message queue of the business party.
4. The method according to claim 1 or 2, characterized in that, Before pushing the first target data to the message queue of the business party, the process also includes: The first target data is preprocessed based on the set data processing rules; wherein the set data processing rules include at least one of the following: permission verification rules and data cleaning rules.
5. The method according to claim 1 or 2, characterized in that, Before pushing the first target data to the message queue of the business party, the process also includes: The first target data is filtered based on the set business rules to obtain the filtered first target data; Accordingly, pushing the first target data to the message queue of the business party includes: The filtered first target data is pushed to the message queue of the business party.
6. The method according to claim 1, characterized in that, Correcting the first target data in the message queue based on the second target data includes: Compare the second target data with the first target data; If the second target data is the same as the first target data, then the first target data in the message queue remains unchanged; If the second target data is different from the first target data, then the second target data replaces the first target data and is used as the new first target data in the message queue.
7. A data push device, characterized in that, include: The first target data acquisition module is used to acquire the first target data from the data source based on the data subscription request from the business party. The data push module is used to push the first target data to the message queue of the business party; The second target data acquisition module is used to acquire second target data from the data source based on the data subscription request, wherein the second target data is the same as the first target data, or an updated version of the first target data; The correction module is used to correct the first target data in the message queue based on the second target data, so that the business party can read the corrected first target data from the message queue; The second target data acquisition module is specifically used to: during the period from when the first target data enters the message queue to when it is read out of the message queue, acquire second target data from the data source based on the data subscription request every second set time interval, so as to update the first target data with the second target data.
8. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the data push method as described in any one of claims 1-6.
9. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the data push method as described in any one of claims 1-6.