Data acquisition method, device, electronic device and storage medium
Through the data acquisition method in the message center, the target attributes of the target item are used to update the message and the fixed time interval between the server and the server, and intelligently determine the time when the server pulls the target item file, solving the inefficient efficiency when updating dynamic product advertisements in the prior art. The problem has achieved timely and accurate updates on the server side.
Patent Information
- Application Number
- CN202110235114.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-03-03
- Publication Date
- 2025-02-25
- Estimated Expiration
- 2041-03-03
AI Technical Summary
The prior art is inefficient when updating dynamic product advertisements, especially during major promotions or events, and cannot update product information in a timely and accurate manner, resulting in delays in information or unnecessary information pull.
By implementing a data acquisition method in the message center, the target attributes of the target item are used to update the message and the fixed time interval between the server, intelligently determine the time when the server pulls the target item file, generate a notification message, and notify the server from the target storage device. Pull the updated item file.
It improves the efficiency of updating item files on the server, enables the server to pull the updated item files in a timely and accurate manner, solving the problems of information delay and inefficient updates.
Smart Images

Figure CN113778638B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a data acquisition method, device, electronic device and storage medium. Background Art
[0002] With the advent of the big data era, numerous scenarios enabled by intelligent data have emerged. The advertising field, as an important landing scenario, has spawned dynamic product advertisements (DPA, Dynamic Product Ads) to meet advertisers' demands for batch creative production and efficient marketing conversion.
[0003] Among them, dynamic product advertisements require updates to the products in the product library. In related technologies, the method of uploading product information online is only applicable to a small number of product information entry scenarios, and the update of product information requires manual triggering; the method of manually uploading XML files requires redevelopment of XML file uploads when updating; the method of automatically pulling files regularly has a fixed update frequency and cannot update product information in a timely and accurate manner, and has poor timeliness. During big sales or events, sensitive data such as inventory, prices, promotional information, etc. may only be updated during a certain period of time, and remain unchanged during other periods, which will cause message delays during periods of frequent information changes, and pull the same information during periods of stable information.
[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention
[0005] The purpose of the present disclosure is to provide a data acquisition method, device, electronic device and storage medium, which can improve the efficiency of updating item files on a server side, so that the server side can pull updated item files in a timely and accurate manner.
[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.
[0007] An embodiment of the present disclosure provides a data acquisition method, which is applied to a message center, and the method includes: obtaining a target attribute update message of a target item among items from a client, the target attribute update message including an updated target attribute; obtaining a fixed time interval for actively pulling an item file of the item from a server, the item file of the item being stored in a target storage device; generating a notification message according to the target attribute update message and the fixed time interval of the target item, the notification message being used to notify the server to pull the target item file of the target item from the target storage device.
[0008] In some exemplary embodiments of the present disclosure, a notification message is generated based on a target attribute update message of a target item and a fixed time interval, including: determining a current time window based on the updated target attributes and their corresponding weights; determining a current information entropy based on the current time window and the fixed time interval; if the current information entropy is greater than or equal to an information entropy threshold, generating a notification message for pulling a target item file corresponding to the target item; if the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; if the current information entropy is a preset value, generating a notification message for not pulling the target item file corresponding to the target item.
[0009] In some exemplary embodiments of the present disclosure, a notification message is generated based on a target attribute update message of a target item and a fixed time interval, including: generating a queue corresponding to the attribute of the item based on the attribute of the item; adding the target item to the queue corresponding to the target attribute; determining a total priority based on the priority of each queue and the number of items in each queue; if the total priority is greater than or equal to a priority threshold, generating a notification message to pull a target item file corresponding to the target item; if the total priority is less than the priority threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval; if the total priority is a preset value, generating a notification message not to pull the target item file corresponding to the target item.
[0010] In some exemplary embodiments of the present disclosure, a notification message is generated based on a target attribute update message of a target item and a fixed time interval, including: determining the update quantity of the target item; if the update quantity is greater than or equal to a quantity threshold, generating a notification message to pull a target item file corresponding to the target item; if the update quantity is less than the quantity threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval; if the update quantity is a preset value, generating a notification message that does not pull the target item file corresponding to the target item.
[0011] The disclosed embodiment provides a data acquisition method, which is applied to a client, and the method includes: acquiring source data of an item; converting the format of the source data of the item to obtain an item file in a preset format; storing the item file in the preset format in a target storage device; obtaining an item index file of the item file, and configuring the item index file on a server; acquiring a target item to be updated among the items and its target attributes; updating an item file and an item index file in a preset format according to the target item to be updated among the items and its target attributes; storing the updated item file in the preset format in a target storage device, and configuring the updated item index file on a server; and sending a target attribute update message of the target item to a message center, wherein the target attribute update message includes the updated target attributes.
[0012] In some exemplary embodiments of the present disclosure, an item file and an item index file in a preset format are updated according to a target item to be updated in an item and its target attributes, including: obtaining a target item file corresponding to the target item to be updated from a target storage medium; updating the target attributes in the target item file to obtain an updated target item file; and storing the updated target item file in a target storage device.
[0013] The disclosed embodiment provides a data acquisition method, which is applied to a server, and the method includes: acquiring an item index file of an item from a client, the item file of the item is stored in a target storage device, the item index file includes a storage index of the item file in the target storage device; setting a fixed time interval for actively pulling the item file from the target storage device according to the item index file; when a target attribute of a target item in the item is updated, acquiring the updated item index file from the client; receiving a notification message from a message center, the notification message is generated based on update information of the target attribute of the target item and the fixed time interval; in response to the notification message, pulling the target item file of the target item corresponding to the updated item index file from the target storage device; and pushing a directional display link including the target item file to a target media playback platform.
[0014] The disclosed embodiment provides a data acquisition device, which is applied to a message center. The device includes: an update message acquisition module, which is used to obtain a target attribute update message of a target item in the items from a client, wherein the target attribute update message includes an updated target attribute; a time interval acquisition module, which is used to obtain a fixed time interval for actively pulling an item file of an item from a server, wherein the item file of the item is stored in a target storage device; and a notification message generation module, which is used to generate a notification message according to the target attribute update message of the target item and the fixed time interval, wherein the notification message is used to notify the server to pull the target item file of the target item from the target storage device.
[0015] The disclosed embodiment provides a data acquisition device, which is applied to a client, and the device includes: a source data acquisition module, which is used to acquire source data of an item; a format conversion module, which is used to convert the format of the source data of the item and obtain an item file in a preset format; an item file storage module, which is used to store the item file in the preset format in a target storage device; an index file acquisition module, which is used to acquire an item index file of the item file and configure the item index file on a server; a target item acquisition module, which is used to acquire a target item to be updated among the items and its target attributes; a file update module, which is used to update the item file and the item index file in the preset format according to the target item to be updated among the items and its target attributes; an index file configuration module, which is used to store the updated item file in the preset format in the target storage device and configure the item index file on the server; an update message sending module, which is used to send a target attribute update message of the target item to the client, wherein the target attribute update message includes the updated target attribute.
[0016] The disclosed embodiment provides a data acquisition device, which is applied to a server, and the device includes: an index file acquisition module, which is used to acquire an item index file of an item from a client, wherein the item file of the item is stored in a target storage device, and the item index file includes a storage index of the item file in the target storage device; a time interval setting module, which is used to set a fixed time interval for actively pulling item files from the target storage device according to the item index file; an index file update module, which is used to acquire an updated item index file from a client when a target attribute of a target item in the item is updated; a notification message receiving module, which is used to receive a notification message from a message center, wherein the notification message is generated based on update information of the target attribute of the target item and a fixed time interval; an item file pulling module, which is used to pull a target item file of a target item corresponding to the updated item index file from the target storage device in response to the notification message; and a display connection sending module, which is used to push a directional display link including the target item file to a target media playback platform.
[0017] An embodiment of the present disclosure provides an electronic device, comprising: at least one processor; and a storage device for storing at least one program, wherein when the at least one program is executed by the at least one processor, the at least one processor implements any one of the above-mentioned data acquisition methods.
[0018] An embodiment of the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, any of the above-mentioned data acquisition methods is implemented.
[0019] The data acquisition method provided by the embodiment of the present disclosure can be used by a message center to update a target attribute message of a target item and obtain a fixed time interval for actively pulling the item file of the item on the server side, intelligently determine the timing for the server side to pull the target item file of the target item, generate a notification message, and notify the server side to pull the target item file of the target item from the target storage device; the method can determine the timing for the server to pull the updated file according to the change of the item information, and can improve the efficiency of the server side in updating the item file, so that the server side can pull the updated item file in a timely and accurate manner.
[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.
[0022] Figure 1 The diagram shows an exemplary system architecture to which the data acquisition method according to the embodiment of the present disclosure can be applied.
[0023] Figure 2 The diagram shows an exemplary system architecture to which the data acquisition method according to the embodiment of the present disclosure can be applied.
[0024] Figure 3 is a flow chart showing a data acquisition method according to an exemplary embodiment.
[0025] Figure 4 is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0026] Figure 5 is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0027] Figure 6 is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0028] Figure 7 is a flow chart of yet another data acquisition method according to an exemplary embodiment.
[0029] Figure 8 is a flow chart of yet another data acquisition method according to an exemplary embodiment.
[0030] Fig. 9 The figure is a schematic diagram showing a server-side parsing file according to an exemplary embodiment.
[0031] Fig.10 A block diagram of a data acquisition device is shown according to an exemplary embodiment.
[0032] Fig.11 A block diagram of a data acquisition device is shown according to an exemplary embodiment.
[0033] Fig.12 A block diagram of a data acquisition device is shown according to an exemplary embodiment.
[0034] Fig.13 A schematic structural diagram of an electronic device is shown according to an exemplary embodiment. DETAILED DESCRIPTION
[0035] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0036] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.
[0037] Figure 1 The diagram shows an exemplary system architecture to which the data acquisition method according to the embodiment of the present disclosure can be applied.
[0038] like Figure 1 As shown, the system architecture may include a client 110, a message center 120 and a server 130. The message center 120 may be an independent terminal device or a server, or may be configured in the client 110 or in the server 130, and the present disclosure does not limit this.
[0039] The client 110 may be a smart phone, a tablet computer, a portable computer, a desktop computer, a smart speaker, a wearable smart device, a virtual reality device, an augmented reality device, etc., but the present disclosure is not limited thereto.
[0040] The server 130 can be an independent server, or a server cluster or distributed system composed of multiple servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), as well as big data and artificial intelligence platforms.
[0041] In the embodiment of the present disclosure, the client 110 can obtain the source data of the item, convert the format of the source data of the item, and obtain the item file in the preset format; the client 110 can store the item file in the preset format in the target storage device; the client 110 can generate an item index file of the item file, and configure the item index file on the server 130, and the server 130 can obtain the item index file of the item from the client 110, and the item index file includes the storage index of the item file in the target storage device; when there is a target item update in the item, the client 110 can obtain the target item to be updated in the item and its target attribute; the client 110 can update the item file and the item index file in the preset format according to the target item to be updated in the item and its target attribute; the client 110 can store the updated item file in the preset format in the target storage device, and configure the updated item index file on the server 130; the server 130 can obtain the updated item index file from the client 110; the client 110 can send a target attribute update message of the target item to the message center 120, and the target attribute update message may include the updated target attribute.
[0042] In the embodiment of the present disclosure, the message center 120 can obtain a target attribute update message of a target item in the items from the client 110; the server 130 can set a fixed time interval for actively pulling the item file from the target storage device according to the item index file, and send the fixed time interval to the message center 120; the message center 120 can obtain the fixed time interval from the server 130; the message center 120 can generate a notification message according to the target attribute update message of the target item and the fixed time interval, and the message center 120 can send the notification message to the server 130 to notify the server 130 to pull the target item file of the target item from the target storage device; the server 130 can receive the notification message from the message center 120; the server 130 can respond to the notification message and pull the target item file of the target item corresponding to the updated item index file from the target storage device; the server 130 can push a directional display link including the target item file to the target media playback platform.
[0043] Figure 2The diagram shows an exemplary system architecture to which the data acquisition method according to the embodiment of the present disclosure can be applied.
[0044] like Figure 2 As shown, the embodiment of the present disclosure may adopt a CS architecture, including a client, a message center, and a server. The client may include a target data generation module, an intelligent message determination module, and a real-time information processing module to complete the collection, update, and message notification of commodity information; the message center serves as a communication bridge between the client and the server to transmit messages; the server may include a heterogeneous data analysis module, a real-time message processing module, and a storage module to complete the storage and update of commodity data.
[0045] The client can obtain the source data of the item, the client can convert the format of the source data of the item, and can obtain the target data in a preset format; when the item is updated, the client can update the target data; the client can verify the legality of the updated target data, and when the updated target data is legal, the updated target data can be stored.
[0046] The client can send an update message of the target data to the message center; the client can also make a preliminary judgment on the update message and then send an update message of the target data to the message center through the message interaction module.
[0047] The client can generate a data file including an item index file, and configure the data file on the server; the message receiving module of the server can obtain the item index file from the client, the item index file includes a storage index of the item file in the target storage device, and the server can configure the item index file in the configuration center; the server can receive a notification message from the message center, and can respond to the notification message, pull the target item file of the target item corresponding to the item index file from the target storage device, verify the legitimacy of the target item file, standardize the verified target item file, and store the target item file.
[0048] It should be understood that Figure 1 The number of terminal devices, message centers and servers in the example is only for illustration, and any number of terminal devices, message centers and servers may be provided according to actual needs.
[0049] It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principle of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.
[0050] Below, each step of the data acquisition method in the exemplary embodiment of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.
[0051] Figure 3 is a flow chart of a data acquisition method according to an exemplary embodiment. The method provided by the embodiment of the present disclosure can be performed as follows Figure 1 The process is executed in the message center shown in the figure, but the present disclosure is not limited thereto.
[0052] like Figure 3 As shown, the data acquisition method provided by the embodiment of the present disclosure may include the following steps.
[0053] In step S302, a target attribute update message of a target item among the items is obtained from the client, and the target attribute update message includes an updated target attribute.
[0054] In the embodiments of the present disclosure, the article may be a commodity, and the article may include daily necessities, electronic products, books, food, etc., and the present disclosure does not impose any limitation on this.
[0055] The attributes of an item may be, for example, the item name, item ID, store name, item price, etc.
[0056] The target item may be an item whose attribute is changed, and the target attribute may be an attribute whose attribute is changed.
[0057] Let's take the target item as laundry detergent and the target attribute as price as an example.
[0058] If the price of the washing powder changes, the client can send a price update message of the washing powder to the message center, where the price update message includes the updated price, and the message center obtains the price update message of the washing powder from the client.
[0059] In step S304, a fixed time interval for actively pulling the item file of the item is obtained from the server, and the item file of the item is stored in the target storage device.
[0060] In the embodiment of the present disclosure, the server can set a fixed time interval for the server to actively pull the item files of the item from the target storage device, where the target storage device can be a cloud storage device, and the fixed time interval can be set according to actual conditions, for example, 1 hour, 1 day, 1 week, etc.
[0061] The server may send the set fixed time interval for actively pulling item files to the message center, and the message center may obtain the fixed time interval from the server.
[0062] In step S306, a notification message is generated according to the target attribute update message of the target object and the fixed time interval, and the notification message is used to notify the server to pull the target object file of the target object from the target storage device.
[0063] In the disclosed embodiment, the notification message may be generated by a weighted average method, a multi-priority queue method, or a fixed number of commodities method.
[0064] In the disclosed embodiment, the message center can determine the current information entropy based on the updated target attributes in the target attribute update message of the target item and the fixed time interval; if the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; if the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated; if the current information entropy is a preset value, a notification message is generated not to pull the target item file corresponding to the target item.
[0065] In the disclosed embodiment, the message center can generate a queue corresponding to the attributes of the item based on the attributes of the item, and add the target item to the queue corresponding to the target attribute; determine the total priority based on the priority of each queue and the number of items in each queue; if the total priority is greater than or equal to the priority threshold, generate a notification message to pull the target item file corresponding to the target item; if the total priority is less than the priority threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated; if the total priority is a preset value, generate a notification message that does not pull the target item file corresponding to the target item.
[0066] In the disclosed embodiment, the message center may compare the update quantity of the target item with a quantity threshold; if the update quantity is greater than or equal to the quantity threshold, a notification message is generated to pull the target item file corresponding to the target item; if the update quantity is less than the quantity threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval; if the update quantity is a preset value, a notification message is generated not to pull the target item file corresponding to the target item.
[0067] After the message center generates the notification message, it can send the notification message to the server. After receiving the notification message, the server can pull the target object file of the target object from the target storage device.
[0068] The data acquisition method provided by the embodiment of the present disclosure can be used by a message center to update a target attribute message of a target item and obtain a fixed time interval for actively pulling the item file of the item on the server side, intelligently determine the timing for the server side to pull the target item file of the target item, generate a notification message, and notify the server side to pull the target item file of the target item from the target storage device; the method can determine the timing for the server to pull the updated file according to the change of the item information, and can improve the efficiency of the server side in updating the item file, so that the server side can pull the updated item file in a timely and accurate manner.
[0069] Figure 4is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0070] In the embodiment of the present disclosure, Figure 3 Based on the data acquisition method shown, Figure 4 The data acquisition method shown further provides how to generate a notification message according to the target attribute update message and the fixed time interval of the target object, that is, provides an embodiment of the above step S306.
[0071] like Figure 4 As shown, step S306 may include the following steps.
[0072] In step S402, the current time window is determined according to the updated target attributes and their corresponding weights.
[0073] In the disclosed embodiments, different weights may be set for different attributes of an item. For example, the price of an item has a greater impact on the item, so the weight corresponding to the price may be set higher; the category of an item has a smaller impact on the item, so the weight corresponding to the category may be set lower.
[0074] In the embodiment of the present disclosure, the time window may be initialized first, and the initial value of the time window may be set to T0, where T0 may be 0.
[0075] In the embodiment of the present disclosure, the current time window can be determined according to the following formula:
[0076]
[0077] Among them, TIME(range) represents the current time window, K represents the updated target attribute, and V represents the weight corresponding to the target attribute.
[0078] In step S404, the current information entropy is determined according to the current time window and the fixed time interval.
[0079] In the embodiment of the present disclosure, the current information entropy can be determined according to the following formula:
[0080] R=TIME(range) / T1
[0081] Among them, R represents the current information entropy, and T1 represents a fixed time interval.
[0082] In step S406, if the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item.
[0083] The information entropy threshold can be set according to actual conditions, and the information entropy threshold can be represented by R0.
[0084] In the disclosed embodiment, if the current information entropy R is greater than or equal to R0, a notification message for pulling the target item file corresponding to the target item is generated to notify the server to immediately pull the target item file.
[0085] In the disclosed embodiment, after generating a notification message for pulling a target item file corresponding to a target item, the message center may reset the current time window and the current information entropy.
[0086] In step S408, if the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval.
[0087] In the embodiment of the present disclosure, the time when the server pulls the file can be set according to a fixed time interval. For example, if the fixed time interval is 1 hour, the time when the server pulls the file can be 1:00, 2:00, 3:00, etc.
[0088] If the current information entropy R is less than the information entropy threshold R0, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated.
[0089] Among them, the server can pull the target item file from the target storage device at a fixed time interval. When the current time is greater than or equal to the time when the server pulls the file, the server has pulled the target item file from the target storage device at the time when the server pulls the file, and the current information entropy is less than the information entropy threshold, that is, the server does not need to pull the target item file from the target storage device at this time, and the message center does not generate a notification message.
[0090] In step S410, if the current information entropy is a preset value, a notification message is generated not to pull the target item file corresponding to the target item.
[0091] The preset value can be set to 0 or can be set according to actual conditions.
[0092] In the disclosed embodiment, if the current information entropy is 0, that is, no item is updated, the message center generates a notification message not to pull the target item file corresponding to the target item, and sends the notification message to the server. After receiving the notification message, the server does not pull the target item file.
[0093] The data acquisition method provided by the embodiment of the present disclosure intelligently determines the timing of triggering file pulling by setting a time window, intelligently determines the weights corresponding to the attributes of different items by configuring rule strategies, and determines the timing of pulling files, thereby improving the updating efficiency of item files and solving the problem of items not being able to be updated in time and low updating efficiency.
[0094] Figure 5is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0095] In the embodiment of the present disclosure, Figure 3 Based on the data acquisition method shown, Figure 5 The data acquisition method shown further provides how to generate a notification message according to the target attribute update message and the fixed time interval of the target object, that is, provides an embodiment of the above step S306.
[0096] like Figure 5 As shown, step S306 may include the following steps.
[0097] In step S502, based on the attributes of the items, a queue corresponding to the attributes of the items is generated.
[0098] In the disclosed embodiment, a queue corresponding to the attribute of each item may be set for the attribute of each item, and each queue includes items whose attributes have changed.
[0099] For example, the items that have changed are item A, item B, and item C. The price attribute and category attribute of item A have changed, the price attribute and store name attribute of item B have changed, and the store name attribute of item C has changed. Then the queue corresponding to the price attribute includes item A and item B, the queue corresponding to the category attribute includes item A, and the queue corresponding to the store name attribute includes item B and item C.
[0100] In step S504, the target item is added to a queue corresponding to the target attribute.
[0101] For example, the target item is item D, and the target attribute is price, that is, if the price attribute of the target item changes, item D can be added to the queue corresponding to the price attribute.
[0102] In step S506, the overall priority is determined according to the priority of each queue and the number of items in each queue.
[0103] In the disclosed embodiment, the priority of each queue corresponding to each attribute can be set according to the importance of the attribute. For example, the price attribute is more important to the item, and the priority of the queue corresponding to the price attribute can be set higher; the store name attribute is not important to the item, and the priority of the queue corresponding to the store name attribute can be set lower.
[0104] In the disclosed embodiment, the message center may determine the overall priority based on the priority of each queue and the number of items in each queue.
[0105] For example, the priority of each queue can be multiplied by the number of items in each queue and then summed to obtain the total priority.
[0106] In step S508, if the total priority is greater than or equal to the priority threshold, a notification message is generated to pull the target item file corresponding to the target item.
[0107] The priority threshold can be set according to actual conditions.
[0108] In the disclosed embodiment, if the total priority is greater than or equal to the priority threshold, a notification message for pulling the target item file corresponding to the target item is generated to notify the server to immediately pull the target item file.
[0109] In step S510, if the total priority is less than the priority threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval.
[0110] In the embodiment of the present disclosure, the time when the server pulls the file can be set according to a fixed time interval. For example, if the fixed time interval is 1 hour, the time when the server pulls the file can be 1:00, 2:00, 3:00, etc.
[0111] If the total priority is less than the priority threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated.
[0112] Among them, the server can pull the target item file from the target storage device at a fixed time interval. When the current time is greater than or equal to the time when the server pulls the file, the server has pulled the target item file from the target storage device at the time when the server pulls the file, and the current information entropy is less than the information entropy threshold, that is, the server does not need to pull the target item file from the target storage device at this time, and the message center does not generate a notification message.
[0113] In step S512, if the total priority is a preset value, a notification message is generated not to pull the target item file corresponding to the target item.
[0114] The preset value can be set to 0 or can be set according to actual conditions.
[0115] In the disclosed embodiment, if the total priority is 0, that is, no item is updated, the message center generates a notification message not to pull the target item file corresponding to the target item, and sends the notification message to the server. After receiving the notification message, the server does not pull the target item file.
[0116] Figure 6 is a flow chart showing another data acquisition method according to an exemplary embodiment.
[0117] In the embodiment of the present disclosure, Figure 3 Based on the data acquisition method shown, Figure 6 The data acquisition method shown further provides how to generate a notification message according to the target attribute update message and the fixed time interval of the target object, that is, provides an embodiment of the above step S306.
[0118] like Figure 6 As shown, step S306 may include the following steps.
[0119] In step S602, the update quantity of the target item is determined.
[0120] In the disclosed embodiment, a globally unified update quantity update of the target items may be set, and the initial value of the update quantity update may be set to 0.
[0121] In step S604, if the update quantity is greater than or equal to the quantity threshold, a notification message is generated to pull the target item file corresponding to the target item.
[0122] The quantity threshold can be set according to actual conditions.
[0123] In the disclosed embodiment, if the update quantity is greater than or equal to the quantity threshold, a notification message for pulling the target item file corresponding to the target item is generated to notify the server to immediately pull the target item file.
[0124] In step S606, if the update quantity is less than the quantity threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to a fixed time interval.
[0125] In the embodiment of the present disclosure, the time when the server pulls the file can be set according to a fixed time interval. For example, if the fixed time interval is 1 hour, the time when the server pulls the file can be 1:00, 2:00, 3:00, etc.
[0126] If the number of updates is less than the number threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated.
[0127] Among them, the server can pull the target item file from the target storage device at a fixed time interval. When the current time is greater than or equal to the time when the server pulls the file, the server has pulled the target item file from the target storage device at the time when the server pulls the file, and the current information entropy is less than the information entropy threshold, that is, the server does not need to pull the target item file from the target storage device at this time, and the message center does not generate a notification message.
[0128] In step S608, if the update quantity is a preset value, a notification message is generated not to pull the target item file corresponding to the target item.
[0129] The preset value can be set to 0 or can be set according to actual conditions.
[0130] In the disclosed embodiment, if the update quantity is 0, that is, no item is updated, the message center generates a notification message not to pull the target item file corresponding to the target item, and sends the notification message to the server. After receiving the notification message, the server does not pull the target item file.
[0131] Figure 7 FIG. 1 is a flow chart of another data acquisition method according to an exemplary embodiment. The method provided in the embodiment of the present disclosure may be performed as follows: Figure 1 The process is executed in the client shown in the figure, but the present disclosure is not limited to this.
[0132] like Figure 7 As shown, the data acquisition method provided by the embodiment of the present disclosure may include the following steps.
[0133] In step S702, source data of the item is obtained.
[0134] In the disclosed embodiment, the user can upload the source data of the item through the client, and the user can also select the file format of the source data, for example, XML (Extensible Markup Language) file format, SQL (Structured Query Language) file format, CVS (Concurrent Version System) file format, etc.
[0135] In the disclosed embodiment, users can upload files in various formats, and the usage threshold of the client is relatively low.
[0136] The item may be a commodity, and the source data of the commodity may include information details of the commodity, such as price, number, brand, etc.
[0137] In the disclosed embodiment, the source data of the item may be verified first, for example, whether the data source of the source data of the item is valid, whether the path exists, whether the file format is supported, etc.
[0138] In step S704, the format of the source data of the item is converted to obtain an item file in a preset format.
[0139] In the embodiment of the present disclosure, the preset format may be a Map file format, a List file format, an XML file format, an SQL file format, a CSV file format, and the like.
[0140] In the disclosed embodiment, the user can select a preset format through the client. The client can parse the source data through SDK (Software Development Kit) according to the preset format selected by the user, and convert the source data of the item into an item file in the preset format.
[0141] In the embodiment of the present disclosure, files of multiple formats can be supported as the transmission medium for item information. Compared with the related art that only supports XML files, the embodiment of the present disclosure can support heterogeneous data sources such as JSON and SQL, and can have higher processing efficiency and experience diversity.
[0142] In addition, by supporting multiple heterogeneous data source modes, different systems can avoid redevelopment based on their own existing technical solutions, thereby improving development flexibility and efficiency; secondly, it reduces the granularity of concurrent competition in a distributed environment and increases the concurrency of product information updates.
[0143] In the disclosed embodiment, a target generation file entity class may be selected according to a preset format; and an item file of a preset format may be generated through the target generation file entity class.
[0144] Among them, SDK can provide development components for the product library for source data parsing, target item file generation and target item file verification.
[0145] In step S706, the item file in the preset format is stored in the target storage device.
[0146] In the embodiment of the present disclosure, the target storage device may be a cloud storage device.
[0147] In the disclosed embodiment, the item file in a preset format may be first checked to determine whether it satisfies preset rules, for example, a structure check and a data validity check may be performed on the item file.
[0148] In step S708, an item index file of the item file is obtained, and the item index file is configured on the server.
[0149] In the disclosed embodiment, the client can encapsulate the item file, obtain the Sitemap item index file, and configure the Sitemap index file on the server. The server can obtain the item index file, obtain the item file generated by the client through the item index file, and build a product library.
[0150] In step S710, the target item to be updated and its target attribute are obtained.
[0151] In the disclosed embodiment, when a target attribute of a target item is updated, the client may be notified via a message such as MQ (MessageQueue). After receiving the message, the client may obtain the target item to be updated and its target attribute.
[0152] In step S712, the item file and the item index file in a preset format are updated according to the target item to be updated and its target attribute.
[0153] In the disclosed embodiment, the client may update the item file and the item index file in a full coverage mode or an update mode.
[0154] The full coverage mode can, for example, allow the client to obtain the item file corresponding to the target item to be updated, and obtain all items corresponding to the item file; perform new logic on all items to generate a new item file; upload the new item file to the cloud storage device, overwrite the original item file, and update the item index file.
[0155] When the number of items in a single item file is small, it is efficient to generate the item file in full coverage mode, avoiding file downloading, indexing, and updating operations.
[0156] In an exemplary embodiment, according to the target item to be updated in the items and its target attributes, the item file and the item index file in a preset format are updated, including: obtaining the target item file corresponding to the target item to be updated from the target storage medium; updating the target attributes in the target item file to obtain an updated target item file; and storing the updated target item file in the target storage device.
[0157] The update mode may be, for example, that the client obtains the item file containing the target item to be updated from the cloud storage medium, downloads the item file to the local computer, updates the information corresponding to the target item to be updated in the item file through the identifier of the target item to be updated, uploads the updated item file to the cloud storage device, and updates the item index file.
[0158] The update mode can reduce the granularity of concurrent contention in a distributed environment and increase the concurrency of item information updates.
[0159] In step S714, the updated item file in the preset format is stored in the target storage device, and the updated item index file is configured on the server.
[0160] In the embodiment of the present disclosure, the client can store the updated item file in a preset format in a target storage device, so that the server can subsequently obtain the updated item file from the target storage device; the client can configure the updated item index file on the server, so that the server can obtain the updated item file according to the updated item index file.
[0161] In step S716, a target attribute update message of the target item is sent to the message center, where the target attribute update message includes the updated target attribute.
[0162] In the disclosed embodiment, the client may send a target attribute update message of the target item to the message center, so that the message center can notify the server to pull the updated item file through a time window-based message notification mechanism.
[0163] Figure 8 FIG. 1 is a flow chart of another data acquisition method according to an exemplary embodiment. The method provided in the embodiment of the present disclosure may be performed as follows: Figure 1 The process is executed in the server shown, but the present disclosure is not limited to this.
[0164] like Figure 8 As shown, the data acquisition method provided by the embodiment of the present disclosure may include the following steps.
[0165] In step S802, an item index file of an item is obtained from a client, the item file of the item is stored in a target storage device, and the item index file includes a storage index of the item file in the target storage device.
[0166] In the disclosed embodiment, the server may obtain an item index file of an item from the client, and based on the item index file, the server may obtain an item file corresponding to the item index file from a cloud storage device.
[0167] In step S804, a fixed time interval for actively pulling item files from the target storage device according to the item index file is set.
[0168] In the embodiment of the present disclosure, the server can pre-set a fixed time interval for actively pulling item files from the cloud storage device based on the item index file, and determine the time to actively pull item files based on the fixed time interval. At every fixed time interval, the server can actively pull item files from the cloud storage device for update.
[0169] In step S806, when the target attribute of the target item in the items is updated, the updated item index file is obtained from the client.
[0170] In the disclosed embodiment, when the target attribute of a target item in the items is updated, the server can obtain the updated item index file from the client, so that the server can subsequently pull the updated item file from the cloud storage device.
[0171] In step S808, a notification message is received from the message center, where the notification message is generated based on the update information of the target attribute of the target item and a fixed time interval.
[0172] In the embodiment of the present disclosure, the server can receive a notification message from the message center. If the notification message received is for pulling the target item file, the server immediately pulls the target item file from the cloud storage device; if the notification message received is for not pulling the target item file, it means that the item file has not been updated at present, and the server may not pull the target item file even if the time for pulling the file arrives.
[0173] In step S810, in response to the notification message, a target item file of a target item corresponding to the updated item index file is pulled from a target storage device.
[0174] In the disclosed embodiment, the server may pull the target item file of the target item corresponding to the updated item index file from the cloud storage device.
[0175] Fig. 9 The figure is a schematic diagram showing a server-side parsing file according to an exemplary embodiment.
[0176] In the present disclosure, Fig. 9 As shown, after pulling the target item file (target data file), the server can determine the file parsing rules according to the file format of the target item file. For example, if the file format of the target item file is Map, select a parsing mode that can parse Map; if the file format of the target item file is List, select a parsing mode that can parse List.
[0177] The server can parse the target item file content into the unified data specification of the commodity library and store it, realizing the standardization and normalization of heterogeneous data. The server can also perform legality verification on the parsed target item file.
[0178] In step S812, a directional display link including the target object file is pushed to the target media playback platform.
[0179] The target media playback platform may be, for example, a shopping application, a ticket purchasing platform, etc., and the directional display link may be, for example, an advertisement.
[0180] For example, the server may push an advertisement including a target item file to a shopping application.
[0181] The data acquisition method provided by the embodiment of the present disclosure can display advertisements to users in a timely and effective manner based on updated item information, thereby avoiding the problem of false advertising caused by untimely information updates in advertisements.
[0182] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.
[0183] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.
[0184] Fig.10 A block diagram of a data acquisition device according to an exemplary embodiment is shown. The data acquisition device can be applied to a message center.
[0185] like Fig.10 As shown, the data acquisition device 1000 may include: an update message acquisition module 1002 , a time interval acquisition module 1004 and a notification message generation module 1006 .
[0186] Among them, the update message acquisition module 1002 can be used to obtain the target attribute update message of the target item in the item from the client, and the target attribute update message includes the updated target attribute; the time interval acquisition module 1004 can be used to obtain the fixed time interval for actively pulling the item file of the item from the server, and the item file of the item is stored in the target storage device; the notification message generation module 1006 can be used to generate a notification message according to the target attribute update message and the fixed time interval of the target item, and the notification message is used to notify the server to pull the target item file of the target item from the target storage device.
[0187] In an exemplary embodiment, the notification message generation module 1006 may include: a time window determination unit, which can be used to determine the current time window based on the updated target attributes and their corresponding weights; an information entropy determination unit, which can be used to determine the current information entropy based on the current time window and a fixed time interval; a first judgment unit, which can be used to generate a notification message for pulling the target item file corresponding to the target item if the current information entropy is greater than or equal to the information entropy threshold; a second judgment unit, which can be used to not generate a notification message if the current information entropy is less than the information entropy threshold and the current time is greater than or equal to the time when the server pulls the file, and the time when the server pulls the file is determined according to the fixed time interval; a third judgment unit, which can be used to generate a notification message for not pulling the target item file corresponding to the target item if the current information entropy is a preset value.
[0188] In an exemplary embodiment, the notification message generation module 1006 may include: a queue generation unit, which can be used to generate a queue corresponding to the attributes of the item according to the attributes of the item; an item queue addition unit, which can be used to add the target item to the queue corresponding to the target attribute; a total priority determination unit, which can be used to determine the total priority according to the priority of each queue and the number of items in each queue; a fifth judgment unit, which can be used to generate a notification message for pulling the target item file corresponding to the target item if the total priority is greater than or equal to the priority threshold; a sixth judgment unit, which can be used to not generate a notification message if the total priority is less than the priority threshold and the current time is greater than or equal to the time when the server pulls the file, and the time when the server pulls the file is determined according to a fixed time interval; a seventh judgment unit, which can be used to generate a notification message for not pulling the target item file corresponding to the target item if the total priority is a preset value.
[0189] In an exemplary embodiment, the notification message generation module 1006 may include: an update quantity determination unit, which can be used to determine the update quantity of the target item; an eighth judgment unit, which can be used to generate a notification message for pulling the target item file corresponding to the target item if the update quantity is greater than or equal to the quantity threshold; a ninth judgment unit, which can be used to not generate a notification message if the update quantity is less than the quantity threshold and the current time is greater than or equal to the time when the server pulls the file, and the time when the server pulls the file is determined according to a fixed time interval; a tenth judgment unit, which can be used to generate a notification message for not pulling the target item file corresponding to the target item if the update quantity is a preset value.
[0190] Fig.11 A block diagram of a data acquisition device according to an exemplary embodiment is shown, and the data acquisition device can be applied to a client.
[0191] like Fig.11 As shown, the data acquisition device 1100 may include: a source data acquisition module 1102, a format conversion module 1104, an item file storage module 1106, an index file acquisition module 1108, a target item acquisition module 1110, a file update module 1112, an index file configuration module 1114 and an update message sending module 1116.
[0192] Among them, the source data acquisition module 1102 can be used to acquire the source data of the item; the format conversion module 1104 can be used to convert the format of the source data of the item to obtain an item file in a preset format; the item file storage module 1106 can be used to store the item file in a preset format in a target storage device; the index file acquisition module 1108 can be used to obtain the item index file of the item file, and configure the item index file on the server; the target item acquisition module 1110 can be used to acquire the target item to be updated and its target attributes in the item; the file update module 1112 can be used to update the item file and item index file in a preset format according to the target item to be updated and its target attributes in the item; the index file configuration module 1114 can be used to store the updated item file in a preset format in the target storage device, and configure the item index file on the server; the update message sending module 1116 can be used to send a target attribute update message of the target item to the client, and the target attribute update message includes the updated target attribute.
[0193] In an exemplary embodiment, the file update module 1112 may include: a file acquisition unit, which can be used to acquire a target item file corresponding to the target item to be updated from a target storage medium; a file acquisition unit, which can be used to update the target attribute in the target item file to obtain an updated target item file; and a file storage unit, which can be used to store the updated target item file in a target storage device.
[0194] Fig.12 A block diagram of a data acquisition device according to an exemplary embodiment is shown. The data acquisition device can be applied to a server.
[0195] like Fig.12 As shown, the data acquisition device 1200 may include: an index file acquisition module 1202, a time interval setting module 1204, an index file update module 1206, a notification message receiving module 1208, an item file pulling module 1210 and a display connection sending module 1212.
[0196] Among them, the index file acquisition module 1202 can be used to obtain the item index file of the item from the client, the item file of the item is stored in the target storage device, and the item index file includes the storage index of the item file in the target storage device; the time interval setting module 1204 can be used to set a fixed time interval for actively pulling the item file from the target storage device according to the item index file; the index file update module 1206 can be used to obtain the updated item index file from the client when the target attribute of the target item in the item is updated; the notification message receiving module 1208 can be used to receive a notification message from the message center, and the notification message is generated based on the update information of the target attribute of the target item and the fixed time interval; the item file pulling module 1210 can be used to respond to the notification message and pull the target item file of the target item corresponding to the updated item index file from the target storage device; the display connection sending module 1212 can be used to push a directional display link including the target item file to the target media playback platform.
[0197] It should be noted that the block diagrams shown in the above figures are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.
[0198] Fig.13 A schematic diagram of the structure of an electronic device according to an exemplary embodiment is shown. It should be noted that: Fig.13 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.
[0199] like Fig.13 As shown, the electronic device 1300 includes a central processing unit (CPU) 1301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1302 or a program loaded from a storage part 1308 into a random access memory (RAM) 1303. In the RAM 1303, various programs and data required for the operation of the system 1300 are also stored. The CPU 1301, the ROM 1302, and the RAM 1303 are connected to each other via a bus 1304. An input / output (I / O) interface 1305 is also connected to the bus 1304.
[0200] The following components are connected to the I / O interface 1305: an input section 1306 including a keyboard, a mouse, etc.; an output section 1307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1308 including a hard disk, etc.; and a communication section 1309 including a network interface card such as a LAN card, a modem, etc. The communication section 1309 performs communication processing via a network such as the Internet. A drive 1310 is also connected to the I / O interface 1305 as needed. A removable medium 1311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1310 as needed, so that a computer program read therefrom is installed into the storage section 1308 as needed.
[0201] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1309, and / or installed from a removable medium 1311. When the computer program is executed by a central processing unit (CPU) 1301, the above-mentioned functions defined in the system of the present invention are executed.
[0202] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.
[0203] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0204] The units involved in the embodiments of the present invention may be implemented in software or hardware. The units described may also be provided in a processor. For example, they may be described as follows: a processor includes a sending unit, an acquiring unit, a determining unit, and a first processing unit. The names of these units do not, in some cases, limit the units themselves. For example, the sending unit may also be described as a "unit that sends a request for obtaining an image to a connected server."
[0205] As another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes: obtaining a target attribute update message of a target item in the item from the client, the target attribute update message includes the updated target attribute; obtaining a fixed time interval for actively pulling the item file of the item from the server, the item file of the item is stored in the target storage device; generating a notification message according to the target attribute update message and the fixed time interval of the target item, the notification message is used to notify the server to pull the target item file of the target item from the target storage device.
[0206] The exemplary embodiments of the present invention are specifically shown and described above. It should be understood that the present invention is not limited to the detailed structure, configuration or implementation method described herein; on the contrary, the present invention is intended to cover various modifications and equivalent configurations included in the spirit and scope of the appended claims.
Claims
1. A data acquisition method, characterized in that: Applied to a message center, the method includes: Acquire a target attribute update message of a target item among the items from the client, wherein the target attribute update message includes an updated target attribute; Obtaining a fixed time interval for actively pulling the item file of the item from the server, the item file of the item being stored in a target storage device; generating a notification message according to the target attribute update message of the target item and the fixed time interval, wherein the notification message is used to notify the server to pull the target item file of the target item from the target storage device; The step of generating a notification message according to the target attribute update message of the target item and the fixed time interval includes: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
2. The method according to claim 1, characterized in that Generate a notification message according to the target attribute update message of the target object and the fixed time interval, including: According to the attributes of the items, a queue corresponding to the attributes of the items is generated; Adding the target item to a queue corresponding to the target attribute; Determine the overall priority based on the priority of each queue and the number of items in each queue; If the total priority is greater than or equal to the priority threshold, generating a notification message for pulling the target item file corresponding to the target item; If the total priority is less than the priority threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the total priority is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
3. The method according to claim 1, characterized in that Generate a notification message according to the target attribute update message of the target item and the fixed time interval, including: Determining an update quantity of the target item; If the update quantity is greater than or equal to the quantity threshold, generating a notification message for pulling the target item file corresponding to the target item; If the update quantity is less than the quantity threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the update quantity is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
4. A data acquisition method, characterized in that: Applied to a client, the method comprises: Get the source data of the item; Converting the format of the source data of the item to obtain an item file in a preset format; storing the item file in the preset format in a target storage device; Obtaining an item index file of the item file, and configuring the item index file on the server; Obtaining a target item to be updated and its target attribute from the items; According to the target items to be updated in the items and their target attributes, updating the item file in the preset format and the item index file; storing the updated item file in the preset format in the target storage device, and configuring the updated item index file on the server; Sending a target attribute update message of the target item to a message center, so that the message center generates a notification message according to the target attribute update message of the target item and the fixed time interval for actively pulling the item file of the item, wherein the notification message is used to notify the server to pull the target item file of the target item from the target storage device; wherein the target attribute update message includes the updated target attribute; the fixed time interval for actively pulling the item file of the item is obtained by the message center from the server, and the item file of the item is stored in the target storage device; The message center generates a notification message according to the fixed time interval of the target attribute update message of the target item and actively pulling the item file of the item, including: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
5. The method according to claim 4, characterized in that According to the target items to be updated in the items and their target attributes, updating the item file in the preset format and the item index file includes: Acquire a target object file corresponding to the target object to be updated from a target storage medium; Updating the target attribute in the target object file to obtain an updated target object file; The updated target item file is stored in the target storage device.
6. A data acquisition method, characterized in that: Applied to the server, the method includes: Acquire an item index file of an item from a client, wherein the item file of the item is stored in a target storage device, and the item index file includes a storage index of the item file in the target storage device; Setting a fixed time interval for actively pulling the item file from the target storage device according to the item index file; When a target attribute of a target item among the items is updated, obtaining an updated item index file from the client; receiving a notification message from a message center, wherein the notification message is generated by the message center based on update information of the target attribute of the target item and the fixed time interval; In response to the notification message, pulling a target item file of the target item corresponding to the updated item index file from the target storage device; Pushing a directional display link including the target item file to a target media playback platform; The message center generates the notification message based on the update information of the target attribute of the target item and the fixed time interval, including: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
7. A data acquisition device, characterized in that: Applied to a message center, the device comprises: An update message acquisition module, used to acquire a target attribute update message of a target item among the items from a client, wherein the target attribute update message includes an updated target attribute; A time interval acquisition module, used to acquire a fixed time interval for actively pulling the item file of the item from the server, the item file of the item being stored in a target storage device; a notification message generating module, used to generate a notification message according to the target attribute update message of the target item and the fixed time interval, wherein the notification message is used to notify the server to pull the target item file of the target item from the target storage device; The notification message generation module generates a notification message according to the target attribute update message of the target item and the fixed time interval, including: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
8. A data acquisition device, characterized in that: Applied to a client, the device comprises: A source data acquisition module, used to acquire source data of items; A format conversion module, used for converting the format of the source data of the item to obtain an item file in a preset format; An item file storage module, used for storing the item file in the preset format in a target storage device; An index file obtaining module, used to obtain an item index file of the item file and configure the item index file on the server; A target item acquisition module, used to acquire the target item to be updated and its target attribute from the items; A file updating module, used for updating the item file in the preset format and the item index file according to the target item to be updated and its target attribute among the items; An index file configuration module, used to store the updated item file in the preset format in the target storage device, and configure the item index file on the server; An update message sending module is used to send a target attribute update message of the target item to the client, so that the message center generates a notification message according to the target attribute update message of the target item and the fixed time interval for actively pulling the item file of the item, and the notification message is used to notify the server to pull the target item file of the target item from the target storage device; wherein the target attribute update message includes the updated target attribute; the fixed time interval for actively pulling the item file of the item is obtained by the message center from the server, and the item file of the item is stored in the target storage device; The message center generates a notification message according to the fixed time interval of the target attribute update message of the target item and actively pulling the item file of the item, including: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
9. A data acquisition device, characterized in that: Applied to the server, the device comprises: An index file acquisition module, used to acquire an item index file of an item from a client, wherein the item file of the item is stored in a target storage device, and the item index file includes a storage index of the item file in the target storage device; A time interval setting module, used to set a fixed time interval for actively pulling the item file from the target storage device according to the item index file; An index file updating module, configured to obtain an updated item index file from the client when a target attribute of a target item in the items is updated; A notification message receiving module, configured to receive a notification message from a message center, wherein the notification message is generated by the message center based on update information of the target attribute of the target item and the fixed time interval; an item file pulling module, configured to pull, in response to the notification message, a target item file of the target item corresponding to the updated item index file from the target storage device; A display link sending module, used for pushing a directional display link including the target item file to a target media playback platform; The message center generates the notification message based on the update information of the target attribute of the target item and the fixed time interval, including: Determining a current time window according to the updated target attributes and their corresponding weights; Determining current information entropy according to the current time window and the fixed time interval; If the current information entropy is greater than or equal to the information entropy threshold, a notification message is generated to pull the target item file corresponding to the target item; If the current information entropy is less than the information entropy threshold, and the current time is greater than or equal to the time when the server pulls the file, no notification message is generated, and the time when the server pulls the file is determined according to the fixed time interval; If the current information entropy is a preset value, a notification message is generated for not pulling the target item file corresponding to the target item.
10. An electronic device, characterized in that: include: at least one processor; A storage device for storing at least one program, when the at least one program is executed by the at least one processor, so that the at least one processor implements the data acquisition method as described in any one of claims 1 to 3 or the data acquisition method as described in claim 4 or 5 or the data acquisition method as described in claim 6.
11. A computer-readable storage medium having computer-executable instructions stored thereon, characterized in that: When the executable instructions are executed by the processor, the data acquisition method according to any one of claims 1 to 3, the data acquisition method according to claim 4 or 5, or the data acquisition method according to claim 6 is implemented.
Citation Information
Patent Citations
Mixed updating method e-commerce index file based on Solr
CN108280215A