A method, device, equipment and medium for filtering commodity data

Abstract

The present invention discloses a method, device, equipment and medium for filtering commodity data, belonging to the technical field of data processing. The method includes: obtaining the commodity inventory data from the service unit through the timing task unit; dividing the commodity inventory data according to the number of indexes of the commodity inventory data; writing the divided multiple commodity inventory data structures into the database; obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in; and filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plug-in. The present invention realizes data synchronization through the ES plug-in, and provides a method for real-time and accurate data query and filtering, improves the real-time performance of commodity data update, enhances the search service ability, and improves the efficiency of the search service.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly to a method, device, equipment and medium for filtering commodity data. Background Art

[0002] The core of the search and recommendation system of an e-commerce platform is to select appropriate commodities from a massive commodity database according to the search intent and interests of users and display them to users; providing accurate and real-time data to users can more efficiently connect users with commodities / merchants, thereby obtaining higher market revenue.

[0003] The index data update of the ES distributed, highly scalable, and highly real-time search and data analysis engine (ElasticSearch) is divided into two types: full update and incremental update; the data of the full update is comprehensive and accurate, but not real-time enough and takes a long time to build; the incremental update data needs to merge segments, which consumes a large amount of CPU resources, thereby causing stability problems in the overall server. Summary of the Invention

[0004] In order to overcome the above technical defects, the purpose of the present invention is to provide a method, device, equipment and medium for filtering commodity data. The method of the present invention includes: obtaining the commodity inventory data from the service unit through the timing task unit; dividing the commodity inventory data according to the number of indexes of the commodity inventory data; writing the divided multiple commodity inventory data structures into the database; obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in; filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's receiving address and the commodity city code, and through the ES plug-in. The present invention realizes data synchronization through the ES plug-in and provides a method for real-time and accurate data query and filtering, improves the real-time performance of commodity data update, enhances the search service ability, and improves the efficiency of the search service.

[0005] The specific technical solutions provided by the embodiments of the present invention are as follows:

[0006] In a first aspect, a method for filtering commodity data is provided. The method for filtering commodity data is applied to a commodity data filtering system, which includes a service unit, a scheduled task unit, a database, an ES plugin, and a user query unit. The database includes a full - volume database and an incremental database. A first output end of the scheduled task unit is connected to an input end of the service unit, a second output end of the scheduled task unit is connected to a first input end of the database, an output end of the user query unit is connected to an input end of the ES plugin, and an output end of the ES plugin is connected to a second input end of the database. The method includes:

[0007] Obtaining the commodity inventory data from the service unit through the scheduled task unit;

[0008] Dividing the commodity inventory data according to the number of indexes of the commodity inventory data;

[0009] Writing the divided multiple commodity inventory data structures into the database;

[0010] Obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin;

[0011] Filtering the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code.

[0012] Further, the steps of obtaining the commodity inventory data from the service unit through the scheduled task unit, dividing the commodity inventory data according to the number of indexes of the commodity inventory data, and writing the divided multiple commodity inventory data structures into the database include:

[0013] The scheduled task unit obtains the full - volume commodity inventory data from the service unit, performs a modulo operation on the commodity ID data according to the first number of indexes of the full - volume commodity inventory data, divides the full - volume commodity inventory data, and writes the divided multiple full - volume commodity inventory data structures into the database;

[0014] The scheduled task unit obtains the incremental commodity inventory data from the service unit, performs a modulo operation on the commodity ID data according to the second number of indexes of the incremental commodity inventory data, divides the incremental commodity inventory data, and writes the divided multiple incremental commodity inventory data structures into the database.

[0015] Further, the step of obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin includes:

[0016] Obtain the IP addresses and index numbers of all ES plugin servers through the native ES interface;

[0017] Confirm the ES index number corresponding to the ES plugin according to the IP address of the ES plugin server.

[0018] Further, the step of obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin further includes:

[0019] Obtain the full - volume commodity inventory data and incremental commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin.

[0020] Further, the step of filtering the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code includes:

[0021] Judge whether the ES plugin includes the inventory data information corresponding to the commodity ID;

[0022] If so, filter the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code; if not, confirm that the system includes the inventory of the commodity.

[0023] Further, the step of filtering the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code further includes:

[0024] Confirm the city code corresponding to the user's delivery address according to the ID information of the commodity;

[0025] Confirm the commodity city code corresponding to the commodity according to the inventory information of the commodity;

[0026] Perform an intersection operation on the city code corresponding to the user's delivery address and the commodity city code corresponding to the commodity, filter the commodity inventory data corresponding to the ES index number through the ES plugin, and confirm the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plugin.

[0027] Further, after filtering the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code, it includes:

[0028] Match and compare the product inventory information queried by the user query unit with the product inventory information after filtering the product inventory data corresponding to the ES index number through the ES plug-in;

[0029] When the product inventory information after filtering the product inventory data corresponding to the ES index number by the ES plug-in is a null value, it means that the filtering of the product inventory data corresponding to the ES index number by the ES plug-in is successful.

[0030] In a second aspect, a product data filtering device is provided. The device includes:

[0031] An acquisition module, configured to obtain the product inventory data from the service unit through the timing task unit; obtain the product inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in;

[0032] A partitioning module, configured to partition the product inventory data according to the number of indexes of the product inventory data;

[0033] A processing module, configured to write the divided multiple product inventory data structures into the database;

[0034] A filtering module, configured to filter the product inventory data corresponding to the ES index number through the ES plug-in according to the city code corresponding to the user's delivery address and the product city code.

[0035] In a third aspect, a computer device is provided. The device includes:

[0036] A memory, a processor, and a computer program stored on the memory. The processor executes the computer program to implement the steps of the product data filtering method according to any one of the first aspects.

[0037] In a fourth aspect, a computer storage medium is provided. The medium includes:

[0038] A computer program is stored thereon, and when the computer program is executed by the memory, it implements the steps of the product data filtering method according to any one of the first aspects.

[0039] Compared with the prior art, the commodity data filtering method provided by the embodiment of the present invention is applied to a commodity data filtering system, which includes a service unit, a timing task unit, a database, an ES plugin, and a user query unit. The database includes a full-volume database and an incremental database. The first output end of the timing task unit is connected to the input end of the service unit, the second output end of the timing task unit is connected to the first input end of the database, the output end of the user query unit is connected to the input end of the ES plugin, and the output end of the ES plugin is connected to the second input end of the database. The method includes: obtaining the commodity inventory data from the service unit through the timing task unit; dividing the commodity inventory data according to the number of indexes of the commodity inventory data; writing the divided multiple commodity inventory data structures into the database; obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin; and filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plugin. The present invention realizes data synchronization through an ES plugin, and provides a method for real-time and accurate data query and filtering, improving the real-time performance of commodity data update, enhancing the search service ability, and improving the efficiency of the search service.

[0040] The technical solution provided by the embodiment of the present invention can realize the synchronization of various service data through a custom plugin, with high scalability. At the same time, it does not require frequent index construction, and the system stability is high.

[0041] For the commodity information that often changes, the technical solution provided by the embodiment of the present invention does not directly write it into the ES index field, but synchronizes the full-volume or incremental data through the ES plugin to obtain quasi-real-time data. At the same time, it does not require merging segments, which does not affect the system stability.

[0042] The technical solution provided by the embodiment of the present invention divides the full-volume or incremental data according to the number of indexes and writes it into the database. Each ES instance obtains a part of the data as needed, saving bandwidth and improving the data synchronization speed at the same time. Description of the Drawings

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0044] Figure 1The first flowchart of the commodity data filtering method provided in Embodiment 1 of the present invention;

[0045] Figure 2 The second flowchart of the commodity data filtering method provided in Embodiment 2 of the present invention;

[0046] Figure 3 The flowchart of writing all inventory data into the database provided in Embodiment 2 of the present invention;

[0047] Figure 4 The flowchart of writing incremental inventory data into the database provided in Embodiment 2 of the present invention;

[0048] Figure 5 The specific flowchart of the commodity data filtering method provided in Embodiment 2 of the present invention;

[0049] Figure 6 The structure diagram of the commodity data filtering device provided in Embodiment 3 of the present invention;

[0050] Figure 7 An exemplary system that can be used to implement the various embodiments described in the present application provided in Embodiment 5 of the present invention. Detailed implementation manners

[0051] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] It should be noted that, unless the context clearly requires otherwise, the words such as "including" and "comprising" throughout the specification and claims should be interpreted as the meaning of including rather than exclusive or exhaustive; that is, the meaning of "including but not limited to".

[0053] In addition, in the description of the present invention, unless otherwise specified, the meaning of "a plurality of" is two or more.

[0054] Embodiment 1

[0055] The embodiment of the present invention provides a commodity data filtering method, as Figure 1As shown in the figure, the commodity data filtering method is applied to a commodity data filtering system, which includes a service unit, a timed task unit, a database, an ES plugin, and a user query unit. The database includes a full-volume database and an incremental database. The first output end of the timed task unit is connected to the input end of the service unit, the second output end of the timed task unit is connected to the first input end of the database, the output end of the user query unit is connected to the input end of the ES plugin, and the output end of the ES plugin is connected to the second input end of the database. The method includes:

[0056] Obtain the commodity inventory data from the service unit through the timed task unit;

[0057] Divide the commodity inventory data according to the number of indexes of the commodity inventory data;

[0058] Write the divided multiple commodity inventory data structures into the database;

[0059] Obtain the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin;

[0060] Filter the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code.

[0061] The beneficial effects brought by the technical solution provided by the embodiment of the present invention are:

[0062] The present invention realizes data synchronization through an ES plugin, and provides a method for real-time and accurate data query and filtering, improves the real-time performance of commodity data update, enhances the search service ability, and improves the efficiency of the search service.

[0063] The technical solution provided by the embodiment of the present invention can realize the synchronization of various service data through a custom plugin, has high scalability, and does not require frequent index construction, and the system has high stability.

[0064] Embodiment 2

[0065] The embodiment of the present invention provides a commodity data filtering method, as Figure 2 shown, the method includes:

[0066] Step S01, obtain the commodity inventory data from the service unit through the timed task unit;

[0067] Divide the commodity inventory data according to the number of indexes of the commodity inventory data;

[0068] Write the divided multiple commodity inventory data structures into the database.

[0069] Specifically, the frequently changing product information includes product inventory information, product label information, and product city visibility information. Taking the scenario of product inventory information data synchronization as an example, the description is as follows:

[0070] For example, there are a total of 10 million products in the product index of the search system. According to the experience of product data performance testing and stress testing, the product data here includes 10 indexes, each index is named like ysearch0 - 9, and each index contains 1 million different products.

[0071] That is, the full - volume or incremental inventory data information of the product is respectively divided into 10 data structures, namely, put into the database in the form of hash hset: redis key = goodsid mod 10, Redis Value = stockInfo.

[0072] Among them, the hash type is similar to HashMap in Java, which is a collection of key - value pairs, and supports the operations of adding, deleting, modifying, and querying a single key. Both the hash hset and the inventory stock are data structures.

[0073] Step S01 further includes:

[0074] Step S011, the timing task unit obtains the full - volume inventory data of the product from the service unit, performs a modulo operation on the product ID data according to the number of the first indexes of the full - volume inventory data of the product, divides the full - volume inventory data of the product, and writes the divided multiple full - volume inventory data structures into the database;

[0075] The timing task unit obtains the incremental inventory data of the product from the service unit, performs a modulo operation on the product ID data according to the number of the second indexes of the incremental inventory data of the product, divides the incremental inventory data of the product, and writes the divided multiple incremental inventory data structures into the database.

[0076] Specifically, as Figure 3 shown, every day the timing task unit calls the business system interface to obtain the full - volume inventory data of the product, performs a modulo operation on the product ID data according to the number of the first indexes of the full - volume inventory data of the product, divides the full - volume inventory data of the product, and writes the divided multiple full - volume inventory data structures into the database.

[0077] Here, the number of the first indexes is 10.

[0078] The modulo operation is to find the remainder of the division of two numbers.

[0079] As Figure 4As shown, the timing task unit continuously consumes the change messages in the message queue MQ at all times to obtain the incremental inventory data of the goods. It performs a modulo operation on the goods ID data according to the number of the second indexes of the incremental inventory data of the goods, divides the incremental inventory data of the goods, and writes the divided multiple incremental inventory data structures of the goods into the database.

[0080] Redis key = (goodsid mod indexNum) + timestamp, Redis Value = stockInfo. Among them, the number of the second indexes is 10.

[0081] Step S02: Obtain the goods inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in.

[0082] Specifically, each ES instance only needs to obtain the inventory change information of 1 million goods corresponding to the ES index number of the ES instance. Therefore, for example, the ES instance with the index number ysearchN only needs to obtain the goods inventory information of stockN, or for example, the ES instance with the index number ysearch0 only needs to pull the goods inventory information of stock0.

[0083] Step S02 further includes:

[0084] Step S021: Obtain the IP addresses and index numbers of all ES plug-in servers through the ES native interface.

[0085] Confirm the ES index number corresponding to the ES plug-in according to the IP address of the ES plug-in server.

[0086] Specifically, the ES plug-in only needs to obtain the goods inventory information corresponding to the index number of the ES plug-in. First, it calls the ES native interface to obtain the basic information of the ES instance, so as to obtain the index number N corresponding to the ES plug-in.

[0087] Call the ES native interface GET / _cat / shards / ysearch*?h = ip,shard,index to obtain the IP addresses and index numbers of all ES plug-in servers, and then obtain the index number corresponding to the ES plug-in according to the IP address of the ES plug-in server.

[0088] Step S022: Obtain the full inventory data and incremental inventory data of the goods corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in.

[0089] Specifically, after obtaining the index number N corresponding to the ES plugin, the full inventory data and incremental inventory data information stockN of the product corresponding to the ES index number are obtained from the database according to the ES index number corresponding to the ES plugin, so that each ES instance has obtained relatively real-time product inventory information.

[0090] Step S03: Filter the product inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the product city code.

[0091] Specifically, since there is no inventory field in the ES plugin index, it is impossible to use the native query and filter interface of ES to filter products based on inventory conditions. Here, Java code is written through the plugin mechanism of ES to achieve this.

[0092] Step S03 further includes:

[0093] Step S031: Determine whether the ES plugin includes the inventory data information corresponding to the product ID;

[0094] If so, filter the product inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the product city code through the ES plugin; if not, confirm that the system includes the inventory of the product.

[0095] Specifically, if there is no inventory data information of the product ID in the ES plugin, it is considered that the product is in stock nationwide, and the search system returns true;

[0096] Here, in order to save system storage space, in most cases, the product is in stock nationwide, and the situation of out-of-stock nationwide is only recorded in a few scenarios.

[0097] If there is inventory data information of the product ID in the ES plugin, convert the inventory information of the product into a BitSet of the product city code, and then perform an intersection operation with the BitSet of the user city code, and return the operation result.

[0098] Step S032: Confirm the city code corresponding to the user's delivery address according to the ID information of the product;

[0099] Confirm the product city code corresponding to the product according to the inventory information of the product;

[0100] Perform an intersection operation on the city code corresponding to the user's delivery address and the commodity city code corresponding to the commodity, filter the commodity inventory data corresponding to the ES index number through the ES plug-in, and confirm the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plug-in.

[0101] Specifically, for example, the ID information of the commodity is 6919428961756006856, and the commodity is on sale but the inventory is 0.

[0102] Query according to the commodity ID + ES plug-in inventory filtering method, mid = 6919428961756006856, and the city code corresponding to the user's delivery address corresponding to the commodity ID information = 1011011101;

[0103] Since the commodity has no inventory, it is confirmed that the bitmap bitset of the commodity city code is all 0 according to the inventory information of the commodity;

[0104] Then perform an intersection operation on the user city code and the commodity city code, and the ES plug-in real_stock_filter filters and returns false.

[0105] Expected result: The commodity is filtered and not returned.

[0106] Step S04, match and compare the commodity inventory information queried by the user query unit with the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number through the ES plug-in;

[0107] When the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plug-in is a null value, the filtering of the commodity inventory data corresponding to the ES index number by the ES plug-in is successful.

[0108] Specifically, only query according to the user query unit, query according to the commodity ID, mid = 6919428961756006856, execution result: The ES return result is normal, and the number of commodities = 1.

[0109] And the execution result of the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number through the ES plug-in: The ES return result is normal, and the number of commodities = 0.

[0110] It can be understood that it shows that the filtering logic of the ES plug-in for commodity inventory data information is executed correctly and the filtering function is achieved.

[0111] Such as Figure 5 The specific flowchart of the commodity data filtering method is shown. The technical method of the present invention includes two processes: a write process and a read process:

[0112] 1) Writing process: Each business timed task unit (such as inventory operation, label operation) is responsible for calling the downstream business system interface to obtain product data, dividing the product data according to the number of indexes or the number of shards, dividing it into multiple keys / values, and then writing the divided multiple product data structures into the database (Redis).

[0113] Among them, the key / value is the attribute value of the element, and the key / value includes full amount data and incremental data. Here, the product data includes product inventory data and product label data.

[0114] 2) Reading process: The ES plugin calls the original interface to obtain the ES index number or shard number corresponding to the ES plugin, obtains the product data (full amount / incremental data) corresponding to the ES index number in the database according to the index number or shard number, and finally matches and compares the product information queried by the user query unit with the product information after filtering the product data corresponding to the ES index number through the ES plugin, achieving the effects of ES product data query Query and filtering Filter.

[0115] Without departing from the technical solution of the present invention, several improvements and optimizations can be made to the method for filtering product data provided by the embodiments of the present invention, and these improvements and optimizations should also be regarded as the protection scope of the present invention.

[0116] At the same time, the technical solution provided by the embodiments of the present invention can be applied to filtering of product inventory information and shielding of product label information in a search system.

[0117] The beneficial effects brought by the technical solution provided by the embodiments of the present invention are:

[0118] An embodiment of the present invention provides a method for filtering commodity data. The technical solution provided by the embodiment of the present invention for the commodity data filtering method is applied to a commodity data filtering system. The commodity data filtering system includes a service unit, a timing task unit, a database, an ES plugin, and a user query unit. The database includes a full - volume database and an incremental database. The first output end of the timing task unit is connected to the input end of the service unit, the second output end of the timing task unit is connected to the first input end of the database, the output end of the user query unit is connected to the input end of the ES plugin, and the output end of the ES plugin is connected to the second input end of the database. The method includes: obtaining the commodity inventory data from the service unit through the timing task unit; dividing the commodity inventory data according to the number of indexes of the commodity inventory data; writing the divided multiple commodity inventory data structures into the database; obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin; and filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's receiving address and the commodity city code through the ES plugin. The present invention realizes data synchronization through the ES plugin and provides a method for real - time and accurate data query and filtering, improving the real - time performance of commodity data update, enhancing the search service ability, and improving the efficiency of the search service.

[0119] For the commodity information that often changes, in the technical solution provided by the embodiment of the present invention, it is not directly written into the ES index field, but the full - volume or incremental data is synchronized through the ES plugin to obtain quasi - real - time data. At the same time, there is no need to merge segments, which does not affect the stability of the system.

[0120] Embodiment III

[0121] The present invention provides a commodity data filtering device, as Figure 6 shown. The device includes an acquisition module, a division module, a processing module, a filtering module, and a comparison module.

[0122] In this embodiment, the acquisition module is used to obtain the commodity inventory data from the service unit through the timing task unit; and obtain the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin;

[0123] The division module is used to divide the commodity inventory data according to the number of indexes of the commodity inventory data;

[0124] The processing module is used to write the divided multiple commodity inventory data structures into the database;

[0125] A filtering module, configured to filter the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code, and through the ES plug-in.

[0126] Further, a processing module is further configured to enable the timing task unit to obtain the full commodity inventory data from the service unit, perform a modulo operation on the commodity ID data according to the first index number of the full commodity inventory data, divide the full commodity inventory data, and write the divided multiple full commodity inventory data structures into the database;

[0127] The timing task unit obtains the incremental commodity inventory data from the service unit, performs a modulo operation on the commodity ID data according to the second index number of the incremental commodity inventory data, divides the incremental commodity inventory data, and writes the divided multiple incremental commodity inventory data structures into the database.

[0128] Further, an obtaining module is further configured to obtain the IP addresses and index numbers of all ES plug-in servers through the ES native interface;

[0129] Confirm the ES index number corresponding to the ES plug-in according to the IP address of the ES plug-in server.

[0130] Further, the obtaining module is further configured to obtain the full commodity inventory data and the incremental commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in.

[0131] Further, the filtering module is further configured to determine whether the ES plug-in includes the inventory data information corresponding to the commodity ID;

[0132] If so, filter the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code, and through the ES plug-in; if not, confirm that the system includes the inventory of the commodity.

[0133] Further, the filtering module is further configured to confirm the city code corresponding to the user's delivery address according to the ID information of the commodity;

[0134] Confirm the commodity city code corresponding to the commodity according to the inventory information of the commodity;

[0135] Perform an intersection operation on the city code corresponding to the user's delivery address and the commodity city code corresponding to the commodity, filter the commodity inventory data corresponding to the ES index number through the ES plug-in, and confirm the commodity inventory information after the ES plug-in filters the commodity inventory data corresponding to the ES index number.

[0136] Further, a comparison module is configured to match and compare the product inventory information queried by the user query unit with the product inventory information obtained by filtering the product inventory data corresponding to the ES index number through the ES plug-in;

[0137] When the product inventory information obtained by filtering the product inventory data corresponding to the ES index number through the ES plug-in is a null value, it indicates that the filtering of the product inventory data corresponding to the ES index number by the ES plug-in is successful.

[0138] The beneficial effects brought by the technical solution provided by the embodiment of the present invention are:

[0139] For frequently changing product information, the technical solution provided by the embodiment of the present invention does not directly write it into the ES index field, but obtains quasi-real-time data through the method of synchronizing full or incremental data by the ES plug-in. At the same time, there is no need to merge segments, which does not affect the stability of the system.

[0140] The technical solution provided by the embodiment of the present invention divides the full or incremental data according to the number of indexes and writes it into the database. Each ES instance obtains a part of the data as needed, saving bandwidth and improving the speed of data synchronization at the same time.

[0141] Embodiment 4

[0142] The present invention provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it can execute the following method for filtering product data:

[0143] Obtain the product inventory data from the service unit through the timing task unit;

[0144] Divide the product inventory data according to the number of indexes of the product inventory data;

[0145] Write the divided multiple product inventory data structures into the database;

[0146] Obtain the product inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in;

[0147] According to the city code corresponding to the user's delivery address and the product city code, and filter the product inventory data corresponding to the ES index number through the ES plug-in.

[0148] The beneficial effects brought by the technical solution provided by the embodiment of the present invention are:

[0149] The technical solution provided by the embodiment of the present invention can realize various business data synchronizations through custom plugins, with high scalability. At the same time, it does not require frequent index construction, and the system stability is high.

[0150] For the frequently changing product information, the technical solution provided by the embodiment of the present invention does not directly write to the ES index field here, but synchronizes the full or incremental data through the ES plugin to obtain near-real-time data. At the same time, it does not require merging segments, which does not affect the system stability.

[0151] Embodiment 5

[0152] The present invention provides a computer storage medium, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented:

[0153] Obtain the product inventory data from the service unit through the timing task unit;

[0154] Divide the product inventory data according to the number of indexes of the product inventory data;

[0155] Write the divided multiple product inventory data structures into the database;

[0156] Obtain the product inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin;

[0157] Filter the product inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the product city code, and through the ES plugin.

[0158] Further, the steps of obtaining the product inventory data from the service unit through the timing task unit, dividing the product inventory data according to the number of indexes of the product inventory data, and writing the divided multiple product inventory data structures into the database include:

[0159] The timing task unit obtains the full product inventory data from the service unit, performs a modulo operation on the product ID data according to the first index number of the full product inventory data, divides the full product inventory data, and writes the divided multiple full product inventory data structures into the database;

[0160] The timing task unit obtains the incremental product inventory data from the service unit, performs a modulo operation on the product ID data according to the second index number of the incremental product inventory data, divides the incremental product inventory data, and writes the divided multiple incremental product inventory data structures into the database.

[0161] Further, obtaining the commodity inventory data corresponding to the ES index number according to the ES index number corresponding to the ES plug-in in the database includes:

[0162] Obtaining the IP addresses and index numbers of all ES plug-in servers through the ES native interface;

[0163] Confirming the ES index number corresponding to the ES plug-in according to the IP address of the ES plug-in server.

[0164] Further, obtaining the commodity inventory data corresponding to the ES index number according to the ES index number corresponding to the ES plug-in in the database further includes:

[0165] Obtaining the full commodity inventory data and incremental commodity inventory data corresponding to the ES index number according to the ES index number corresponding to the ES plug-in in the database.

[0166] Further, filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code, and through the ES plug-in, includes:

[0167] Judging whether the ES plug-in includes the inventory data information corresponding to the commodity ID;

[0168] If so, filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code, and through the ES plug-in; if not, confirming that the system includes the inventory of the commodity.

[0169] Further, filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code, and through the ES plug-in further includes:

[0170] Confirming the city code corresponding to the user's delivery address according to the ID information of the commodity;

[0171] Confirming the commodity city code corresponding to the commodity according to the inventory information of the commodity;

[0172] Performing an intersection operation on the city code corresponding to the user's delivery address and the commodity city code corresponding to the commodity, filtering the commodity inventory data corresponding to the ES index number through the ES plug-in, and confirming the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plug-in.

[0173] Further, after obtaining the city code corresponding to the user's delivery address and the commodity city code, and filtering the commodity inventory data corresponding to the ES index through the ES plugin, the following steps are included:

[0174] Match and compare the commodity inventory information queried by the user query unit with the commodity inventory information after filtering the commodity inventory data corresponding to the ES index through the ES plugin;

[0175] When the commodity inventory information after filtering the commodity inventory data corresponding to the ES index through the ES plugin is a null value, it indicates that the filtering of the commodity inventory data corresponding to the ES index by the ES plugin is successful.

[0176] Figure 7 This is an exemplary system provided in Embodiment 5 of the present invention and can be used to implement each of the embodiments described in the present application.

[0177] As Figure 7 shown, in some embodiments, the system can be used as any one of the above devices for filtering commodity data in each of the embodiments. In some embodiments, the system may include one or more computer-readable media having results (e.g., system memory or NVM / storage device) and one or more processors (e.g., (one or more) processors) coupled to the one or more computer-readable media and configured to execute the results to implement modules so as to perform the actions described in the present application.

[0178] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by relevant hardware through a computer program. The above computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided by the present invention can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0179] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0180] The above-described embodiments only represent several implementation manners of the present invention. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the invention patent should be subject to the appended claims.

Claims

1. A method for filtering commodity data, characterized in that, The described commodity data filtering method is applied to a commodity data filtering system, which includes a service unit, a timed task unit, a database, an ES plugin, and a user query unit. The database includes a full - volume database and an incremental database. The first output end of the timed task unit is connected to the input end of the service unit, the second output end of the timed task unit is connected to the first input end of the database, the output end of the user query unit is connected to the input end of the ES plugin, and the output end of the ES plugin is connected to the second input end of the database. The method includes: Obtain commodity inventory data from the service unit through the timed task unit; Divide the commodity inventory data according to the number of indexes of the commodity inventory data; Write the divided multiple commodity inventory data structures into the database; Obtain the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin; Filter the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code; The steps of obtaining the commodity inventory data from the service unit through the timed task unit, dividing the commodity inventory data according to the number of indexes of the commodity inventory data, and writing the divided multiple commodity inventory data structures into the database include: The timed task unit obtains the full - volume commodity inventory data from the service unit, performs a modulo operation on the commodity ID data according to the first number of indexes of the full - volume commodity inventory data, divides the full - volume commodity inventory data, and writes the divided multiple full - volume commodity inventory data structures into the database; The timed task unit obtains the incremental commodity inventory data from the service unit, performs a modulo operation on the commodity ID data according to the second number of indexes of the incremental commodity inventory data, divides the incremental commodity inventory data, and writes the divided multiple incremental commodity inventory data structures into the database; The step of obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin includes: Obtain the full - volume commodity inventory data and the incremental commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin.

2. The commodity data filtering method according to claim 1, wherein, The step of obtaining the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plugin includes: Obtain the IP addresses and index numbers of all ES plugin servers through the ES native interface; Confirm the ES index number corresponding to the ES plugin according to the IP address of the ES plugin server.

3. The commodity data filtering method according to claim 1, characterized in that The step of filtering the commodity inventory data corresponding to the ES index number through the ES plugin according to the city code corresponding to the user's delivery address and the commodity city code includes: Judge whether the ES plugin includes the inventory data information corresponding to the commodity ID data; If so, filter the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plug-in; if not, confirm that the system includes the inventory of the commodity.

4. The commodity data filtering method according to claim 3, characterized in that The step of filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plug-in further includes: Confirm the city code corresponding to the user's delivery address according to the ID information of the commodity; Confirm the commodity city code corresponding to the commodity according to the inventory information of the commodity; Perform an intersection operation on the city code corresponding to the user's delivery address and the commodity city code corresponding to the commodity, filter the commodity inventory data corresponding to the ES index number through the ES plug-in, and confirm the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plug-in.

5. The commodity data filtering method according to claim 1, characterized in that After filtering the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plug-in, it includes: Match and compare the commodity inventory information queried by the user query unit with the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number through the ES plug-in; When the commodity inventory information after filtering the commodity inventory data corresponding to the ES index number by the ES plug-in is a null value, the filtering of the commodity inventory data corresponding to the ES index number by the ES plug-in is successful.

6. A commodity data filtering device for implementing the method as claimed in claim 1, characterized in that, The device includes: An acquisition module, configured to acquire the commodity inventory data from the service unit through the timing task unit; acquire the commodity inventory data corresponding to the ES index number in the database according to the ES index number corresponding to the ES plug-in; A partitioning module, configured to partition the commodity inventory data according to the number of indexes of the commodity inventory data; A processing module, configured to write the divided multiple commodity inventory data structures into the database; A filtering module, configured to filter the commodity inventory data corresponding to the ES index number according to the city code corresponding to the user's delivery address and the commodity city code through the ES plug-in.

7. A computer device, comprising a memory, a processor, and a computer program stored on the memory, characterized in that, The processor executes the computer program to implement the steps of the commodity data filtering method according to any one of claims 1 to 5.

8. A computer storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the memory, it implements the steps of the commodity data filtering method according to any one of claims 1 to 5.

Images