Search service dynamic field storage method, device, equipment and medium

Abstract

The present application relates to a search service dynamic field storage method, device, equipment and medium, the method comprising: obtaining the store data of each independent station sent from the upstream; parsing and enumerating the data type of the custom field corresponding to the store data of each independent station, and determining the custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware according to the data type; determining the subfield of the corresponding field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure; determining the custom data attribute and searchable field name of the custom field in the field hierarchy structure of the full-text search engine middleware, and determining the field value corresponding to the custom field according to the custom data attribute and the searchable field name. The present application solves the problem that the custom fields of independent stations are prone to field expansion.

Description

Technical Field

[0001] The present application relates to the field of e-commerce platforms, and in particular to a search service dynamic field storage method, a corresponding device, an electronic device, and a computer-readable storage medium. Background Art

[0002] Cross-border e-commerce is an industry that has developed rapidly in recent years. Independent sites are a new form of cross-border e-commerce. In operation, they have high autonomy and are more flexible, avoiding many constraints of platform rules. In the SaaS background of the independent site e-commerce platform, search service is a core component that independent sites rely heavily on. In order to maximize the utilization of resources, tens of thousands of sellers' data are shared search services.

[0003] Due to the openness of independent station construction, merchants can customize fields for independent station store data such as product titles, store titles or category information. Merchants can freely and flexibly add field names, field types and field values ​​to create custom fields. At the same time, consumer users can also use these custom fields for searching. When merchants increase the custom fields infinitely, each additional custom field will make the overall index larger, which is very likely to cause field expansion and affect cluster stability. Moreover, when the field names of custom fields between independent station stores are the same but the field types are different, the custom field will not be able to be written into the search engine of the independent station e-commerce platform, and the custom fields between independent station stores will affect each other.

[0004] In summary, when merchants add the custom fields, the overall index becomes larger, which is extremely likely to cause field expansion, affect cluster stability, and the custom fields between independent sites and stores influence each other. The applicant has made corresponding explorations to solve this problem. Summary of the invention

[0005] The purpose of the present application is to solve the above-mentioned problem and to provide a search service dynamic field storage method, a corresponding device, an electronic device and a computer-readable storage medium.

[0006] In order to meet the various objectives of this application, this application adopts the following technical solutions:

[0007] A search service dynamic field storage method proposed to meet one of the purposes of this application includes the following steps:

[0008] Respond to search data synchronization events and obtain store data for each independent site sent from the upstream system;

[0009] Parse and enumerate the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determine the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types;

[0010] Determine the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the subfield of the field hierarchy structure;

[0011] In response to an online query instruction carrying a custom field, the custom data attributes and searchable field names of the custom field in the field hierarchy of the full-text search engine middleware are determined, and the field value corresponding to the custom field is determined according to the custom data attributes and searchable field name.

[0012] Optionally, the step of parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determining the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types includes the following steps:

[0013] In response to a field hierarchy generation request, a field hierarchy corresponding to the custom field is constructed, the field hierarchy including an outer layer and an inner layer, the outer layer is used to identify the custom field, the inner layer contains one or more subfields, each subfield is used to store the field name of the custom field and the field value of its corresponding custom data attribute, the subfield corresponds one-to-one to the custom data attribute.

[0014] Optionally, the step of obtaining the store data of each independent station sent from the upstream system in response to the search data synchronization event includes the following steps:

[0015] In response to a timed monitoring event, determining whether a custom field exists in the store data of each independent station sent by the upstream system;

[0016] If there is a custom field in the upstream original data, the corresponding subfield of the field hierarchy structure is determined according to the field value of the custom field and the custom data attribute corresponding to the field value, and the field name and field value of the custom field are written into the subfield of the field hierarchy structure.

[0017] Optionally, the step of determining the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure includes the following steps:

[0018] In response to a custom field storage event, a key-value data structure is used to encode the field value of the custom field and the corresponding custom data attribute to determine a character string or a character string array in a preset format;

[0019] The character string or character string array in the preset format is stored in a custom field of a subfield specified by a field hierarchy structure of the full-text search engine middleware.

[0020] Optionally, the step of parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station includes the following steps:

[0021] Respond to the custom field parsing instruction and call the preset function to parse the custom fields corresponding to the store data of each independent station;

[0022] Get and return the data type, field name and field value corresponding to the field value of the custom field.

[0023] Optionally, the step of responding to an online query instruction carrying a custom field includes the following steps:

[0024] In response to an online query instruction carrying a custom product keyword, parsing and determining the custom product keyword in the online query instruction and its corresponding data type;

[0025] Determine the custom data attribute and searchable field name of the custom product keyword in the field hierarchy structure of the full-text search engine middleware according to the data type corresponding to the custom product keyword;

[0026] Based on the custom data attribute corresponding to the custom product keyword and the searchable field name, a search is performed in the subfield specified by the full-text search engine middleware to find a field value matching the custom product keyword.

[0027] Optionally, the independent site store data includes one or more of the following data: product title, store title or product price.

[0028] A search service dynamic field storage device provided for another purpose of the present application includes:

[0029] A data acquisition module, configured to acquire store data of each independent station sent from an upstream system in response to a search data synchronization event;

[0030] A field parsing module, configured to parse and enumerate the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determine the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types;

[0031] A field storage module, configured to determine the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the subfield of the field hierarchy structure;

[0032] The online query module is configured to respond to online query instructions carrying custom fields, determine the custom data attributes and retrievable field names of the custom fields in the field hierarchy structure of the full-text search engine middleware, and determine the field value corresponding to the custom field based on the custom data attributes and retrievable field name.

[0033] An electronic device provided to meet another purpose of the present application includes a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the search service dynamic field storage method described in the present application.

[0034] A computer-readable storage medium is provided to meet another purpose of the present application, which stores a computer program implemented according to the search service dynamic field storage method in the form of computer-readable instructions. When the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

[0035] Compared with the prior art, the present application aims to solve the problems that when merchants add the custom fields, the overall index becomes larger, which is extremely easy to cause field expansion, affect the stability of the cluster, and the custom fields between independent stations and stores affect each other. The present application obtains and parses the custom fields in the independent station store data, determines the data type corresponding to the custom field, and thus determines the subfield specified by the custom field in the field hierarchy structure of the full-text search engine middleware (Elasticsearch). In each subfield of the field hierarchy structure, all field values ​​in the searchable fields of each independent station store data and the custom data attributes corresponding to the field value are stored in a preset format. Different independent station stores are stored in the same subfield of the field hierarchy structure of the full-text search engine middleware (Elasticsearch). The custom data attributes corresponding to all field values ​​in the searchable fields of the shop data are used to avoid the situation where different independent station stores use the same custom field name but different field value types. The custom field will not be able to be written into the search engine of the independent station e-commerce platform, avoiding the problem of index superposition and amplification caused by custom fields of different data types of each independent station store, so that the index is kept at a constant level, thereby increasing the index space used for custom fields without generating meaningless space occupation, greatly saving the memory resources of the search service of the independent station e-commerce platform, and storing the custom data attributes corresponding to all field values ​​in the searchable fields of different independent station store data in the same subfield of the field hierarchy structure of the full-text search engine middleware (Elasticsearch), including but not limited to the following beneficial effects:

[0036] First, to avoid the risk of field explosion, the maximum number of fields is equal to the number of field types that can be enumerated by the full-text search engine middleware (Elasticsearch);

[0037] Second, there is no restriction on search methods, and it can support any search method supported by the full-text search engine middleware (Elasticsearch);

[0038] Third, the structure is universal, which avoids the problem of different independent station stores having the same custom fields interfering with each other;

[0039] Fourth, iteration is efficient. If no new field types are added, the structure can be used all the time. If a new field data type needs to be added, you only need to synchronously add a new type of processing flow.

[0040] Furthermore, when searching for independent site store data, custom data attributes of the matched field values ​​are obtained by parsing the custom fields in the independent site store data, thereby dynamically acquiring custom data attributes. Therefore, when adding custom data attributes to custom fields in the independent site store data, the server memory resources occupied by the index in the search engine are greatly saved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0042] Figure 1 This is a flowchart of a method for storing dynamic fields of a search service in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of the code structure of the field hierarchy structure of the full-text search engine middleware in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a process for responding to a timed monitoring event and determining whether a custom field exists in the store data of each independent station sent by the upstream system in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of a process for determining a subfield of a field hierarchy structure corresponding to a custom field according to a field value of a custom field and a custom data attribute corresponding to the field value in an embodiment of the present application;

[0046] Figure 5 A flowchart of parsing and enumerating data types of field values ​​of custom fields corresponding to store data of each independent station in an embodiment of the present application;

[0047] Figure 6 This is a flow chart of responding to an online query instruction carrying a custom product keyword in an embodiment of the present application;

[0048] Figure 7 This is a principle block diagram of a dynamic field storage device for a search service in an embodiment of the present application;

[0049] Figure 8 It is a schematic diagram of the structure of the computer device in the embodiment of the present application. DETAILED DESCRIPTION

[0050] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be interpreted as limiting the present application.

[0051] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0052] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0053] It will be understood by those skilled in the art that the "client", "terminal" and "terminal device" used herein include both devices with wireless signal receivers, which are devices with only wireless signal receivers without transmission capabilities, and devices with receiving and transmitting hardware, which are devices with receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such devices may include: cellular or other communication devices such as personal computers, tablet computers, which have single-line displays or multi-line displays or cellular or other communication devices without multi-line displays; PCS (Personal Communications Service, personal communication system), which can combine voice, data processing, fax and / or data communication capabilities; PDA (Personal Digital Assistant, personal digital assistant), which may include a radio frequency receiver, pager, Internet / intranet access, web browser, notepad, calendar and / or GPS (Global Positioning System, global positioning system) receiver; conventional laptop and / or palmtop computers or other devices, which have and / or include a conventional laptop and / or palmtop computer or other device with and / or including a radio frequency receiver. The "client", "terminal" and "terminal device" used herein may be portable, transportable, installed in a vehicle (air, sea and / or land), or suitable for and / or configured to run locally, and / or in a distributed form, at any other location on the earth and / or in space. The "client", "terminal" and "terminal device" used herein may also be a communication terminal, an Internet terminal, a music / video playing terminal, for example, a PDA, a MID (Mobile Internet Device) and / or a mobile phone with a music / video playing function, or a smart TV, a set-top box and other devices.

[0054] The hardware referred to by the names such as "server", "client", and "service node" in this application is essentially an electronic device with capabilities equivalent to those of a personal computer. It is a hardware device that has the necessary components revealed by the von Neumann principle, such as a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device. Computer programs are stored in its memory, and the central processing unit calls the program stored in the external memory into the internal memory for execution, executes the instructions in the program, and interacts with the input and output devices to complete specific functions.

[0055] It should be pointed out that the concept of "server" referred to in this application can also be extended to the case of server clusters. According to the network deployment principle understood by those skilled in the art, the servers should be logically divided. In physical space, these servers can be independent of each other but can be called through interfaces, or integrated into a physical computer or a set of computer clusters. Those skilled in the art should understand this flexibility, and should not use it to restrict the implementation of the network deployment method of this application.

[0056] The technical solution of the present application uses the operating environment of the e-commerce service platform as its application environment. The e-commerce platform can realize its corresponding independent station for each merchant to run its corresponding online store, and provide the merchant with a corresponding business management engine instance for the merchant to establish, maintain, and run one or more online stores in one or more independent stations. The business management engine instance can be used for content management, task automation, and data management of one or more online stores, and various specific business processes of the online store can be configured through interfaces or built-in components to support the implementation of business activities. The independent station is the infrastructure of the e-commerce platform with cross-border service functions. Merchants can maintain their online stores more centrally and autonomously based on the independent station. Independent stations usually have a domain name and storage space dedicated to merchants, and different independent stations are relatively independent. The e-commerce platform can provide standardized or personalized technical support for a large number of independent stations, so that merchant users can customize their own business management engine instance and use this business management engine instance to maintain one or more online stores they own.

[0057] The e-commerce service platform may be an e-commerce service platform that opens independent site services, such as a cross-border e-commerce service platform. Such platforms need to consider the network environment between different regions of the world and the independence between merchants, and configure each merchant's store as an independent site, so that the e-commerce service platform serves a large number of such independent sites. The independent station store refers to an e-commerce independent station in an e-commerce service platform. The independent station store is different from the e-commerce store in the traditional e-commerce platform. The independent station store has an independent e-commerce online service. The independent station store can be regarded as a traditional e-commerce platform. The buyer users of the independent station store are its own user group, which is different from the buyer users of each store in the traditional e-commerce platform, which is the user group owned by the platform. The independent station e-commerce platform will provide corresponding network e-commerce functions to the e-commerce independent stations under the platform. In the independent station operation background, the search service is a core component that the independent station relies heavily on. Tens of thousands of independent station stores share the search service, which provides consumers with accurate and real-time independent station store data retrieval capabilities. It carries a heavier data detection task. In the field of independent station e-commerce platforms, due to the openness of independent station construction, merchants can search for independent station store data such as product information, product titles, store titles or classification information. According to custom fields, merchants can freely and flexibly add field names, field types and field values ​​to create custom fields. At the same time, consumer users can also use these custom fields for searching. In the search engine of the independent station e-commerce platform, the custom field information in each independent station store will be processed to generate its corresponding index. When consumer users search for each independent station store data, they can directly search for the searchable fields of the index to quickly find the independent station store data corresponding to the custom field. It is not difficult to understand that when merchants increase the custom fields infinitely, each additional custom field will make the overall index larger, which is extremely easy to cause field expansion and affect the stability of the cluster. Moreover, when the field names of the custom fields between independent station stores are the same but the field types are different, the custom field will not be able to be written into the search engine of the independent station e-commerce platform, and there will be mutual influence between independent station stores.

[0058] See also Figure 1 In one embodiment, the search service dynamic field storage method of the present application includes the following steps:

[0059] Step S10, in response to a search data synchronization event, obtaining store data of each independent station sent from an upstream system;

[0060] The independent station store data includes but is not limited to: characters, words, sentences or pictures, etc. The independent station store data can be product titles, store titles or product prices, etc. The search data synchronization service in the search service of the independent station e-commerce platform can respond to search data synchronization events and obtain the original data corresponding to each independent station store data continuously sent from the upstream system. The search data synchronization service is responsible for receiving the original data of each independent station store data in the upstream system, calling the field type identification and mapping public service in the search service to classify the original data corresponding to each independent station store data according to its corresponding data type, and then write it into the sub-fields corresponding to the field hierarchy structure of the full-text search engine middleware (Elasticsearch) in array format.

[0061] Step S20, parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determining the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types;

[0062] The field type identification and mapping public service in the search service of the independent station e-commerce platform can parse and enumerate the data types of the field values ​​of the custom fields corresponding to the store data of each independent station. The field type identification and mapping public service is responsible for classifying the original data corresponding to the store data of each independent station continuously sent by the upstream system according to its corresponding data type during search data synchronization, and providing the function of assembling the final field name according to the field data type during retrieval. The custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware (Elasticsearch) is determined according to the data type of the custom field. The data types include string, integer, floating point, etc. The custom data attribute is the data of the field value of the custom field corresponding to the store data of the independent station. Corresponding to the type, the custom data attribute includes one or more data types. For example, the data types of each independent station store data are enumerated as string, integer and floating point types. According to the data types of each independent station store data, it can be mapped to the custom data attributes of the full-text search engine middleware (Elasticsearch), which are keyword (string), long (integer) and double (floating point type), and further stored in the field hierarchy structure of the full-text search engine middleware (Elasticsearch). It should be noted that the custom data attributes exemplified here are only exemplary and do not constitute a limitation of the present invention. Those skilled in the art can understand that the custom data attribute can be named with any name that is different from the custom data attributes of other searchable fields.

[0063] In some embodiments, the field type identification and mapping public service in the search service parses and enumerates the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determines the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types, including the following steps: the field type identification and mapping public service in the search service responds to the field hierarchy structure generation request to build the field hierarchy structure corresponding to the custom field, and the field hierarchy structure of the full-text search engine middleware (Elasticsearch) can adopt the following structure:

[0064] The field hierarchy structure includes an outer layer and an inner layer, the outer layer is used to identify the custom field, and the inner layer contains one or more subfields, each subfield is used to store the field name of the custom field and the field value of its corresponding custom data attribute, and the subfield corresponds to the custom data attribute one by one.

[0065] Step S30: determining the corresponding subfield of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure;

[0066] After determining the custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware according to the data type of the custom field corresponding to the store data of each independent station, determine the custom data attribute corresponding to the custom field in the field hierarchy structure of the full-text search engine middleware (Elasticsearch) based on the data type of the field value of the custom field in the store data of each independent station enumerated above, determine the subfield of the field hierarchy structure corresponding to the custom field according to the field value of the custom field and the custom data attribute corresponding to the field value, the subfield of the field hierarchy structure represents a custom data attribute of the custom field, the custom attribute includes one or more data types, and a subfield of the field hierarchy structure includes one or more custom fields, And write the field name and field value of the custom field into the subfields of the field hierarchy structure, and the search data synchronization service in the search service writes the custom fields in the independent site store data in array format into the subfields corresponding to the field hierarchy structure of the full-text search engine middleware (Elasticsearch). It is not difficult to understand that the custom data attributes in the subfields of the field hierarchy structure of the full-text search engine middleware (Elasticsearch) are classified according to the data types corresponding to the custom fields in the independent site store data, and the custom fields in the independent site store data of each data type are written into the subfields of the field hierarchy structure of the full-text search engine middleware (Elasticsearch) according to the pre-classified custom attributes.

[0067] In some embodiments, see Figure 2 The field hierarchy structure of the full-text search engine middleware (Elasticsearch) may include an outer layer and an inner layer, etc., wherein the outer metafield field is used to identify a custom field, and the inner layer has subfields of the data types of the independent station store data enumerated above, and each subfield may be a nested structure, etc. There are two key-value fields in the nested structure, and the type of the field name key is keyword (string), which is used to store the field name of the custom field, and the type of the field value value is bound to the meaning of the field name defined in the outer layer, and the type of the field value value is set to the custom data attribute keyword (string), long (integer) and double (floating point) of the full-text search engine middleware (Elasticsearch) according to the data type classification of the independent station store data, which is used to store the field values ​​of custom fields of different data types.

[0068] Step S40, responding to an online query instruction carrying a custom field, determining the custom data attributes and searchable field names of the custom field in the field hierarchy structure of the full-text search engine middleware, and determining the field value corresponding to the custom field according to the custom data attributes and searchable field name.

[0069] The online query service in the search service can be responsible for assembling the final search statement. The online query service can determine the retrievable field name and custom data attribute of the custom field in the independent site store data in the subfield corresponding to the field hierarchy structure of the full-text search engine middleware (Elasticsearch) by calling the field type identification and mapping public service in the search service. Based on the final retrievable field name and custom data attribute, the nested search method provided by the full-text search engine middleware (Elasticsearch) is used to obtain the field value matching the custom field according to the subfield specified by the retrievable field name and custom data attribute in the field hierarchy structure of the full-text search engine middleware (Elasticsearch). Based on the field value corresponding to the custom field, the search engine in the search service is accessed to determine its corresponding product data.

[0070] It can be seen from the above embodiments that, compared with the prior art, the present application aims to solve the problems that when merchants add the custom fields, the overall index becomes larger, which is extremely easy to cause field expansion, affects the stability of the cluster, and the custom fields between independent stations and stores affect each other. The present application obtains and parses the custom fields in the independent station store data, determines the data type corresponding to the custom field, and thus determines the subfield specified by the custom field in the field hierarchy structure of the full-text search engine middleware (Elasticsearch), in which each subfield of the field hierarchy structure stores all field values ​​in the searchable fields of each independent station store data and the custom data attributes corresponding to the field value in a preset format, and stores different The custom data attributes corresponding to all field values ​​in the searchable fields of the independent station store data avoid the situation where different independent station stores use custom fields with the same field name but different field value types. The custom fields will not be able to be written into the search engine of the independent station e-commerce platform, avoiding the problem of index superposition and amplification caused by custom fields of different data types in each independent station store, keeping the index at a constant level, thereby increasing the index space used for custom fields without generating meaningless space occupation, greatly saving the memory resources of the search service of the independent station e-commerce platform, and storing the custom data attributes corresponding to all field values ​​in the searchable fields of different independent station store data in the same subfield of the field hierarchy structure of the full-text search engine middleware (Elasticsearch), including but not limited to the following beneficial effects:

[0071] First, to avoid the risk of field explosion, the maximum number of fields is equal to the number of field types that can be enumerated by the full-text search engine middleware (Elasticsearch);

[0072] Second, there is no restriction on search methods, and it can support any search method supported by the full-text search engine middleware (Elasticsearch);

[0073] Third, the structure is universal, which avoids the problem of different independent station stores having the same custom fields interfering with each other;

[0074] Fourth, iteration is efficient. If no new field types are added, the structure can be used all the time. If a new field data type needs to be added, you only need to synchronously add a new type of processing flow.

[0075] Furthermore, when searching for independent site store data, custom data attributes of the matched field values ​​are obtained by parsing the custom fields in the independent site store data, thereby dynamically acquiring custom data attributes. Therefore, when adding custom data attributes to custom fields in the independent site store data, the server memory resources occupied by the index in the search engine are greatly saved.

[0076] Based on any embodiment of this application, please refer to Figure 3 , in response to the search data synchronization event, the steps of obtaining the store data of each independent station sent from the upstream system include the following steps:

[0077] Step S101, responding to a timed monitoring event and determining whether there is a custom field in the store data of each independent station sent by the upstream system;

[0078] The search data synchronization service in the search service of the independent station e-commerce platform responds to the search data synchronization event to obtain the original data corresponding to the store data of each independent station continuously sent from the upstream system, and writes the store data of each independent station into the full-text search engine middleware (Elasticsearch). At the same time, the search data synchronization service responds to the timed monitoring event to determine whether there is a custom field in the independent station store data.

[0079] Step S103: If there is a custom field in the upstream original data, determine the corresponding subfield of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the subfield of the field hierarchy structure.

[0080] When the upstream system synchronizes the store data of each independent station to the full-text search engine middleware (Elasticsearch) in the search service, if the store data of each independent station synchronized to the full-text search engine middleware (Elasticsearch) contains custom fields, the corresponding subfield of the field hierarchy structure is determined according to the field value of the custom field and the custom data attribute corresponding to the field value, the subfield of the field hierarchy structure represents a custom data attribute of the custom field, and a subfield of the field hierarchy structure contains one or more custom fields, then the field name and field value of the custom field in the independent station store data are written into the subfield of the custom data attribute corresponding to the field value of the field hierarchy structure according to the specified encoding method, the custom data attribute in the subfield of the field hierarchy structure of the full-text search engine middleware (Elasticsearch) is classified according to the data type corresponding to the field value of the custom field in the independent station store data, and the custom data attribute contains one or more data types.

[0081] In some embodiments, the data type of the custom field includes string type, integer, and floating point type, etc., so that the subfield of the field hierarchy structure corresponding to the custom field can be used to store the field value of the custom field and its corresponding custom data attributes, which are combined in a specified format, and the like.

[0082] It can be seen from the above embodiments that this embodiment determines whether there are custom fields in the data of each independent station store sent by the upstream system by responding to the timed monitoring event, determines the corresponding sub-field of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and writes the field name and field value of the custom field into the sub-field of the field hierarchy structure, which can avoid the risk of field explosion and the problem of index superposition and amplification caused by custom fields of different data types in each independent station store, and keeps the index at a constant level, thereby increasing the index space used for custom fields without generating meaningless space occupation, greatly saving the memory resources of the search service of the independent station e-commerce platform, and the maximum number of fields is equivalent to the number of field types that can be enumerated by the full-text search engine middleware (Elasticsearch).

[0083] Based on any embodiment of this application, please refer to Figure 4 The step of determining the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure comprises the following steps:

[0084] Step S301, in response to a custom field storage event, a key-value data structure is used to encode the field value of the custom field and the corresponding custom data attribute to determine a character string or a character string array in a preset format;

[0085] When the upstream system synchronizes the data of each independent station store to the full-text search engine middleware (Elasticsearch) in the search service, if the custom fields in the data of each independent station store synchronized to the full-text search engine middleware (Elasticsearch) contain one or more field values ​​to which custom data attributes are to be added, a key-value data structure is used to encode the one or more field values ​​of the custom fields in the data of each independent station store synchronized to the full-text search engine middleware (Elasticsearch) and the custom data attributes corresponding to the one or more field values ​​to obtain a string or a string array in a specified format, and the string or the string array is written to the subfield specified by the field hierarchy structure of the full-text search engine middleware.

[0086] Step S303: Store the character string or character string array in the preset format in a custom field of a subfield specified by the field hierarchy structure of the full-text search engine middleware.

[0087] After determining a string or string array in a preset format, when the field value of a custom field in the store data of each independent station synchronized to the full-text search engine middleware (Elasticsearch) is one, a key-value data structure can be used to encode the field value of the custom field in the store data of each independent station synchronized to the full-text search engine middleware (Elasticsearch) and the custom data attribute corresponding to the field value to obtain a string in a specified format, and then the string in the specified format is saved in the sub-field specified by the field hierarchy structure of the full-text search engine middleware (Elasticsearch); when the field value of a custom field in the store data of each independent station synchronized to the full-text search engine middleware (Elasticsearch) is multiple, a key-value data structure can be used to encode the multiple field values ​​of the custom field in the store data of each independent station synchronized to the full-text search engine middleware (Elasticsearch) and the custom data attribute corresponding to the multiple field values ​​to obtain a string array in a specified format, and then the string array in the specified format is saved in the sub-field specified by the field hierarchy structure of the full-text search engine middleware (Elasticsearch).

[0088] It can be seen from the above embodiments that a key-value data structure is used to encode the one or more field values ​​of the custom fields in the data of each independent station store synchronized to the full-text search engine middleware (Elasticsearch) and the custom data attributes corresponding to the one or more field values ​​to obtain a string or string array in a specified format, and the string or string array is written into the subfield specified by the field hierarchy structure of the full-text search engine middleware. The custom data attributes corresponding to all field values ​​in the searchable fields of different independent station store data are stored in the same subfield of the field hierarchy structure of the full-text search engine middleware (Elasticsearch). This avoids the situation where the field names of custom fields used by different independent station stores are the same but the field value types are different, and the custom fields cannot be written into the search engine of the independent station e-commerce platform. This avoids the problem of index superposition and amplification caused by custom fields of different data types of each independent station store, so that the index is kept at a constant level, thereby increasing the index space used for custom fields without generating meaningless space occupation, greatly saving the memory resources of the search service of the independent station e-commerce platform.

[0089] Based on any embodiment of this application, please refer to Figure 5 , the step of parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station includes the following steps:

[0090] Step S201, responding to the custom field parsing instruction, calling the preset function to parse the custom fields corresponding to the store data of each independent station;

[0091] The field type identification and mapping public service in the search service of the independent station e-commerce platform can call the preset function to parse and enumerate the data types of the field values ​​of the custom fields corresponding to the store data of each independent station. The field type identification and mapping public service is responsible for classifying the original data corresponding to the store data of each independent station continuously sent by the upstream system according to their corresponding data types when searching for data synchronization, and providing the function of assembling the final field name according to the field data type during retrieval. The custom data attributes of the custom field in the field hierarchy structure of the full-text search engine middleware (Elasticsearch) are determined according to the data type of the field value of the custom field.

[0092] Step S203: Obtain and return the data type, field name, and field value corresponding to the field value of the custom field.

[0093] The field type identification and mapping public service in the search service of the independent site e-commerce platform can call the preset function to parse and enumerate the data types of the field values ​​of the custom fields corresponding to the data of each independent site store, and then obtain and return the data type, field name and field value corresponding to the field value of the custom field.

[0094] Based on any embodiment of this application, please refer to Figure 6 , the step of responding to an online query instruction carrying a custom field includes the following steps:

[0095] Step S401: responding to an online query instruction carrying a custom product keyword, parsing and determining the custom product keyword in the online query instruction and its corresponding data type;

[0096] The online query service in the search service responds to the online query instruction carrying the custom product keyword, and calls the field type identification and mapping public service in the search service to parse and determine the custom product keyword in the online query instruction and its corresponding data type.

[0097] Step S403: determining the custom data attribute and the searchable field name of the custom product keyword in the field hierarchy structure of the full-text search engine middleware according to the data type corresponding to the custom product keyword;

[0098] The field type identification and mapping public service in the search service determines the custom data attributes and searchable field names of the custom product keywords in the field hierarchy structure of the full-text search engine middleware according to the data types corresponding to the custom product keywords after determining the custom product keywords in the online query instruction.

[0099] Step S405: Search in the subfields specified by the full-text search engine middleware based on the custom data attributes corresponding to the custom product keyword and the searchable field name to find a field value matching the custom product keyword.

[0100] After determining the custom data attributes and retrievable field names of the custom product keyword in the field hierarchy structure of the full-text search engine middleware, the online query service in the search service searches in the subfields specified by the full-text search engine middleware based on the custom data attributes and retrievable field names corresponding to the custom product keyword, finds the field value that matches the custom product keyword, and accesses the search engine in the search service based on the field value corresponding to the custom product keyword to determine the corresponding product data.

[0101] It can be seen from the above embodiments that when searching for independent station store data, the custom data attributes of the matched field values ​​are obtained by parsing the custom fields in the independent station store data, thereby dynamically acquiring the custom data attributes. Therefore, when adding custom data attributes to the custom fields in the independent station store data, the server memory resources occupied by the index in the search engine are greatly saved.

[0102] See also Figure 7 A dynamic field storage device for a search service provided to meet one of the purposes of the present application includes a data acquisition module 1100, a field parsing module 1200, a field storage module 1300 and an online query module 1400. Among them, the data acquisition module 1100 is configured to respond to the search data synchronization event and obtain the store data of each independent station sent from the upstream system; the field parsing module 1200 is configured to parse and enumerate the data type of the field value of the custom field corresponding to the store data of each independent station, and determine the custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware according to the data type; the field storage module 1300 is configured to determine the corresponding sub-field of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the sub-field of the field hierarchy structure; the online query module 1400 is configured to respond to the online query instruction carrying the custom field, determine the custom data attribute and the searchable field name of the custom field in the field hierarchy structure of the full-text search engine middleware, and determine the field value corresponding to the custom field according to the custom data attribute and the searchable field name.

[0103] Based on any embodiment of the present application, the field parsing module 1200 further includes:

[0104] A field structure construction unit is configured to construct a field hierarchy structure corresponding to the custom field in response to a field hierarchy structure generation request, wherein the field hierarchy structure includes an outer layer and an inner layer, wherein the outer layer is used to identify the custom field, and the inner layer includes one or more subfields, each subfield is used to store the field name of the custom field and the field value of its corresponding custom data attribute, and the subfield corresponds one-to-one to the custom data attribute.

[0105] Based on any embodiment of the present application, the data acquisition module 1100 further includes:

[0106] The field monitoring unit is configured to respond to a timed monitoring event and determine whether there is a custom field in the store data of each independent station sent by the upstream system. If there is a custom field in the upstream original data, the subfield of the field hierarchy structure corresponding to the custom field is determined according to the field value of the custom field and the custom data attribute corresponding to the field value, and the field name and field value of the custom field are written into the subfield of the field hierarchy structure.

[0107] Based on any embodiment of the present application, the field storage module 1300 includes:

[0108] A field encoding unit, configured to encode the field value of the custom field and the corresponding custom data attribute using a key-value data structure in response to a custom field storage event, and determine a character string or a character string array in a preset format;

[0109] The storage unit is configured to store the character string or character string array in the preset format in a custom field of a subfield specified by the field hierarchy structure of the full-text search engine middleware.

[0110] Based on any embodiment of the present application, the field parsing module 1200 includes:

[0111] The parsing unit is configured to respond to the custom field parsing instruction and call the preset function to parse the custom fields corresponding to the store data of each independent station;

[0112] The acquisition unit is configured to acquire and return the data type, field name and field value corresponding to the field value of the custom field.

[0113] Based on any embodiment of the present application, the online query module 1400 includes:

[0114] A query instruction parsing unit, configured to respond to an online query instruction carrying a custom product keyword, and parse and determine the custom product keyword in the online query instruction and its corresponding data type;

[0115] An attribute determination unit, configured to determine a custom data attribute and a searchable field name of the custom product keyword in a field hierarchy structure of the full-text search engine middleware according to a data type corresponding to the custom product keyword;

[0116] The retrieval unit is configured to search in the subfields specified by the full-text search engine middleware based on the custom data attributes corresponding to the custom product keywords and the searchable field names, and find the field values ​​matching the custom product keywords.

[0117] Based on any embodiment of this application, please refer to Figure 8 Another embodiment of the present application further provides an electronic device, which can be implemented by a computer device, such as Figure 8 As shown, a schematic diagram of the internal structure of a computer device. The computer device includes a processor, a computer-readable storage medium, a memory, and a network interface connected via a system bus. Among them, the computer-readable storage medium of the computer device stores an operating system, a database, and computer-readable instructions. The database may store a control information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a search service dynamic field storage method. The processor of the computer device is used to provide computing and control capabilities to support the operation of the entire computer device. The memory of the computer device may store computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor may execute the search service dynamic field storage method of the present application. The network interface of the computer device is used to connect and communicate with a terminal. Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0118] In this embodiment, the processor is used to execute Figure 7 The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to transmit data between user terminals or servers. The memory in this embodiment stores the program code and data required to execute all modules / submodules in the search service dynamic field storage device of this application, and the server can call the program code and data of the server to execute the functions of all submodules.

[0119] The present application also provides a storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the search service dynamic field storage method described in any embodiment of the present application.

[0120] The present application also provides a computer program product, including a computer program / instruction, which, when executed by one or more processors, implements the steps of the search service dynamic field storage method described in any embodiment of the present application.

[0121] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0122] The above description is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

[0123] To summarize, when searching for independent site store data, the custom data attributes of the matched field values ​​are obtained by parsing the custom fields in the independent site store data, so as to dynamically obtain the custom data attributes. Therefore, when adding custom data attributes to the custom fields in the independent site store data, the server memory resources occupied by the index in the search engine are greatly saved.

Claims

1. A method for storing dynamic fields of a search service, characterized in that: The steps include: Responding to a search data synchronization event to obtain the data of each independent station store sent from an upstream system, comprising the following steps: responding to a timed monitoring event to determine whether there is a custom field in the data of each independent station store sent by the upstream system; if there is a custom field in the upstream original data, determining the corresponding subfield of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure; Parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, including the following steps: responding to the custom field parsing instruction, calling the preset function to parse the custom fields corresponding to the store data of each independent station; obtaining and returning the data type, field name and field value corresponding to the field value of the custom field; determining the custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware according to the data type; Determine the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the subfield of the field hierarchy structure; In response to an online query instruction carrying a custom field, the custom data attributes and searchable field names of the custom field in the field hierarchy of the full-text search engine middleware are determined, and the field value corresponding to the custom field is determined according to the custom data attributes and searchable field name.

2. The search service dynamic field storage method according to claim 1, characterized in that: The step of parsing and enumerating the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and determining the custom data attributes of the custom fields in the field hierarchy structure of the full-text search engine middleware according to the data types, comprises the following steps: In response to a field hierarchy generation request, a field hierarchy corresponding to the custom field is constructed, the field hierarchy including an outer layer and an inner layer, the outer layer is used to identify the custom field, the inner layer contains one or more subfields, each subfield is used to store the field name of the custom field and the field value of its corresponding custom data attribute, the subfield corresponds one-to-one to the custom data attribute.

3. The search service dynamic field storage method according to claim 1, characterized in that: The step of determining the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure comprises the following steps: In response to a custom field storage event, a key-value data structure is used to encode the field value of the custom field and the corresponding custom data attribute to determine a character string or a character string array in a preset format; The character string or character string array in the preset format is stored in a custom field of a subfield specified by a field hierarchy structure of the full-text search engine middleware.

4. The search service dynamic field storage method according to claim 1, characterized in that: The step of responding to an online query instruction carrying a custom field includes the following steps: In response to an online query instruction carrying a custom product keyword, parsing and determining the custom product keyword in the online query instruction and its corresponding data type; Determine the custom data attribute and searchable field name of the custom product keyword in the field hierarchy structure of the full-text search engine middleware according to the data type corresponding to the custom product keyword; Based on the custom data attribute corresponding to the custom product keyword and the searchable field name, a search is performed in the subfield specified by the full-text search engine middleware to find a field value matching the custom product keyword.

5. The method for storing dynamic fields of a search service according to any one of claims 1 to 4, characterized in that: The independent website store data includes one or more of the following data: product title, store title or product price.

6. A search service dynamic field storage device, characterized in that: include: The data acquisition module is configured to acquire the store data of each independent station sent from the upstream system in response to the search data synchronization event, and includes the following steps: in response to the timed monitoring event, determining whether there is a custom field in the store data of each independent station sent by the upstream system; if there is a custom field in the upstream original data, determining the corresponding subfield of the field hierarchy structure according to the field value of the custom field and the custom data attribute corresponding to the field value, and writing the field name and field value of the custom field into the subfield of the field hierarchy structure; The field parsing module is configured to parse and enumerate the data types of the field values ​​of the custom fields corresponding to the store data of each independent station, and includes the following steps: responding to the custom field parsing instruction, calling the preset function to parse the custom fields corresponding to the store data of each independent station; obtaining and returning the data type, field name and field value corresponding to the field value of the custom field; determining the custom data attribute of the custom field in the field hierarchy structure of the full-text search engine middleware according to the data type; A field storage module, configured to determine the subfield of the field hierarchy structure corresponding to the field value of the custom field and the custom data attribute corresponding to the field value, and write the field name and field value of the custom field into the subfield of the field hierarchy structure; The online query module is configured to respond to online query instructions carrying custom fields, determine the custom data attributes and retrievable field names of the custom fields in the field hierarchy structure of the full-text search engine middleware, and determine the field value corresponding to the custom field based on the custom data attributes and retrievable field name.

7. An electronic device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 5 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images