A data query method, device and readable storage medium thereof

Abstract

The present invention discloses a data query method, device and readable storage medium thereof, the method comprising: receiving a query instruction with a target document number, performing segmented compression of the document information on the document list in the predetermined order according to the query instruction to obtain multiple data compression blocks, obtaining a target compressed data block according to the target document number and the tag, decompressing the target compressed data block to obtain a document list of the corresponding segment; and querying data of the document information corresponding to the target document number from the document list of the corresponding segment. The document list is segmented and compressed, and only the content of the corresponding segment in the list is queried, which greatly reduces the amount of data processing, solves the technical problem that the memory consumption surges, resulting in a significant increase in time consumption and long response time when querying inverse data, and affects the user experience.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a data query method, device and readable storage medium thereof. Background Art

[0002] At present, keyword search is an important technical field in modern computer technology. For example, when shopping on e-commerce websites, you often need to enter keywords to search. Accordingly, e-commerce websites will display the specific products that users want. The reason why keyword search can accurately find the target product is that it is achieved by constructing a full inverted index for the website products, so the full inverted index is an important search method.

[0003] The full inverted index technology indexes the file list of the search keyword and the location where the keyword appears, and calls the corresponding document in the file list through the keyword query, as well as the specific content in the document for display. However, when the full inverted index data volume is huge, the memory consumption will also surge, and the time consumption will increase significantly when querying the inverted data, and the response time will be long, which affects the user experience.

[0004] Therefore, the inventors have found that there are at least the following problems in the prior art: when the amount of full inverted data is huge, memory consumption will also surge, resulting in a significant increase in the time consumption and response time when querying inverted data, which affects the user experience. Summary of the invention

[0005] The present application provides a data query method, the purpose of which is to compress a document list into segments, and greatly reduce the amount of data processing by only querying the content of the corresponding segment in the list.

[0006] The method includes:

[0007] Receiving a query instruction with a target document number, wherein the target document number is a document number corresponding to the pre-queried document information, the document number is a sequence number corresponding to the document information in a pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source;

[0008] According to the query instruction, the document list is segmented and compressed in the predetermined order to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark recording the arrangement position of the compressed corresponding segment document information in the document list;

[0009] Acquire a target compressed data block according to the target document number and the mark, wherein the target compressed data block is a compressed data block containing document information corresponding to the target document number;

[0010] Decompressing the target compressed data block to obtain a document list of the corresponding segment;

[0011] The data of the document information corresponding to the target document number is searched from the document list of the corresponding segment.

[0012] Optionally, the mark includes the maximum document number or the minimum document number of the file information in the corresponding segment of the file list compressed by the data compression block, and the sequence mark of the data compression block, and the sequence mark is the sequence number of the corresponding segment document information in the file list compressed by the data compression block.

[0013] Optionally, acquiring a target compressed data block according to the target document number and the tag includes:

[0014] Determine the numerical range of the sequence numbers of the file information in the compressed corresponding segment of each data compression block in the file list according to the maximum file number or the minimum file number and the sequence number;

[0015] The data compression block corresponding to the target document number belonging to the serial number data interval is obtained, and the data compression block is used as the target compressed data block.

[0016] Optionally, each piece of the document information further has predetermined query data location information, wherein the query data location information is used to locate the document query data location corresponding to the link address;

[0017] After the step of searching the document list of the corresponding segment for the data of the document information corresponding to the target document number, the method further comprises:

[0018] Get the link address of the corresponding file information;

[0019] Get the corresponding document on the network according to the link address;

[0020] The acquired document is located according to the query data location information.

[0021] Optionally, the same link address of the document information is stored in a storage unit.

[0022] The present application also provides a data query, the device also includes:

[0023] A receiving unit, configured to receive a query instruction having a target document number, wherein the target document number is a document number corresponding to the pre-queried document information, the document number is a sequence number corresponding to the document information in a pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source;

[0024] A compression unit, configured to perform segmented compression of the document information in the document list in the predetermined order according to the query instruction to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark recording an arrangement position of the compressed corresponding segment of document information in the document list;

[0025] An acquiring unit, configured to acquire a target compressed data block according to the target document number and the mark, wherein the target compressed data block is a compressed data block containing document information corresponding to the target document number;

[0026] A decompression unit, used for decompressing the target compressed data block to obtain a document list of a corresponding segment;

[0027] The query unit is used to query the data of the document information corresponding to the target document number from the document list of the corresponding segment.

[0028] The present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the data query method are implemented.

[0029] As can be seen above, based on the above embodiment, by compressing the document list in segments and only querying the content of the corresponding segment in the list, the data processing amount is greatly reduced, and the technical problem of a surge in memory consumption, which leads to a significant increase in the time consumption and long response time when querying inverse data, affecting the user experience is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a process 100 of a method for querying data in the present application;

[0031] Figure 2 This is a schematic diagram of the first-level index of this application;

[0032] Figure 3 This is a schematic diagram of the secondary index for this application;

[0033] Figure 4 This is a schematic diagram of the three-level index of this application;

[0034] Figure 5 This is a schematic diagram of the list of documents for this application;

[0035] Figure 6 This is a schematic diagram of the data structure of segmented compression for this application;

[0036] Figure 7 A schematic diagram of a process 200 of a method for querying data in the present application;

[0037] Figure 8 A schematic diagram of a process 300 of a method for querying data in the present application;

[0038] Fig. 9 A schematic diagram of the structure of a data query device of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0040] First, a brief description of the full inverted index technology is given to facilitate subsequent explanations. It should be pointed out that the full inverted index technology is only an implementation method in the application of this application, and is not a specific limitation of this application.

[0041] For example, take the search for mobile phone products in e-commerce. Mobile phone can be used as a keyword for search. When mobile phone is used as a keyword for search through the search tool, mobile phones of different brands may be displayed on the display. In addition, some information on different displayed contents will be marked, such as highlighting certain content in the detailed description of the mobile phone.

[0042] Then, the background processing can be carried out through the following steps, the specific steps are:

[0043] First of all, the background stores multiple documents about mobile phones. These documents contain keywords such as "mobile phone". The documents can be understood as text descriptions of mobile phones of different brands, such as Apple phones, Huawei phones, etc.

[0044] Then, a document list will be created in the background. Generally speaking, the document list will be classified by keywords, and a document list will be created for documents that record the keyword "mobile phone". Specifically, a document number is matched for the document with the keyword, and the storage path corresponding to the document, that is, the link address. The document number and link address of each document will exist in the form of document information. Document information can be understood as pieces of data information. Since there are usually multiple pieces of document information, multiple pieces of document information are listed to form a document list.

[0045] During the search process, after the user enters a keyword, the background will find the document list according to the keyword of the document list. When searching, the file list will be called, and then the file list will display each document information. After display, it can be displayed as multiple pieces of document information in the display, such as Apple mobile phone, Huawei mobile phone, etc. When the user selects Apple mobile phone, Huawei mobile phone or other mobile phone again, that is, selects one of the document information, the link address corresponding to the document information can be selected to enter the document in the same document information.

[0046] The above is a brief introduction to the inverted index technology. It should be pointed out that in fact, the inverted index technology is widely used in the search field, so the technical solution of this application is not limited to product retrieval in e-commerce.

[0047] Figure 1 This is a schematic diagram of a process 100 of a method for querying data in this application. Figure 2 This is a schematic diagram of the first-level index of this application. Figure 3 This is a schematic diagram of the secondary index for this application. Figure 4 This is a schematic diagram of the three-level index of this application. Figure 5 This is a schematic diagram of the list of application documents. Figure 6 This is a schematic diagram of the data structure of the segmented compression of this application. Figures 1 to 6 As shown, in one embodiment, the present application provides a data query method, the method comprising:

[0048] S101, receiving a query instruction with a target document number, wherein the target document number is a document number corresponding to pre-queried document information, the document number is a sequence number of the corresponding document information in a pre-built document list, the document list is a list of multiple pieces of document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source;

[0049] In this step, a specific method for receiving the target document number for subsequent query is provided. For the search field, according to the above description, when the user performs a keyword query, such as Figure 2 As shown in the figure, for example, when a user enters "mobile phone" in the search toolbar, a list of options for multiple brands of mobile phones will appear. The keyword input can be understood as a primary index, and when we enter the option list, it can be understood as a secondary index, such as Figure 3 As shown, the method of the present application is for the steps after entering the secondary index. When entering one of the option lists, the displayed content can be understood as a tertiary index. Figure 4 As shown, when an option in the option list is selected, it is equivalent to issuing a query instruction to query the document corresponding to the target document number, so the background needs to receive the query instruction with the target document number.

[0050] S102, according to the query instruction, segmentally compress the document information in the document list in the predetermined order to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark recording the arrangement position of the compressed corresponding segment of document information in the document list;

[0051] like Figure 5 Shown and Figure 6As shown, in this step, the document list is compressed in segments. For example, if there are 1,000 document information in the list, the first 200 are compressed into a data compression block, and the document information from 201 to 400 are compressed into a data compression block, and so on. All 1,000 document information are compressed in segments. It should be pointed out that the number of document information in each data compression block is not the same.

[0052] In addition, a header information can be loaded on each data compression block. The header information stores a mark, which is used to record the arrangement position of the compressed corresponding segment document information in the document list to facilitate subsequent processing.

[0053] For example, the first data compression block in the above example can record a mark, indicating that the document information in the data compression block is the first 200 document information.

[0054] It should be noted that the order of the file information of the corresponding segment compressed in each data compression block is not disrupted. For example, in the first data compression block mentioned above, the order from the 1st file information to the 200th file information still exists.

[0055] S103, acquiring a target compressed data block according to the target document number and the mark, wherein the target compressed data block is a compressed data block containing document information corresponding to the target document number;

[0056] In this step, the target compressed data block is obtained. Specifically, the document information corresponding to the target document number is found according to the tag, and then the compressed data block with the document information of the target document number is selected for subsequent processing.

[0057] S104, decompressing the target compressed data block to obtain a document list of the corresponding segment;

[0058] After obtaining the target compressed data block, the target compressed data block is decompressed. After decompression, a section of the document list compressed in the compressed data block can be obtained. The document information data in this section of the document list is not disrupted, and each document information has a number in the document list.

[0059] S105, searching the document list of the corresponding segment for data of document information corresponding to the target document number.

[0060] In this step, a query is performed from the document list decompressed from the target compressed data block according to the target document number, and then the data of the document information corresponding to the target document number is found.

[0061] In this embodiment, first, according to receiving a query instruction with a target document number, wherein the target document number is the document number corresponding to the pre-queried document information, the document number is the serial number of the corresponding document information in the pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source; then, according to the query instruction, the document list is segmented and compressed in the predetermined order to obtain multiple data compression blocks, wherein each of the data compression blocks has a mark for recording the arrangement position of the compressed corresponding segment document information in the document list; according to the target document number and the mark, a target compressed data block is obtained, wherein the target compressed data block is a compressed data block with the document information corresponding to the target document number; the target compressed data block is decompressed to obtain a document list of the corresponding segment; finally, the data of the document information corresponding to the target document number is queried from the document list of the corresponding segment.

[0062] The above method no longer queries all the document information in the document list, but only queries a section of the document list in a compressed data block, and finds the document information corresponding to the target file number. In addition, the compression of the document list greatly reduces the amount of data processing, and solves the technical problem that the full inverted index in the existing technology consumes a surge in memory in a big data environment, resulting in a significant increase in the time consumption and long response time when querying inverted data, which affects the user experience.

[0063] In one embodiment, the mark includes the maximum document number or the minimum document number of the file information in the corresponding segment of the file list compressed by the data compression block, and the sequence mark of the data compression block, and the sequence mark is the sequence number of the corresponding segment document information in the file list compressed by the data compression block.

[0064] This embodiment provides a specific content of a mark, and data positioning of a subsequent target document number can be performed according to the maximum document number or the minimum document number and the sequence mark of the data compression block.

[0065] In addition to the sequential marking method of data compression blocks, it can also be achieved through the number difference method. When each data compression block is loaded with the maximum document number or the minimum document number, if you know how many numbers are between the first document information and the last document information data in a data compression block, that is, the number difference, you can also determine whether the data compression block contains the document information corresponding to the target document number.

[0066] Figure 7 FIG. 2 is a schematic diagram of a process 200 of a method for querying data in the present application. Figure 7As shown, in one embodiment, obtaining a target compressed data block according to the target document number and the tag includes:

[0067] S201, determining the numerical range of the sequence numbers of the file information in the corresponding segment compressed by each data compression block in the file list according to the maximum file number or the minimum file number and the sequence number;

[0068] This step provides a specific method for establishing a document information number range based on the maximum document number or the minimum document number and the sequence number of the data compression block. For example, the maximum document number of the compressed data block with the sequence number 3 is 500, and the maximum document number of the previous data compression block, that is, the sequence number 2 is 300, then the sequence number data value range of the compressed data block file information with the sequence number 3 should be 301 to 500.

[0069] S202, obtaining a data compression block corresponding to the target document number belonging to the serial number data interval, and using the data compression block as the target compressed data block.

[0070] This step provides a specific method for obtaining the corresponding target compressed data block according to the target document number. According to the range of the serial number data value in the previous step, for example, the target document number is 325, and the serial number data value range is 301 to 500. Then, 325 is obviously within the range of 301 to 500, so the compressed data block with the serial number data value range of 301 to 500 is selected.

[0071] In this embodiment, a specific method for obtaining a target compressed data block according to the target document number and the tag is provided. First, the serial number value interval of the file information in the corresponding segment compressed by each data compression block in the file list is determined according to the maximum document number or the minimum document number and the sequential number; then the data compression block corresponding to the target document number belonging to the sequential number data interval is obtained, and the data compression block is used as the target compressed data block.

[0072] Figure 8 FIG. 300 is a schematic diagram of a process 300 of a method for querying data in the present application. Figure 8 As shown, in one embodiment, each piece of the document information also has predetermined query data location information, wherein the query data location information is used to locate the document query data location corresponding to the link address;

[0073] After the step of searching the document list of the corresponding segment for the data of the document information corresponding to the target document number, the method further comprises:

[0074] S301, obtaining the link address of the file information corresponding to the target document number;

[0075] In this step, a specific method for obtaining the link address of the file information corresponding to the target document number is provided. After the file information corresponding to the target document number is obtained according to the previous steps, the link address corresponding to the file information is obtained.

[0076] S302, obtaining a corresponding document in the network according to the link address;

[0077] According to the link address, the corresponding document can be connected, which can be understood as displaying the corresponding document content, and is now in the third level index.

[0078] S303: performing data location on the acquired document according to the query data location information.

[0079] According to the query data location information in the document information, some specific data or descriptions in the document are highlighted or extracted. The query data location information locates these highlighted or extracted specific data or descriptions, such as highlighting some descriptive language in the mobile phone or some content in the name, etc.

[0080] This embodiment can be understood as when a user enters the third-level index, the document corresponding to the file information needs to be identified, and the link address of the file information corresponding to the target document number is first obtained; then the corresponding document in the network is obtained according to the link address; finally, the obtained document is located according to the query data location information, and the located data can be highlighted or processed in other ways. It should be pointed out that not all solutions of this application have a third-level index. If there is no third-level index, there will be no method of this embodiment. Figure 4 As shown, the content of the brand A mobile phone displayed in gray can be understood as the specific content that needs to be displayed specified by querying the data location information.

[0081] In one embodiment, the same link address of the document information is stored in one storage unit.

[0082] In this embodiment, when the link address of the document information points to the same location, a copy of the same location is saved. When different document information calls the link address, the link addresses of different document information are pointed to the storage unit through the pointer, which greatly reduces the storage space of repeated link addresses and reduces redundant data. Figure 5 As shown, the link addresses corresponding to Field and Pos are displayed in gray. The link addresses point to the same location. Field A and Pos B can be stored in a storage unit. When calling the document information corresponding to Field A and Pos B, it is called through the storage unit pointed to by the pointer.

[0083] Fig. 9 This is a schematic diagram of the architecture of a data query device in this application. Fig. 9 As shown, in one embodiment, the present application further provides a data query device, which further includes:

[0084] The receiving unit 101 is used to receive a query instruction with a target document number, wherein the target document number is a document number corresponding to the pre-queried document information, the document number is a sequence number corresponding to the document information in a pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information with a link address of a pre-stored document in a data source;

[0085] A compression unit 102, configured to perform segmented compression of the document information on the document list in the predetermined order according to the query instruction to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark recording an arrangement position of the compressed corresponding segment of document information in the document list;

[0086] An acquiring unit 103 is configured to acquire a target compressed data block according to the target document number and the tag, wherein the target compressed data block is a compressed data block containing document information corresponding to the target document number;

[0087] A decompression unit 104, configured to decompress the target compressed data block to obtain a document list of a corresponding segment;

[0088] The query unit 105 is used to query the data of the document information corresponding to the target document number from the document list of the corresponding segment.

[0089] In one embodiment, a computer-readable storage medium stores a computer program, which implements the steps of the data query method when executed by a processor.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A data query method, characterized in that: The method includes: Receiving a query instruction with a target document number, wherein the target document number is a document number corresponding to the pre-queried document information, the document number is a sequence number corresponding to the document information in a pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source; According to the query instruction, the document list is segmented and compressed in the predetermined order to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark for recording the arrangement position of the compressed corresponding segment document information in the document list, wherein the compressed corresponding segment document information in each data compression block is not disrupted in order; The target compressed data block is obtained according to the target document number and the mark, wherein the target compressed data block is a compressed data block compressed with document information corresponding to the target document number, wherein the mark includes the maximum document number or the minimum document number of the file information in the corresponding segment of the file list compressed by the data compression block, and the sequence mark of the data compression block, wherein the sequence mark is the sequence number of the corresponding segment of the document information in the file list compressed by the data compression block in the file list, and the obtaining of the target compressed data block according to the target document number and the mark includes: Determine the numerical range of the sequence numbers of the file information in the compressed corresponding segment of each data compression block in the file list according to the maximum file number or the minimum file number and the sequence number; Acquire the data compression block corresponding to the target document number belonging to the sequence number data interval, and use the data compression block as the target compressed data block; Decompressing the target compressed data block to obtain a document list of the corresponding segment; The data of the document information corresponding to the target document number is searched from the document list of the corresponding segment.

2. The data query method according to claim 1, characterized in that: Each piece of document information also has predetermined query data location information, wherein the query data location information is used to locate the document query data location corresponding to the link address; After the step of searching the document list of the corresponding segment for the data of the document information corresponding to the target document number, the method further comprises: Obtain the link address of the file information corresponding to the target document number; Get the corresponding document on the network according to the link address; The acquired document is located according to the query data location information.

3. The data query method according to claim 2, characterized in that: The same link address of the document information is stored in a storage unit.

4. A data query device, characterized in that: The device is used to implement the data query method according to any one of claims 1 to 3, and the device also includes: A receiving unit, configured to receive a query instruction having a target document number, wherein the target document number is a document number corresponding to the pre-queried document information, the document number is a sequence number corresponding to the document information in a pre-built document list, the document list is a list of multiple pieces of the document information arranged in a predetermined order, and the document information is data information having a link address of a document pre-stored in a data source; A compression unit, configured to perform segmented compression of the document information on the document list in the predetermined order according to the query instruction to obtain a plurality of data compression blocks, wherein each of the data compression blocks has a mark recording an arrangement position of compressed corresponding segment document information in the document list, wherein the compressed corresponding segment document information in each data compression block has an order that is not disrupted; An acquiring unit, configured to acquire a target compressed data block according to the target document number and the mark, wherein the target compressed data block is a compressed data block containing document information corresponding to the target document number; A decompression unit, used for decompressing the target compressed data block to obtain a document list of a corresponding segment; The query unit is used to query the data of the document information corresponding to the target document number from the document list of the corresponding segment.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the data query method described in any one of claims 1 to 3 are implemented.

Images