A search result ranking position determination method, a mixed arrangement method and device

Abstract

The application discloses a search result sorting position determination method, a mixed arrangement method and device, and relates to the technical field of Internet, the search result sorting position determination method comprises the following steps: obtaining statistical data of a first type of search result corresponding to a keyword, the statistical data comprising exposure and first point rate, wherein the first point rate is the ratio of the first click volume to the exposure; based on the exposure, the first type of search result is divided into a high exposure group and a low exposure group; according to the first point rate, the high exposure group and the low exposure group are sorted respectively, a first high sorting result and a first low sorting result are obtained, and the results are combined to obtain a first sorting result. According to the grouping based on the exposure, the problem that the sorting result is unreasonable due to the high click rate caused by the low exposure is avoided. The first type of search result is sorted by using the first point rate instead of the click rate, so that the user's preference can be more truly reflected, and the problem that the result is inaccurate due to the direct use of the click rate sorting is further relieved.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a method for determining the ranking position of search results, a mixed ranking method, and an apparatus. Background Technology

[0002] In today's information age, search is one of the main ways for internet users to obtain information. With the continuous development of search technology, the information display formats of search results are becoming increasingly diversified. These include not only webpage cards containing text, images, and videos (natural flow), but also special cards composed of structured data or aggregated information (hybrid flow). As the number of hybrid flows in search results pages increases, the combined effect of natural and hybrid flows has a growing impact on the user's search experience.

[0003] Existing mixed ranking technology mainly sorts search results based on the click-through rate (CTR) of mixed and natural streams. However, the exposure position affects the CTR of the mixed stream. The earlier the exposure position, the greater the exposure and the lower the CTR. Conversely, the later the exposure position, the lower the exposure and the higher the CTR. So, sorting directly based on CTR can cause mixed streams with low exposure and high CTR to be ranked higher, resulting in inaccurate ranking results.

[0004] Therefore, there is an urgent need for a method to determine the location of search results in order to solve the problems existing in the above solutions. Summary of the Invention

[0005] Therefore, the present invention provides a method for determining the location of search results to solve or at least alleviate the problems mentioned above.

[0006] According to one aspect of the present invention, a method for determining the ranking position of search results is provided, suitable for execution in a computing device. The method includes: acquiring statistical data of a first type of search results corresponding to keywords, the statistical data including exposure volume and first-click rate, wherein the first-click rate is the ratio of the number of first clicks to the exposure volume; dividing the first type of search results into a high-exposure group and a low-exposure group based on the exposure volume; sorting the high-exposure group and the low-exposure group according to the first-click rate to obtain a first high-ranking result and a first low-ranking result; merging the first high-ranking result and the first low-ranking result to obtain a first ranking result of the first type of search results.

[0007] Optionally, in the method for determining the search result ranking position according to the present invention, before dividing the first type of search results into a high-exposure group and a low-exposure group based on the exposure volume, the method further includes: obtaining the click-through rate of the first type of search results and the second type of search results corresponding to the keywords; and filtering the first type of search results based on the click-through rate of the first type of search results and the second type of search results.

[0008] Optionally, in the method for determining the search result ranking position according to the present invention, filtering the first type of search results based on the click-through rates of the first type of search results and the second type of search results includes: calculating the average click-through rate of the second type of search results based on the click-through rate of the second type of search results; deleting search results in the first type of search results whose click-through rate is lower than the average click-through rate, thereby obtaining the filtered first type of search results.

[0009] Optionally, in the method for determining the search result sorting position according to the present invention, before dividing the first type of search results into a high-exposure group and a low-exposure group based on exposure, the method further includes: setting the fixed-position search results and / or newly added search results in the first type of search results to a default position according to business needs, and the default position will no longer participate in the subsequent search result sorting process.

[0010] Optionally, in the method for determining the search result ranking position according to the present invention, dividing the first type of search results into a high-exposure group and a low-exposure group based on the exposure amount includes: classifying the first type of search results with an exposure amount greater than or equal to a first threshold into a high-exposure group; and classifying the first type of search results with an exposure amount less than the first threshold into a low-exposure group.

[0011] Optionally, in the method for determining the search result ranking position according to the present invention, the first type of search results includes multiple mixed streams, each mixed stream being a knowledge card composed of multiple structured data; the second type of search results includes multiple natural streams, each natural stream being a web page card containing text, images and / or videos.

[0012] Optionally, in the method for determining the search result ranking position according to the present invention, the statistical data is data obtained by statistically analyzing the exposure and click data included in the exposure and click log within a preset period.

[0013] According to one aspect of the present invention, a search result ranking position determination device is provided, residing in a computing device, the device comprising: an acquisition module, adapted to acquire statistical data of a first type of search result and a second type of search result corresponding to a keyword, the statistical data including exposure volume, click-through rate, and first-click rate, wherein the first-click rate is the ratio of the first click volume to the exposure volume; a classification module, adapted to divide the first type of search result into a high-exposure group and a low-exposure group based on the exposure volume; a ranking module, adapted to rank the high-exposure group and the low-exposure group respectively according to the first-click rate to obtain a first high-ranking result and a first low-ranking result; and a processing module, adapted to merge the first high-ranking result and the first low-ranking result to obtain a first ranking result.

[0014] According to one aspect of the present invention, a method for merging search results is provided, suitable for execution in a computing device, wherein the computing device stores a first ranking result obtained by the method for determining the ranking position of search results according to the present invention. The method includes: receiving a search request, the search request including keywords; obtaining a first type of search results and a second type of search results corresponding to the keywords; ranking the second type of search results according to click-through rate to obtain a second ranking result; merging the first ranking result of the first type of search results and the second ranking result of the second type of search results to obtain a final ranking result; and merging the first type of search results and the second type of search results according to the final ranking result to obtain a final search result.

[0015] Optionally, in the search result merging method according to the present invention, merging the first sorting result and the second sorting result to obtain the final sorting result includes: determining the position of each search result in the first sorting result according to a preset ratio, with the remaining positions being the remaining positions; filling each search result in the second sorting result into the remaining positions to obtain the final sorting result.

[0016] Optionally, in the search result mixing method according to the present invention, the preset ratio is 1:3.

[0017] Optionally, in the search result mixing method according to the present invention, the first type of search results includes multiple mixed streams, each mixed stream being a knowledge card composed of multiple structured data; the second type of search results includes multiple natural streams, each natural stream being a web page card containing text, images and / or videos.

[0018] According to one aspect of the present invention, a computing device is provided, comprising: at least one processor; and a memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the search result ranking position determination method and the search result mixing method according to the present invention.

[0019] According to one aspect of the present invention, a readable storage medium storing program instructions is provided, which, when read and executed by a computing device, causes the computing device to perform the search result sorting position determination method and the search result mixing method according to the present invention.

[0020] According to the technical solution of the present invention, a method for determining the ranking position of search results involves dividing a first category of search results into a high-exposure group and a low-exposure group based on exposure volume. Then, based on the first-click rate, the high-exposure group and the low-exposure group are ranked separately to obtain a first high-ranking result and a first low-ranking result. Finally, the first high-ranking result and the first low-ranking result are merged to obtain a first ranking result. Dividing the first category of search results based on exposure volume before ranking them separately avoids the problem of unreasonable ranking results caused by high click-through rates due to low exposure volume. Using the first-click rate instead of the click-through rate to rank the first category of search results more accurately reflects user preferences. Since the first-click rate takes into account the impact of exposure volume on the click-through rate, it further alleviates the problem of inaccurate results caused by directly using the click-through rate for ranking.

[0021] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0022] To achieve the foregoing and related objectives, certain illustrative aspects are described herein in conjunction with the following description and accompanying drawings. These aspects indicate various ways in which the principles disclosed herein may be practiced, and all aspects and their equivalents are intended to fall within the scope of the claimed subject matter. The foregoing and other objectives, features, and advantages of this disclosure will become more apparent from the following detailed description, taken in conjunction with the accompanying drawings. Throughout this disclosure, the same reference numerals generally refer to the same parts or elements.

[0023] Figure 1 A schematic diagram of a search result sorting position determination system 100 according to an embodiment of the present invention is shown;

[0024] Figure 2 A schematic diagram of a computing device 200 according to an embodiment of the present invention is shown;

[0025] Figure 3 A flowchart illustrating a method 300 for determining the sorting position of search results according to an embodiment of the present invention is shown.

[0026] Figure 4 A flowchart illustrating a search result mixing method 400 according to an embodiment of the present invention is shown;

[0027] Figure 5 A flowchart illustrating a search result mixing method 500 according to yet another embodiment of the present invention is shown;

[0028] Figure 6A schematic diagram of a search result sorting position determination device 600 according to an embodiment of the present invention is shown. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] Figure 1 A schematic diagram of a search result ranking position determination system 100 according to an embodiment of the present invention is shown. The search result ranking position determination system 100 includes a user terminal 110 and a computing device 200.

[0031] User terminal 110 refers to the terminal device used by the user. Specifically, it can be a personal computer such as a desktop computer or laptop, or a mobile phone, tablet computer, multimedia device, smart wearable device, etc., but is not limited to these. A browser or web application resides in user terminal 110, which accesses computing device 200 on the Internet through the browser or web application. In the following text, the browser and web application are collectively referred to as the client, and correspondingly, computing device 200 is the server. Computing device 200 provides services to user terminal 110. It can be implemented as a server, such as an application server or web server; or it can be implemented as a desktop computer, laptop computer, processor chip, tablet computer, etc., but is not limited to these.

[0032] According to one embodiment, computing device 200 can provide content search services to user terminal 110. When a user opens the client on user terminal 110, user terminal 110 sends a search request to computing device 200, and computing device 200 typically returns one or more search results to the client. The search results and their positions are automatically generated by computing device 200. The process of automatically generating search results can be broadly divided into two stages: a recall stage and a ranking stage. In the recall stage, based on the user's interests and historical behavior, a small set of candidate search results (hundreds to thousands) of results of interest to the user is selected from a resource pool (search result vector set). In the ranking stage, more precise calculations are performed, and according to a predetermined ranking strategy, a small number of high-quality search results (e.g., a dozen or so) of the most interesting results to the user are selected from the candidate search result set.

[0033] In one embodiment, the search result location determination system 100 further includes a data storage device 120. The data storage device 120 can be a relational database such as MySQL or Access, or a non-relational database such as NoSQL; it can be a local database residing in the computing device 200, or it can be a distributed database such as HBase located in multiple geographical locations. In short, the data storage device 120 is used to store data, and this invention does not limit the specific deployment or configuration of the data storage device 120. The computing device 200 can connect to the data storage device 120 and obtain the data stored in the data storage device 120. For example, the computing device 200 can directly read the data in the data storage device 120 (when the data storage device 120 is a local database of the computing device 200), or it can access the Internet via wired or wireless means and obtain the data in the data storage device 120 through a data interface.

[0034] In embodiments of the present invention, the data storage device 120 is adapted to store vector sets of natural and mixed stream resources (a set of search result vectors composed of massive natural and mixed streams). The data storage device 120 is also adapted to store various strategies and models used in the recall and ranking phases. The data storage device is further adapted to store click-through logs; here, the data storage device is also adapted to be implemented as a log server, and the stored content may include exposures, clicks, etc.

[0035] As more and more internet users obtain information through searches, search technology continues to develop, and the display formats of search results are becoming increasingly diverse. Fluid layouts are increasingly used to display content on both web and mobile devices. In a fluid layout, the width of elements on the page adapts to the screen resolution; the size of elements may change, but the layout remains constant. The content displayed in search results in a fluid layout includes not only one or more of the following: text, images, videos, and other information in the web page cards (natural flow), but also special cards composed of structured data or aggregated information (hybrid flow, also known as "magic boxes"; in this specific implementation, "magic boxes" will be used to refer to the hybrid flow). As the number of magic boxes in search results increases, the combined effect of natural flow and magic boxes becomes increasingly important to the user's search experience.

[0036] Existing mixed ranking techniques mainly rank the Magic Box and the Organic Flow based on statistical indicators such as click-through rate (CTR) and search rate. This ranking method can lead to high CTRs due to low exposure (e.g., a Magic Box with 10 exposures and 6 clicks has a CTR of 0.6, while a Magic Box with 10,000 exposures and 5,500 clicks has a CTR of 0.55). It is also easily affected by the exposure position, resulting in higher exposure and lower CTR for higher positions, and vice versa. This leads to Magic Boxes with low exposure and high CTR ranking higher.

[0037] Therefore, in the search result ranking scheme of the present invention, a method for determining the ranking position of search results is provided. This method is used in the ranking stage of the search result position determination system and proposes a new ranking strategy. Specifically, the ranking strategy groups the magic boxes in the search results into high exposure groups and low exposure groups based on the exposure volume. Then, it ranks the results within the high exposure groups and low exposure groups according to the first-click rate. Finally, the ranking results are merged to obtain the final search results and display them to the user.

[0038] The method for determining the search result sorting position of the present invention can be executed in a computing device. Figure 2 A schematic diagram of a computing device 200 according to an embodiment of the present invention is shown. Figure 2 As shown, in a basic configuration, computing device 200 includes at least one processing unit 202 and system memory 204. According to one aspect, depending on the configuration and type of the computing device, the processing unit 202 may be implemented as a processor. System memory 204 includes, but is not limited to, volatile memory (e.g., random access memory), non-volatile memory (e.g., read-only memory), flash memory, or any combination of such memories. According to one aspect, system memory 204 includes an operating system 205.

[0039] According to one aspect, operating system 205 is, for example, suitable for controlling the operation of computing device 200. Furthermore, examples are practiced in conjunction with graphics libraries, other operating systems, or any other applications, and are not limited to any particular application or system. Figure 2 The basic configuration is illustrated by the components within the dashed lines. According to one aspect, the computing device 200 has additional features or functions. For example, according to one aspect, the computing device 200 includes additional data storage devices (removable and / or non-removable), such as disks, optical discs, or magnetic tapes. This additional storage... Figure 2 The middle part is shown by removable storage device 209 and non-removable storage device 210.

[0040] As stated above, according to one aspect, program module 203 is stored in system memory 204. According to one aspect, program module 203 may include one or more applications. The invention does not limit the type of application; for example, applications may include: email and contact applications, word processing applications, spreadsheet applications, database applications, slideshow applications, drawing or computer-aided applications, web browser applications, etc. In an embodiment of the invention, program module 203 includes one or more program instructions for executing the search result ranking position determination method 300 of the invention, these program instructions being configured to execute the search result ranking position determination method 300 of the invention.

[0041] According to one aspect, examples can be practiced on circuits including discrete electronic components, packaged or integrated electronic chips containing logic gates, circuits utilizing microprocessors, or on a single chip containing electronic components or a microprocessor. For example, it can be practiced via wherein... Figure 2 Each or many of the components shown can be implemented as an example by integrating a System-on-a-Chip (SOC) on a single integrated circuit. According to one aspect, such an SOC device may include one or more processing units, graphics units, communication units, system virtualization units, and various application functions, all integrated (or “burned in”) as a single integrated circuit onto a chip substrate. When operating via the SOC, the functions described herein can be operated via dedicated logic integrated on a single integrated circuit (chip) with other components of the computing device 200. Embodiments of the invention can also be implemented using other techniques capable of performing logical operations (e.g., AND, OR, and NOT), including but not limited to mechanical, optical, fluid, and quantum technologies. Additionally, embodiments of the invention can be implemented within a general-purpose computer or in any other circuit or system.

[0042] According to one aspect, computing device 200 may also have one or more input devices 212, such as a keyboard, mouse, pen, voice input device, touch input device, etc. It may also include output devices 214, such as a display, speaker, printer, etc. The foregoing devices are examples and other devices may also be used. Computing device 200 may include one or more communication connections 216 that allow communication with other computing devices 218. Examples of suitable communication connections 216 include, but are not limited to: RF transmitter, receiver and / or transceiver circuitry; Universal Serial Bus (USB), parallel and / or serial ports.

[0043] As used herein, the term computer-readable medium includes computer storage medium. Computer storage medium can include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information (e.g., computer-readable instructions, data structures, or program module 203). System memory 204, removable storage device 209, and non-removable storage device 210 are examples of computer storage media (i.e., memory storage). Computer storage media can include random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape, magnetic tape, disk storage or other magnetic storage devices, or any other article of manufacture that can be used to store information and is accessible by computer device 200. According to one aspect, any such computer storage medium can be part of computing device 200. Computer storage media does not include carrier waves or other transmitted data signals.

[0044] According to one aspect, the communication medium is implemented by computer-readable instructions, data structures, program modules 203, or other data in a modulated data signal (e.g., a carrier wave or other transmission mechanism), and includes any information transmission medium. According to one aspect, the term "modulated data signal" describes a signal having one or more sets of characteristics or altered in a manner that encodes information in the signal. By way of example and not limitation, the communication medium includes wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.

[0045] In an embodiment of the invention, a computing device 200 is configured to perform a search result ranking position determination method 300 according to the invention. The computing device 200 includes one or more processors and one or more readable storage media storing program instructions that, when executed by the one or more processors, cause the computing device to perform the search result ranking position determination method 300 according to the embodiment of the invention.

[0046] Figure 3 A flowchart illustrating a search result ranking position determination method 300 according to an embodiment of the present invention is shown. Method 300 is adapted to be executed in a server (the aforementioned computing device 200), and method 300 is suitable for determining the position of the mixed stream in search results in advance when offline.

[0047] like Figure 3 As shown, method 300 includes steps 310 to 340.

[0048] In step 310, statistical data of the first type of search results corresponding to the keywords are obtained. The statistical data includes exposure and first-click rate, where the first-click rate is the ratio of the first click to the exposure.

[0049] It should be noted that the first type of search results refers to a set containing one or more search results belonging to the first type, and the statistical data here refers to the statistical data of each search result in the set.

[0050] In one embodiment of the present invention, the first type of search results includes multiple mixed streams, each of which is a knowledge card composed of structured data, also known as a magic box. In the present invention, the magic box is equivalent to the mixed stream.

[0051] It's important to note that the first click refers to the action of a user clicking on one of the many search results during a single search. Therefore, the number of first clicks for a particular search result is the total number of initial clicks on that search result across multiple search actions within a certain period.

[0052] In one embodiment of the present invention, the statistical data of the first type of search results corresponding to the keywords is obtained by statistically analyzing the exposure and click data included in the exposure and click log within a preset period. The preset period is preferably the most recent 7 days, but it can also be determined by professionals based on experience. The present invention does not limit this.

[0053] It should be noted that when users browse search results, they browse from the front to the back in sequence, and will click when they see results that meet their needs. Therefore, generally speaking, the higher the position of the search result, the greater the probability that the click at this position is the first click by the user in a search result. That is, if i < j, then P(i) > P(j), where i and j represent positions, and P(i) and P(j) represent the probabilities that these positions are the first clicks in a search result. The first-click rate FCTR(i) at position i = FC(i) / S(i) = C(i)*P(i) / S(i) = CTR(i)*P(i), where FCTR(i) represents the first-click rate at position i, FC(i) represents the total number of first clicks at position i in multiple search results, C(i) represents the number of clicks at position i in multiple search results, S(i) represents the number of exposures at position i in multiple search results, CTR(i) represents the click-through rate at position i in multiple search results, and P(i) represents the probability that position i is the first click in a search result. For i < j, FCTR(i) = CTR(i)*P(i), FCTR(j) = CTR(j)*P(j), where P(i) > P(j). Therefore, the more backward the exposure position of the magic box is, the smaller the first-click rate FCTR is relative to the click-through rate CTR multiplied by the probability P of the first click, thus alleviating the problem that the exposure volume is low and the click-through rate is high when the magic box is in a backward position.

[0054] In an embodiment of the present invention, the click-through rates of the first type of search results and the second type of search results corresponding to the keyword can also be obtained, and the first type of search results can be filtered according to the click-through rates of the first type of search results and the second type of search results.

[0055] It should be noted that the click-through rate here can be statistically calculated based on the click volume and exposure volume of each search result in the first type of search results or the second type of search results obtained from the click exposure log.

[0056] In an embodiment of the present invention, filtering the first type of search results according to the click-through rates of the first type of search results and the second type of search results can be implemented as follows: calculating the average click-through rate of the second type of search results according to the click-through rate of the second type of search results; deleting the search results in the first type of search results whose click-through rates are lower than the average click-through rate of the second type of search results to obtain the filtered first type of search results. The click-through rate can reflect the degree of favor of users for search results. The higher the click-through rate, the higher the demand degree of users for this search result, and vice versa. According to the average click-through rate of the second type of search results, eliminating the search results with lower click-through rates in the first type of search results can filter the first type of search results and retain the search results with high demand degrees of users, making the displayed search results more in line with the true preferences of users.

[0057] It should be noted that the second type of search results refers to a collection that includes one or more search results belonging to the second type.

[0058] In one embodiment of the present invention, the second type of search results includes multiple natural streams, which are web page cards containing text, images and / or videos.

[0059] In one embodiment of the present invention, the recent Magic Box and natural flow statistics based on the keyword "Model A" are shown in Table 1 below:

[0060] Table 1

[0061]

[0062]

[0063] It should be noted that the organic traffic statistics here only include the total exposure and average click-through rate over seven days. Here, we only need to filter the Magic Box based on the average click-through rate of organic traffic, and there is no need to count its first click rate.

[0064] In one embodiment of the present invention, magic boxes with a click-through rate lower than the average click-through rate (7.02%) of the organic traffic under the keyword "model A" (including: dealer magic boxes, maintenance magic boxes, and quality complaint magic boxes) are deleted and do not participate in the subsequent sorting process. The list of remaining magic boxes after deletion is shown in Table 2 below:

[0065] Table 2

[0066] Magic Box Name Exposure Click-through rate First Point Rate Car Series Magic Box 356651 65.90％ 65.90％ Word-of-mouth Magic Box 64843 7.14％ 4.24％ Used Car Magic Box 17926 12.56％ 5.23％ Everyone is searching for Magic Box 14563 7.75％ 3.50％ Same-class car magic box 6714 8.45％ 3.69％ Modification Magic Box 1188 7.27％ 2.20％ Car maintenance cost magic box 623 9.28％ 3.99％ Car Purchase Discount Magic Box 254 9.40％ 2.43％ Official Subsidy Magic Box 62 9.00％ 1.66％

[0067] In step 320, the first type of search results are divided into a high-exposure group and a low-exposure group based on the exposure.

[0068] In one embodiment of the present invention, search results in the first category with an exposure value greater than or equal to a first threshold are classified into a high-exposure group, and search results in the first category with an exposure value less than the first threshold are classified into a low-exposure group. It should be noted that the first threshold mentioned here can be flexibly set according to the needs of the staff, and the present invention does not limit it. Grouping the Magic Box by exposure value can overcome the problem of high click-through rates caused by low exposure value, making the subsequent sorting results more scientific and reasonable.

[0069] In step 330, the high exposure group and the low exposure group are sorted according to the first point rate to obtain the first high sorting result and the first low sorting result.

[0070] In another embodiment of the present invention, before dividing the first category of search results into high-exposure and low-exposure groups based on exposure, fixed-position search results and / or newly added search results in the first category of search results are set to default positions according to business needs. These default positions will no longer participate in the subsequent search result sorting process. By pre-setting the positions of fixed-position search results and / or newly added search results, the problem of newly added blind boxes lacking statistical information and thus unable to participate in sorting can be overcome. It can also overcome the problem that certain special blind boxes need to be placed in fixed positions due to commercial advertising needs or other factors.

[0071] In one embodiment of the present invention, according to business requirements, the vehicle series magic box under the keyword "model A" is set as a fixed magic box, with the default position set to 1. At this time, the magic box information list is as shown in Table 3:

[0072] Table 3

[0073]

[0074] In step 340, the first high-ranking result and the first low-ranking result are merged to obtain the first ranking result of the first type of search results.

[0075] In one embodiment of the present invention, the first high-ranking result and the first low-ranking result are merged by placing all the first high-ranking results before the first low-ranking results. This merging process does not need to consider the first-click rate and the click-through rate.

[0076] In one embodiment of the present invention, the magic boxes under the keyword "model A" are grouped according to their exposure volume. Here, the first threshold is taken as 5000, and the magic box grouping list is shown in Table 4:

[0077] Table 4

[0078]

[0079] It should be noted that the grouping results in Table 4 are only related to the exposure and the first threshold, and are not related to the click-through rate or first-click rate. Even if the click-through rate or first-click rate is high but the exposure is lower than the first threshold, the Magic Box will be classified as the low exposure group.

[0080] In one embodiment of the present invention, the sorting results of the magic boxes in the high exposure group and the low exposure group are obtained according to the first point rate, as shown in Table 5:

[0081] Table 5

[0082]

[0083]

[0084] It should be noted that in Table 5, the first click rate of the Car Maintenance Costs Magic Box is 3.99%, which is obviously higher than that of the Car Maintenance Costs Magic Box and the Magic Box that everyone is searching for. However, because the exposure of the Car Maintenance Costs Magic Box is lower than the first threshold (5000), while the exposure of the Car Maintenance Costs Magic Box and the Magic Box that everyone is searching for is higher than the first threshold (5000), although the first click rate of the Car Maintenance Costs Magic Box is higher, it still ranks behind the Car Maintenance Costs Magic Box and the Magic Box that everyone is searching for.

[0085] Figure 4 A flowchart illustrating a search result compositing method 400 according to an embodiment of the present invention is shown. Method 400 is adapted to be executed in a server (the aforementioned computing device 200), which stores a first ranking result and click-through rate obtained according to the search result ranking position determination method in any of the foregoing embodiments, and method 400 is applicable to the process of online search result ranking and display.

[0086] like Figure 4 As shown, method 400 includes steps 410 to 450.

[0087] In step 410, a search request is received, which includes keywords.

[0088] In step 420, the first type of search results and the second type of search results corresponding to the keywords are obtained.

[0089] In step 430, the second type of search results are sorted according to the click-through rate to obtain the second sorting result.

[0090] It should be noted that the second type of search results here refers to the search results of the aforementioned organic stream. Organic streams are highly fluid, numerous, and characterized by their unpredictability and timeliness. To ensure the timeliness of organic streams in the search results, there is no need to sort them offline in advance. Instead, the organic streams retrieved by the index are sorted according to their click-through rate during each online search.

[0091] In step 440, the first sorting result of the first type of search results and the second sorting result of the second type of search results are merged to obtain the final sorting result.

[0092] In step 450, the first type of search results and the second type of search results are mixed according to the final sorting result to obtain the final search results.

[0093] In one embodiment of the present invention, merging the first sorting result and the second sorting result to obtain the final sorting result specifically includes: determining the position of each search result in the first sorting result according to a preset ratio, with the remaining positions being the remaining positions; and filling each search result in the second sorting result into the remaining positions to obtain the final sorting result.

[0094] In one embodiment of the present invention, the aforementioned preset ratio is preferably 1:3. Here, the ratio of the first sorting result to the second sorting result can also be changed to a new ratio according to the settings of those skilled in the art. Any sorting ratio will be included within the scope of the present invention.

[0095] In one embodiment of the present invention, the first type of search results refers to a set containing one or more search results belonging to the first type, and the statistical data here refers to the statistical data of each search result in the set. The first type of search results contains multiple hybrid streams, each hybrid stream being a knowledge card composed of structured data, also known as a magic box. In the present invention, the magic box is equivalent to the hybrid stream.

[0096] In one embodiment of the present invention, the second type of search results refers to a set containing one or more search results belonging to the second type. The second type of search results includes multiple organic streams, which are web page cards containing text, images, and / or videos.

[0097] In one embodiment of the present invention, the position results of each magic box under the keyword "model A" are calculated according to the pattern of 1 magic box and 3 natural flows, as shown in Table 5:

[0098] Table 5

[0099]

[0100]

[0101] In one embodiment of the present invention, during the online search results display process, the Magic Box prioritizes positions according to offline calculated locations, and the remaining positions are filled with natural flow. The final displayed search results are shown in Table 6.

[0102] Table 6

[0103]

[0104]

[0105] Figure 5 A flowchart illustrating a method 500 for blending search results according to yet another embodiment of the present invention is shown. Method 500 is adapted to be executed in a server (the aforementioned computing device 200).

[0106] The system receives search requests online and extracts keywords based on those requests. Keyword extraction can employ unsupervised keyword extraction methods such as TD-IDF, TextRank, and LDA, as well as supervised keyword extraction methods. This invention does not limit the methods used; any algorithm capable of extracting keywords is included within the scope of this invention.

[0107] The system retrieves relevant magic boxes from the magic box resource pool based on keywords, and relevant natural streams from the natural stream resource pool using an inverted index based on keywords, then recalls both magic boxes and natural streams. An inverted index, also known as a reverse index, inset archive, or reverse archive, is an indexing method used to store a mapping of the storage location of a word within a document or a group of documents in full-text search. It is the most commonly used data structure in document retrieval systems. Through an inverted index, a list of documents containing a word can be quickly obtained. An inverted index mainly consists of two parts: a "word dictionary" and an "inverted file." It should be noted that this invention does not limit the indexing method; any indexing method that can achieve the indexing effect is included within the scope of this invention.

[0108] Recall involves selecting a small subset of items from a massive dataset as a candidate set, which is then sent to a ranking layer for fine-tuning using a complex model. Recall algorithms can be implemented as single-embedded vector recall, multi-embedded vector recall, TDM deep tree matching recall (complex model + full database search), graph recall, etc. This invention does not limit the recall algorithm; any algorithm that can achieve recall functionality will be included within this invention.

[0109] Based on the locations of the magic boxes obtained offline in method 300, the locations of the relevant magic boxes that were recalled are determined, resulting in a magic box with mixed positions.

[0110] It should be noted that during the indexing and recall phases, magic boxes with higher relevance to the user are obtained based on the user's specific preferences. Therefore, the magic boxes obtained may not be all the magic boxes that were determined in the offline sorting. If the number of recalled magic boxes is less than the number of magic boxes obtained offline, some positions will be vacant. In this case, the subsequent magic boxes will fill the vacant positions in order of their positions to ensure the continuity of the magic box sorting.

[0111] The recalled organic streams are sorted according to their click-through rates (CTRs) to obtain a reordered organic stream. The CTR of the organic streams can be selected as the average CTR of each organic stream over the past seven days.

[0112] It should be noted that organic traffic itself is highly fluid, numerous, and characterized by its unpredictability and strong time sensitivity. To ensure the timeliness of organic traffic in search results, there is no need to sort it offline in advance. Instead, the organic traffic retrieved by the index should be sorted according to the click-through rate during each online search.

[0113] The Magic Box and the Natural Flow are arranged in a certain ratio, such as a 1:3 ratio, to obtain the final search results.

[0114] Figure 6A schematic diagram of a search result location determination device 600 according to an embodiment of the present invention is shown. Figure 6 As shown, the search result determination device 600 includes: an acquisition module 610, a classification module 620, and a sorting module 630.

[0115] The acquisition module 610 acquires statistical data of the first type of search results and the second type of search results corresponding to the keywords. The statistical data includes exposure, click-through rate and first click rate, wherein the first click rate is the ratio of the first click to the exposure.

[0116] The classification module 620 can divide the first category of search results into high-exposure groups and low-exposure groups based on exposure.

[0117] The sorting module 630 can sort the high exposure group and the low exposure group separately according to the first point rate, obtain the first high sorting result and the first low sorting result, and merge the first high sorting result and the first low sorting result to obtain the first sorting result.

[0118] According to the search result mixing method of the present invention, executed on a server, keywords are obtained based on search requests. In offline mode, based on the average click-through rate (CTR) of the natural flow, magic boxes with CTRs lower than the average CTR corresponding to the keywords are eliminated. Then, the magic boxes are grouped based on exposure volume, and sorted according to the first-click rate in both high-exposure and low-exposure groups. The position of each magic box is determined by combining the default positions of newly added and fixed-position magic boxes. During the online search result display process, based on keywords, corresponding magic boxes and natural flow are obtained through indexing and recall. The position of the magic boxes is determined according to the results obtained offline, and the natural flow is sorted according to the CTR. Finally, the magic boxes and natural flow are mixed according to a certain ratio to obtain the final sorted result for display. By filtering magic boxes using the average CTR of the natural flow, magic boxes with lower user demand are eliminated, making the magic boxes in the search results more closely aligned with user needs and arranged more scientifically and rationally. The search results are grouped based on exposure volume, and then sorted within each group according to the first-click rate. This overcomes the problem of high-click-rate search boxes with low exposure ranking higher, ensuring that search boxes that truly receive significant user attention are presented prominently in the results. Default positions are set for newly added or fixed-position search boxes, resolving issues such as new search boxes lacking statistical information being excluded from ranking and certain commercially motivated search boxes having fixed settings. Search results are then sorted in a specific ratio between search boxes and the organic search results, resulting in a more scientific and logical display and overcoming the problems of chaotic and disorganized sorting between search boxes and the organic search results.

[0119] The various techniques described herein can be implemented in combination with hardware or software, or a combination thereof. Thus, the methods and apparatus of the present invention, or certain aspects or portions thereof, can take the form of program code (i.e., instructions) embedded in a tangible medium, such as a removable hard disk, USB flash drive, floppy disk, CD-ROM, or any other machine-readable storage medium, wherein when the program is loaded into and executed by a machine such as a computer, the machine becomes an apparatus for practicing the present invention.

[0120] When the program code is executed on a programmable computer, the mobile terminal generally includes a processor, a processor-readable storage medium (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. The memory is configured to store program code; the processor is configured to execute the data storage method of the present invention according to instructions in the program code stored in the memory.

[0121] By way of example, and not limitation, readable media include readable storage media and communication media. Readable storage media stores information such as computer-readable instructions, data structures, program modules, or other data. Communication media generally embodies computer-readable instructions, data structures, program modules, or other data in the form of modulated data signals such as carrier waves or other transmission mechanisms, and includes any information delivery medium. Any combination of the above is also included within the scope of readable media.

[0122] In the specification provided herein, the algorithms and displays are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used with the examples of this invention. The required structure for constructing such systems is apparent from the above description. Furthermore, this invention is not directed to any particular programming language. It should be understood that the contents of the invention described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the invention.

[0123] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0124] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more aspects of the invention, various features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the above description of exemplary embodiments of the invention. However, this method of disclosure should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0125] Those skilled in the art will understand that modules, units, or components of the devices disclosed in the examples herein can be arranged in the devices described in this embodiment, or alternatively, can be located in one or more devices different from the devices in this example. The modules in the foregoing examples can be combined into a single module or, in addition, can be divided into multiple sub-modules.

[0126] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0127] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0128] Furthermore, some of the embodiments described herein are methods or combinations of method elements that can be implemented by a processor of a computer system or by other means of performing the functions. Therefore, a processor having the necessary instructions for implementing the methods or method elements forms means for implementing the methods or method elements. Furthermore, the elements described herein in the apparatus embodiments are examples of means for implementing the functions performed by elements for the purposes of carrying out the invention.

[0129] As used herein, unless otherwise specified, the use of ordinal numbers such as “first,” “second,” “third,” etc., to describe ordinary objects merely indicates different instances of similar objects and is not intended to imply that the objects being described must have a given order in time, space, ordering, or any other manner.

[0130] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and edibility purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative rather than restrictive, and the scope of the invention is defined by the appended claims.

Claims

1. A method for determining the sorting position of search results, suitable for execution in a computing device, the method comprising: Obtain statistical data of the first type of search results corresponding to the keywords, the statistical data including exposure volume and first click rate, wherein the first click rate is the ratio of the first click volume to the exposure volume; Based on the exposure volume, the first type of search results are divided into a high-exposure group and a low-exposure group; Based on the first point rate, the high exposure group and the low exposure group are sorted respectively to obtain the first high sorting result and the first low sorting result; The first high-ranking result and the first low-ranking result are merged to obtain the first ranking result of the first type of search results. The method of merging the first high-ranking result and the first low-ranking result is to place all the first high-ranking results before the first low-ranking results.

2. The method as described in claim 1, wherein, Before dividing the first type of search results into high-exposure and low-exposure groups based on the exposure volume, the method further includes: Obtain the click-through rate of the first and second category search results corresponding to the keywords; The first type of search results is filtered based on the click-through rate of the first type of search results and the second type of search results.

3. The method as described in claim 2, wherein, The step of filtering the first type of search results based on the click-through rate of the first type of search results and the second type of search results includes: Calculate the average click-through rate of the second type of search results based on the click-through rate of the second type of search results; Delete search results in the first category whose click-through rate is lower than the average click-through rate to obtain the filtered first category of search results.

4. The method of claim 1, wherein, Before dividing the first type of search results into high-exposure and low-exposure groups based on the exposure volume, the method further includes: Based on business needs, the fixed search results and / or newly added search results in the first type of search results are set to the default position, and the default position will no longer participate in the subsequent search result sorting process.

5. The method of claim 1, wherein, The step of dividing the first type of search results into a high-exposure group and a low-exposure group based on the exposure volume includes: The first type of search results with an exposure volume greater than or equal to the first threshold are classified into the high exposure group; The first type of search results with an exposure level less than a first threshold are classified into the low exposure group.

6. The method of claim 2, wherein, The first type of search results contains multiple mixed streams, each of which is a knowledge card composed of multiple structured data. The second type of search results includes multiple organic streams, which are web page cards containing text, images, and / or videos.

7. The method according to any one of claims 1-6, wherein the statistical data is data obtained by statistically analyzing the exposure and click data included in the exposure and click log within a preset period.

8. A search result sorting position determination device, residing in a computing device, the device comprising: The acquisition module is adapted to acquire statistical data of the first type of search results and the second type of search results corresponding to the keywords. The statistical data includes exposure, click-through rate and first click rate, wherein the first click rate is the ratio of the first click to the exposure. The classification module is adapted to divide the first type of search results into a high-exposure group and a low-exposure group based on the exposure volume; The sorting module is adapted to sort the high-exposure group and the low-exposure group separately according to the first-point rate, to obtain a first high-sorting result and a first low-sorting result. The processing module is adapted to merge the first high sorting result and the first low sorting result to obtain a first sorting result, wherein the method of merging the first high sorting result and the first low sorting result is: to place all the first high sorting results before the first low sorting results.

9. A method for sorting search results, suitable for execution in a computing device, said computing device storing a first sorting result obtained by the method for determining the sorting position of search results according to any one of claims 1-7, said method comprising: Receive a search request, the search request including keywords; Obtain the first type of search results and the second type of search results corresponding to the keywords; The second type of search results are sorted according to the click-through rate to obtain the second sorting result; The first sorting result of the first type of search results and the second sorting result of the second type of search results are merged to obtain the final sorting result. Specifically, merging the first sorting result and the second sorting result includes: determining the position of each search result in the first sorting result according to a preset ratio, with the remaining positions being the remaining positions, and filling the remaining positions with each search result in the second sorting result. Based on the final sorting result, the first type of search results and the second type of search results are mixed together to obtain the final search results.

10. The method of claim 9, wherein merging the first sorting result and the second sorting result to obtain the final sorting result includes: The positions of each search result in the first sorting result and the second sorting result are determined according to a preset ratio, and the remaining positions are the remaining positions. The search results from the second sorting result are then filled into the remaining positions to obtain the final sorting result.

11. The method of claim 10, wherein, The preset ratio is 1:

3.

12. The method of claim 10 or 11, wherein, The first type of search results contains multiple mixed streams, each of which is a knowledge card composed of multiple structured data. The second type of search results includes multiple organic streams, which are web page cards containing text, images, and / or videos.

13. A computing device, comprising: At least one processor; as well as A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor, the program instructions including instructions for performing the method as described in any one of claims 1-7, 9-12.

14. A readable storage medium storing program instructions that, when read and executed by a computing device, cause the computing device to perform the method as described in any one of claims 1-7, 9-12.

Images