Search method, apparatus and electronic device

Abstract

The present disclosure provides a search method, device and electronic equipment, the search method comprises: obtaining page content displayed on a screen, wherein the page content comprises text; dividing the page content into a plurality of analysis regions; in the process of displaying the page content, locating an analysis region where a user gazes, and determining a search candidate word corresponding to the gaze according to the text of the gazed analysis region. According to the present disclosure, the search candidate word is refined to the region where the user gazes, so that the search candidate word is more accurately matched with the user interest. In the related art, the association degree between the search based on the label and the hot word and the user browsing content is small, and it is difficult to accurately match the user interest.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of search, and in particular, to a search method, device and electronic equipment. BACKGROUND

[0002] In the related art, search methods include the following: 1) a search box is provided separately for users to search independently; 2) hotword search: hotword-related presets are made in the search box, and users can directly perform quick search when clicking the search box; 3) tagword search: for short videos and personalized content, tagwords are preset for each content, such as sports, stars and events, and several keywords are preset in the search box.

[0003] With the continuous increase of video length and content length, the search method in the related art is difficult to accurately match user interests. SUMMARY

[0004] According to an aspect of the present disclosure, a search method is provided, including: obtaining page content displayed on a screen, wherein the page content includes text; segmenting the page content into multiple analysis regions; in a process of displaying the page content, locating an analysis region gazed by a user, and determining a search candidate corresponding to the gaze according to text of the gazed analysis region.

[0005] According to another aspect of the present disclosure, a search device is provided, including: an obtaining module configured to obtain page content displayed on a screen, wherein the page content includes text; a segmenting module configured to segment the page content into multiple analysis regions; a locating module configured to locate an analysis region gazed by a user in a process of displaying the page content; and a determining module configured to determine a search candidate corresponding to the gaze according to text of the gazed analysis region.

[0006] According to another aspect of the present disclosure, an electronic equipment is provided, including: a processor; and a memory storing a program, wherein the program includes instructions which, when executed by the processor, cause the processor to perform the method of the present disclosure.

[0007] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the method of the present disclosure.

[0008] One or more technical solutions provided in the embodiments of the present application acquire page content displayed on a screen, divide the page content into a plurality of analysis regions, locate an analysis region of a user's gaze in the process of displaying the page content, determine a search candidate word corresponding to the gaze according to text of the analysis region of the gaze, and make the search candidate word more accurately match the user's interest. In the related art, the association degree between the search based on a label and a hot word and the user's browsing content is small, and it is difficult to accurately match the user's interest. BRIEF DESCRIPTION OF DRAWINGS

[0009] In the following description of the exemplary embodiments in conjunction with the accompanying drawings, more details, features and advantages of the present disclosure are disclosed, in which:

[0010] Figure 1 A flowchart of a search method according to an exemplary embodiment of the present disclosure is shown;

[0011] Figure 2 A flowchart of dividing page content into a plurality of analysis regions according to an exemplary embodiment of the present disclosure is shown;

[0012] Figure 3 A schematic diagram of an analysis region according to an exemplary embodiment of the present disclosure is shown;

[0013] Figure 4 A schematic diagram of an analysis region according to an exemplary embodiment of the present disclosure is shown;

[0014] Figure 5 A flowchart of locating an analysis region of a user's gaze according to an exemplary embodiment of the present disclosure is shown;

[0015] Figure 6 A schematic diagram of the offset of the pupil position according to an exemplary embodiment of the present disclosure is shown;

[0016] Figure 7 A schematic diagram of the change of the upper face and the lower face according to an exemplary embodiment of the present disclosure is shown;

[0017] Figure 8 A flowchart of a search method according to an exemplary embodiment of the present disclosure is shown;

[0018] Figure 9 A flowchart of a search method according to an exemplary embodiment of the present disclosure is shown;

[0019] Figure 10 A schematic block diagram of a search device according to an exemplary embodiment of the present disclosure is shown;

[0020] Figure 11 A structural block diagram of an exemplary electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0021] Embodiments of the present disclosure will be described in more detail with reference to the drawings. While certain embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be embodied in various forms and should not be interpreted in a limited sense as set forth in the embodiments set forth herein. Rather, the embodiments are provided to more completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for illustrative purposes and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be understood that each step described in the method embodiments of the present disclosure can be performed in different order and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0023] The term "comprising" and variations thereof as used herein are used in the inclusive sense and mean "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The term "some embodiments" means "at least some embodiments". Related definitions are given throughout the description. It should be noted that the concepts mentioned in the present disclosure are merely illustrative and not restrictive, and those skilled in the art should understand that "one", "multiple" modification is illustrative and not restrictive, and unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0024] It should be noted that the "one", "multiple" modification mentioned in the present disclosure is illustrative and not restrictive, and those skilled in the art should understand that unless otherwise explicitly stated in the context, it should be understood as "one or more".

[0025] The names of the messages or information exchanged between the devices in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0026] The solutions of the present disclosure are described below with reference to the accompanying drawings. The embodiments of the present disclosure can be run on any device with computing and storage capabilities, such as mobile phones, tablets, PCs (Personal Computers), notebooks, servers, etc.; the functions in the embodiments of the present disclosure can also be implemented by logical nodes running on two or more devices. The device can act as a client or a server.

[0027] Figure 1 A flowchart of a search method according to an exemplary embodiment of the present disclosure is shown, as shown in Figure 1 The search method includes steps S101 to S103.

[0028] In step S101, the page content displayed on the screen is acquired, wherein the page content comprises text.

[0029] In this embodiment, the page content displayed on the screen is the content displayed on the screen.

[0030] For a page with a long length, the screen displays part of the page content, and the content displayed on the screen can be updated by sliding or turning the page, for example, to display the content after the current content. For a page with a short length, the screen can display the whole content of the page.

[0031] In step S102, the page content is divided into a plurality of analysis regions.

[0032] As an implementation, at least part of the analysis regions are obtained by vertical division, and at least part of the analysis regions are obtained by vertical division and horizontal division.

[0033] In step S103, in the process of displaying the page content, the analysis region where the user's gaze is located is located, and the search candidate corresponding to the gaze is determined according to the text of the analysis region where the gaze is located.

[0034] The gaze refers to the phenomenon that the gaze stays at a position for a period of time. In this embodiment, a time threshold for continuous gaze can be set to determine whether the gaze behavior occurs.

[0035] As an implementation, the search candidate corresponding to the gaze is determined according to the text of the analysis region where the gaze is located, comprising: extracting the keywords of the text of the analysis region where the gaze is located; determining the hot words in the extracted keywords, and taking the determined hot words as the search candidate corresponding to the gaze.

[0036] In some embodiments, in response to any time the search candidate corresponding to the gaze is determined, the determined search candidate is placed and displayed in the search input box.

[0037] In some embodiments, in response to detecting the user input for initiating the search, the search candidate corresponding to the last gaze before the user input is acquired, and the acquired search candidate is placed in the search input box and displayed.

[0038] The technical solution provided in this embodiment acquires the page content displayed on the screen, divides the page content into a plurality of analysis regions, locates the analysis region where the user's gaze is located in the process of displaying the page content, and determines the search candidate corresponding to the gaze according to the text of the analysis region where the gaze is located. The search candidate is refined to the region where the user's gaze is located, so that the search candidate is more accurately matched with the user's interest.

[0039] A possible implementation of the step S102 of segmenting the page content into a plurality of analysis regions is described as follows.

[0040] Figure 2 A flowchart of segmenting the page content into a plurality of analysis regions according to an example embodiment of the present disclosure is shown in FIG. 2. Figure 2 As shown, the flowchart includes steps S201 to S204.

[0041] In step S201, a text passage contained in the page content is determined.

[0042] For at least one text passage, step S202 is performed.

[0043] In step S202, a region where the text passage is located is segmented into a plurality of vertical regions, wherein each vertical region includes one or more lines of text.

[0044] For at least one vertical region, steps S203 and S204 are performed.

[0045] In step S203, a sentence delimiter of the text in the vertical region is determined.

[0046] In step S204, the vertical region is segmented into at least two horizontal regions with the position of the sentence delimiter in the vertical region as a horizontal segmentation position.

[0047] In this embodiment, the horizontal segmentation based on the sentence delimiter can avoid cutting a complete word. Exemplarily, the sentence delimiter can include a comma, a period, an exclamation mark, a question mark, an ellipsis, etc., but is not limited thereto.

[0048] Each line of text in the vertical region can include one or more sentence delimiters, or can have no sentence delimiter. As an implementation, in step S204, for at least one vertical region, with the position of the sentence delimiter in the vertical region as a horizontal segmentation position, includes: for each line of text in the vertical region, in a case where the line of text contains no sentence delimiter, taking the position of the beginning of the line of text as a horizontal segmentation position; in a case where the line of text includes a plurality of sentence delimiters, taking the position of the sentence delimiter that makes each text segment obtained by segmentation greater than a preset number of words as a horizontal segmentation position. By limiting each text segment obtained by segmentation to be greater than the preset number of words, in a case where the line of text includes a plurality of sentence delimiters, the horizontal region obtained by segmentation is prevented from being too small, and thus the analysis region for locating the user's gaze is prevented from being inaccurate.

[0049] Based on steps S202 to S204, a plurality of analysis regions corresponding to the text passage are segmented. The plurality of analysis regions can include the vertical regions segmented in step S202, and the horizontal regions segmented in steps S203 and S204.

[0050] When dividing page content into multiple analysis regions, the size of each region must be neither too large nor too small. If the analysis region is too large, it is impossible to accurately locate the user's visual area; if the analysis region is too small, the user's visual area judgment will exceed the range of one analysis region. Since the user's distance from the screen varies, the area covered at the same visual angle may also differ. Therefore, in one possible implementation, during the display of page content, the distance between the user's face and the screen is determined, and the page content is divided into multiple analysis regions based on this distance.

[0051] For example, for Figure 2 The segmentation method shown pre-sets a correspondence between segmentation parameters and distances, configured so that the larger the distance, the larger the area of ​​the segmented analysis region. Segmentation parameters corresponding to the distance are determined based on the distance between the face and the screen and the pre-set correspondence between segmentation parameters and distances. These segmentation parameters include at least one of the following: the number of text lines included in the vertical region and the number of horizontal regions segmented from the vertical region. This adapts the distance to the size of the analysis region. Generally, the smaller the distance between the face and the screen, the smaller the analysis region; the larger the distance, the larger the analysis region. For example, when the distance between the face and the screen exceeds a preset threshold, the number of horizontal regions segmented from the vertical region is 1, meaning no horizontal segmentation is performed. Furthermore, the number of text lines included in the vertical region can be increased to enlarge the segmented region.

[0052] For example, the distance can be divided into three intervals from farthest to closest: interval A, interval B, and interval C. The corresponding relationships are set as follows: Interval A: the vertical region contains 5 lines of text, and the number of horizontal regions it is divided into is 1; Interval B: the vertical region contains 3 lines of text, and the number of horizontal regions it is divided into is 2; Interval C: the vertical region contains 2 lines of text, and the number of horizontal regions it is divided into is 3. It should be understood that this embodiment is not limited in this respect; any implementation that results in a smaller analysis area for a smaller distance between the face and the screen, or a larger analysis area for a larger distance between the face and the screen, is feasible and will not be enumerated here.

[0053] Figure 3 A schematic diagram of the analysis region according to an exemplary embodiment of the present disclosure is shown, such as Figure 3 As shown, page content 100 is displayed on the screen, which includes text paragraph 110. The text paragraph 110 is vertically divided into vertical regions 111, 112, 113, and 114, based on three lines of text vertically. This is done according to sentence separators (…). Figure 3The diagram shows how the sentence separator 1113 in vertical region 111 horizontally divides vertical regions 111, 112, 113, and 114. For example... Figure 3 As shown, by connecting the position of the sentence separator in the vertical region 111 from top to bottom, the vertical region 111 is divided into horizontal regions 1111 and 1112. Similarly, the vertical region 112 is divided into horizontal regions 1121 and 1122, the vertical region 113 is divided into horizontal regions 1131 and 1132, and the vertical region 114 is divided into horizontal regions 1141 and 1142. The resulting horizontal regions are used as the analysis regions corresponding to the text paragraph 110. When dividing the vertical region 112 into horizontal regions 1121 and 1122, the vertical region 112 includes two sentence separators, namely sentence separator 1123 and sentence separator 1124. Since the text segment obtained by sentence separator 1123 (the rightmost part of the row in the figure) is less than the preset number of characters, sentence separator 1124 is selected as the horizontal division position of the text line from sentence separator 1123 and sentence separator 1124.

[0054] refer to Figure 3 As shown, for example, when the analysis area of ​​the user's gaze is a horizontal region 1111, the search candidate words corresponding to the gaze are determined based on the text in the horizontal region 1111.

[0055] In some implementations, the page content displayed on the screen may also include images, titles, etc. As one approach, images and titles are treated as separate analysis areas. Figure 4 A schematic diagram of the analysis region according to an exemplary embodiment of the present disclosure is shown, such as Figure 4 As shown, page content 200 is displayed on the screen. Analyzing page content 200, it is determined that from top to bottom, page content 200 consists of the title area 210, the first text paragraph area 220, the image area 230, and the second text paragraph area 230. The title area 210 and the image area 230 are analyzed separately. The first text paragraph area 220 and the second text paragraph area 230 can be further divided vertically, and these vertical areas can be further divided horizontally. See the diagram for the division of the text paragraph areas. Figure 3 As shown.

[0056] The following describes possible implementations of the analysis area for locating user gaze.

[0057] Figure 5 A flowchart illustrating the analysis region for locating user gaze according to an exemplary embodiment of this disclosure is shown, such as... Figure 5 The diagram includes steps S501 to S505.

[0058] Step S501, determine the direction of the user's line of sight relative to the face plane.

[0059] As an implementation, the direction of the user's line of sight relative to the face plane is determined, including: identifying the position of the pupil in the eye according to the eye image of the user; determining the direction of the user's line of sight relative to the face plane according to the offset of the identified pupil position relative to the calibrated pupil position. Since the lower eyelid does not change, the positions of the calibrated pupil and the coordinate system in the eye are determined based on the lower eyelid.

[0060] For example, the calibrated pupil position refers to the position of the pupil in the eye when looking straight horizontally. As shown in the figure, Figure 6 The eye includes the upper eyelid 310, the lower eyelid 320, the calibrated pupil position 330, the identified pupil position 340, and the offset of the identified pupil position 340 relative to the calibrated pupil position 330 is represented as 350. Based on the coordinates of the calibrated pupil position 330 in the coordinate system (XY) and the coordinates of the identified pupil position 340 in the coordinate system (XY), the offset of the identified pupil position 340 relative to the calibrated pupil position 330 can be determined, and the direction of the user's line of sight relative to the face plane can be determined.

[0061] As an implementation, the user is guided to look straight horizontally at the screen, that is, the direction of the user's line of sight is basically the normal direction of the screen plane, at this time the face image is captured, the captured face image is identified, and the pupil position is calibrated.

[0062] Step S502, determine the face contour from the face image of the user, and determine the first angle of the face plane relative to the reference coordinate system according to the change of the determined face contour relative to the calibrated face contour. The calibrated face contour is the face contour when the face plane of the user is at a preset angle relative to the reference coordinate system.

[0063] As an implementation, the face contour is determined from the face image of the user, and the first angle of the face plane relative to the reference coordinate system is determined according to the change of the determined face contour relative to the calibrated face contour, including: determining the upper face and the lower face from the face image of the user and the preset face partition line; determining the areas of the upper face and the lower face of the face contour; and determining the first angle of the face plane relative to the reference coordinate system according to the change of the areas of the upper face and the lower face relative to the areas of the calibrated upper face and lower face. The areas of the calibrated upper face and lower face are the areas of the upper face and lower face when the face plane is at a preset angle relative to the reference coordinate system.

[0064] For example, the calibrated upper face and lower face refers to the upper face and lower face when the face plane is substantially parallel to the screen. As shown in the figure, Figure 7As shown, the left side shows the calibrated upper and lower face parts, the right side shows the determined upper and lower face parts, and the middle shows the comparison of the two. Figure 7 As shown, the calibrated upper and lower face parts are shown when the face plane is substantially parallel to the screen, and the determined upper and lower face parts are shown when the user lowers his head, for example. The determined upper face part has a larger area relative to the calibrated upper face part, and the determined lower face part has a smaller area relative to the calibrated lower face part. Based on the degree of area change, the angle of the face plane relative to the reference coordinate system can be determined.

[0065] For example, a mapping relationship between the angle of the face plane relative to the reference coordinate system and the percentage of change is preset. During the process of displaying the page content, the areas of the upper and lower face parts of the user's face contour are determined in real time, the percentage of change of the areas of the upper and lower face parts relative to the areas of the calibrated upper and lower face parts is determined, and the angle of the face plane relative to the reference coordinate system is determined according to the percentage of area change and the preset mapping relationship.

[0066] Step S503: The second angle of the screen relative to the reference coordinate system is determined.

[0067] For example, a handheld electronic device (such as a smart phone or a tablet computer) can be used, and the angle of the handheld electronic device relative to the reference coordinate system can be determined based on the gravity sensor and the angle sensor of the electronic device.

[0068] Step S504: The position where the user's line of sight intersects the screen is determined according to the direction of the user's line of sight relative to the face plane, the first angle of the face plane relative to the reference coordinate system, and the second angle of the screen relative to the reference coordinate system.

[0069] Step S505: The analysis region that the user gazes at is located according to the position where the user's line of sight intersects the screen and the position of the analysis region on the screen.

[0070] According to the direction of the user's line of sight relative to the face plane, the first angle of the face plane relative to the reference coordinate system, and the second angle of the screen relative to the reference coordinate system, the position where the line of sight intersects the screen can be determined, and further combined with the position of the analysis region on the screen, the analysis region that the user gazes at can be located.

[0071] Generally, when using a screen device such as a mobile phone or a display, the screen device is placed in front of the face. The face plane can be substantially parallel to the screen or have a certain angle with the screen. Taking the world coordinate system as the reference coordinate system, the angle of the screen relative to the reference coordinate system, the angle of the face plane relative to the reference coordinate system, and the direction of the user's line of sight relative to the face plane can be determined. Since the screen device is placed in front of the face, the relative position relationship between the screen and the face is determined, so that the user's line of sight is extended to the screen, and the position where the line of sight intersects the screen can be determined.

[0072] In some embodiments, three-dimensional face information can be collected, which includes distance information of the face from the collection device, and thus the positional relationship between the face and the collection device can be determined, and the positional relationship between the face and the screen can be determined based on the positional relationship between the collection device and the screen.

[0073] The following describes the analysis area of the user's gaze position with a smart phone as an example.

[0074] When the user holds the phone to browse, the phone is considered to be in a relatively static state for a period of time. Then, with the help of the gravity sensor and the angle sensor in the phone, the relative angle of the current phone and the horizontal plane can be calculated.

[0075] Through the front camera, the viewing angle of the user's eyeball and the phone screen is captured. First, the user is reminded to take a photo to collect data when the eyeball and the phone are on the same horizontal line in the vertical angle. In this way, the positional relationship between the pupil and the upper and lower eyelids in the horizontal direction can be located and recorded, and the central axis can also be located in this way.

[0076] When the user looks up or looks down at the phone, the position of the lower eyelid is taken for calculation because the position of the upper eyelid may shift. The pupil is identified through the front camera to find the displacement relationship between the pupil and the lower eyelid and the offset distance from the central axis, and thus the offset angle of the eye.

[0077] Because the user's head is tilted, through the front camera, not only eye information but also head information can be collected. First, the area size of the upper face and the lower face (such as taking the ear as the collection dividing line) in the horizontal state can be collected as a reference. When the user tilts the head, the area size of the upper face and the lower face is collected again. Because the area of the upper face will become larger and the area of the lower face will become smaller according to the principle of near large and far small when the user looks down, the tilt angle of the user's head relative to the vertical can be obtained by calculation.

[0078] Based on the tilt angle of the head, the angle of the eyeball, and the angle of the device, the approximate range of the user's gaze can be calculated.

[0079] Figure 8 A flowchart of a search method according to an example implementation of the present disclosure is shown, as shown in Figure 8 The method includes steps S801 to S806.

[0080] Step S801, display page content and a search input box on the screen.

[0081] The page content includes text.

[0082] For example, when browsing a pure text content, only a part of the complete content, also called a browsing segment, can be displayed on the screen due to the length of the content. The keyword that the user wants to search for can be different in different browsing segments.

[0083] In step S802, the page content displayed on the screen is obtained.

[0084] In step S803, the distance between the face and the screen is determined in real time during the display of the page content, and the page content is divided into a plurality of analysis regions according to the distance.

[0085] In step S804, the analysis region where the user's gaze is located is determined in real time during the display of the page content, and the search candidate word corresponding to the gaze is determined according to the text of the analysis region where the gaze is located.

[0086] In step S805, the search candidate word is placed in the search input box and displayed in response to the determination of the search candidate word.

[0087] In step S806, the search submission operation of the user is detected, and the search is performed with the search candidate word in response to the detection of the search submission operation of the user.

[0088] During the display of the page content, the page content after the user operation such as sliding or page turning is displayed on the screen in response to the user operation. In response to the update of the page content displayed on the screen, steps S802 to S804 are performed.

[0089] As an implementation, in step S803, the page content is divided into a plurality of analysis regions, including: determining a text paragraph contained in the page content; for at least one text paragraph, dividing the region where the text paragraph is located into a plurality of vertical regions, wherein each vertical region includes one or more lines of text; for at least one vertical region, determining a sentence delimiter of the text in the vertical region; dividing the vertical region into at least two horizontal regions with the position of the sentence delimiter in the vertical region as a horizontal division position, to obtain a plurality of analysis regions corresponding to the text paragraph. Wherein, at least one of the number of lines of text included in the vertical region and the number of horizontal regions divided by the vertical region is determined according to the distance between the face and the screen.

[0090] As an implementation, in the step S804, the analysis region of the user's gaze is located by: determining a direction of the user's line of sight relative to the face plane; determining a face contour from the face image of the user, and determining a first angle of the face plane relative to the reference coordinate system according to a change of the determined face contour relative to a calibrated face contour; determining a second angle of the screen relative to the reference coordinate system; and locating the analysis region of the user's gaze according to the direction of the user's line of sight relative to the face plane, the first angle of the face plane relative to the reference coordinate system, the second angle of the screen relative to the reference coordinate system, and a position of the analysis region on the screen.

[0091] Optionally, the direction of the user's line of sight relative to the face plane is determined by: identifying a position of the pupil in the eye according to the eye image of the user; and determining the direction of the user's line of sight relative to the face plane according to a deviation of the identified pupil position relative to a calibrated pupil position.

[0092] Optionally, the face contour is determined from the face image of the user, and the first angle of the face plane relative to the reference coordinate system is determined according to a change of the determined face contour relative to a calibrated face contour, by: determining an upper face and a lower face according to the face image of the user and a preset face segmentation line; determining areas of the upper face and the lower face; and determining the first angle of the face plane relative to the reference coordinate system according to a change of the determined areas of the upper face and the lower face relative to areas of a calibrated upper face and a calibrated lower face.

[0093] Figure 9 A flowchart of a search method according to an example implementation of the present disclosure is shown in FIG. 9, which includes steps S901 to S908. Figure 9

[0094] In step S901, page content is displayed on a screen.

[0095] In step S902, the page content displayed on the screen is acquired.

[0096] The page content includes text. The text can include direct text, or text on a picture, which can be recognized by image recognition or the like.

[0097] In step S903, during the display of the page content, a distance between a face and the screen is determined in real time, and the page content is segmented into a plurality of analysis regions according to the distance.

[0098] In step S904, during the display of the page content, an analysis region of the user's gaze is located in real time, and a search candidate word corresponding to the gaze is determined according to text of the analysis region of the gaze.

[0099] ​Step S905, during the process of recording the display page content, at least part of the gaze corresponding search candidate word is obtained to obtain the time sequence of the search candidate word.

[0100] Step S906, during the process of displaying the page content, the user input for initiating the search is detected.

[0101] Step S907, in response to detecting the user input for initiating the search, the search candidate word corresponding to the last gaze before the user input is obtained from the recorded time sequence of the search candidate word, and the obtained search candidate word is placed in the search input box and displayed.

[0102] In order to avoid possible errors and give the user the most efficient search, a quick search entry of other words can be given on the search result page, and the user can search one-key without additional input.

[0103] Step S908, detecting the search submission operation of the user, and in response to detecting the search submission operation of the user, searching with the search candidate word in the search input box.

[0104] During the process of displaying the page content, in response to the user operation such as sliding and page turning, the page content after the user operation is displayed on the screen. In response to the update of the page content displayed on the screen, steps S902 to S904 are performed.

[0105] Figure 10 A schematic block diagram of a search device according to an example embodiment of the present disclosure is shown, as shown in Figure 10 The search device includes:

[0106] The acquisition module 1010 is configured to acquire the page content displayed on the screen, wherein the page content includes text.

[0107] The segmentation module 1020 is connected with the acquisition module 1010 and is configured to segment the page content into a plurality of analysis regions.

[0108] The positioning module 1030 is connected with the segmentation module 1020 and is configured to position the analysis region gazed by the user during the process of displaying the page content.

[0109] The determination module 1040 is connected with the positioning module 1030 and is configured to determine the search candidate word corresponding to the gaze according to the text of the gazed analysis region.

[0110] As an implementation form, the segmentation module 1020 is specifically configured to: determine a text paragraph contained in the page content; for at least one text paragraph, segment a region where the text paragraph is located into a plurality of vertical regions, wherein each vertical region includes one or more lines of text; for at least one vertical region, determine a sentence delimiter of text in the vertical region; and segment the vertical region into at least two horizontal regions according to the sentence delimiter in the vertical region, to obtain a plurality of analysis regions corresponding to the text paragraph.

[0111] As an implementation form, the distance determination module is further configured to determine a distance between the face and the screen during display of the page content. The segmentation module 1020 is further configured to determine at least one of a number of lines of text included in the vertical region, and a number of horizontal regions into which the vertical region is segmented, according to the distance between the face and the screen.

[0112] As an implementation form, the positioning module 1030 is specifically configured to: determine a direction of a user's line of sight relative to a face plane; determine a face contour according to the face image of the user, and determine a first angle of the face plane relative to a reference coordinate system according to a change of the determined face contour relative to a calibrated face contour; determine a second angle of the screen relative to the reference coordinate system; and position an analysis region fixated by the user according to the direction of the user's line of sight relative to the face plane, the first angle of the face plane relative to the reference coordinate system, the second angle of the screen relative to the reference coordinate system, and a position of the analysis region on the screen.

[0113] As an implementation form, the positioning module 1030 is specifically configured to: identify a position of a pupil in an eye according to an eye image of the user; and determine the direction of the user's line of sight relative to the face plane according to a shift of the identified position of the pupil relative to a calibrated position of the pupil.

[0114] As an implementation form, the positioning module 1030 is specifically configured to: determine an upper face and a lower face according to the face image of the user and a preset face segmentation line; determine areas of the upper face and the lower face; and determine the first angle of the face plane relative to the reference coordinate system according to a change of the determined areas of the upper face and the lower face relative to areas of a calibrated upper face and lower face.

[0115] In some embodiments, the determination module 1040 is specifically configured to: extract a keyword of text of the fixated analysis region; and determine a hot word in the extracted keyword, and take the hot word as a search candidate word corresponding to the fixation.

[0116] In some embodiments, the searching apparatus further comprises a searching module configured to: in response to any one of the determined search candidate words corresponding to the gaze, place and display the determined search candidate word in the search input box; or in response to detecting a user input for initiating a search, acquire the search candidate word corresponding to the last gaze before the user input, and place and display the acquired search candidate word in the search input box.

[0117] An electronic device is also provided in an example embodiment of the present disclosure, comprising: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program capable of being executed by the at least one processor, and the computer program, when executed by the at least one processor, is configured to cause the electronic device to perform the method according to an embodiment of the present disclosure.

[0118] A non-transitory computer-readable storage medium storing a computer program is also provided in an example embodiment of the present disclosure, and the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to an embodiment of the present disclosure.

[0119] A computer program product is also provided in an example embodiment of the present disclosure, comprising a computer program, and the computer program, when executed by a processor of a computer, is configured to cause the computer to perform the method according to an embodiment of the present disclosure.

[0120] Reference Figure 11 A block diagram of an electronic device 1100, which can be used as the server or the client of the present disclosure, will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent a wide variety of digital electronic computing devices, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computing devices. The electronic device can also represent a variety of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections, and their functions, as well as their relationships to one another, are merely examples and are not intended to limit the implementations described and / or claimed in this document to the examples described herein.

[0121] As Figure 11As shown, the electronic device 1100 includes a computing unit 1101 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded into a random access memory (RAM) 1103 from a storage unit 1108. Various programs and data required for the operation of the device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0122] A plurality of components in the electronic device 1100 are connected to the I / O interface 1105, including an input unit 1106, an output unit 1107, a storage unit 1108, and a communication unit 1109. The input unit 1106 can be any type of device that can input information to the electronic device 1100, and can receive inputted digital or character information, and generate key signal inputs related to user settings and / or function controls of the electronic device. The output unit 1107 can be any type of device that can present information, and can include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1108 can include, but is not limited to, a magnetic disk, an optical disk. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0123] The computing unit 1101 can be various general and / or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs various methods and processes described above. For example, in some embodiments, the search method can be implemented as a computer software program that is tangibly embodied in a machine-readable medium, such as the storage unit 1108. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. In some embodiments, the computing unit 1101 can be configured to perform the search method by any other appropriate means, such as by means of firmware.

[0124] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be embodied entirely on a machine, partially on a machine, fully on a machine, partially on a machine and partially on a remote machine, or fully on a remote machine or server.

[0125] In the context of the present disclosure, a machine-readable medium can be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of a processor, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0126] As used in the present disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that can be used to provide machine instructions and / or data to a programmable processor.

[0127] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0128] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0129] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. A search method, characterized in that, include: Obtain the page content displayed on the screen, wherein the page content includes text; Determine the text paragraphs contained in the page content; For at least one text paragraph, the area containing the text paragraph is divided into multiple vertical regions, where each vertical region includes one or more lines of text; For at least one vertical region, determine the sentence separators of the text within that vertical region; Using the position of the sentence separator within the vertical region as the horizontal segmentation position, the vertical region is divided into at least two horizontal regions, resulting in multiple analysis regions corresponding to the text paragraph. During the display of the page content, the analysis area where the user is looking is located, and search candidate words corresponding to the look are determined based on the text in the analysis area of ​​the look.

2. The search method as described in claim 1, characterized in that, For at least one vertical region, the horizontal dividing position is determined by the location of the sentence separator within that vertical region, including: For each line of text within this vertical region, If the text in this line does not contain sentence separators, the horizontal dividing position is the position at the beginning of the text in this line; When the text in a line contains multiple sentence separators, the horizontal segmentation position is the position of the sentence separator that makes each resulting text segment larger than a preset number of characters.

3. The search method as described in any one of claims 1-2, characterized in that, Also includes: During the display of the page content, the distance between the user's face and the screen is determined; Based on the distance between the face and the screen and the preset correspondence between the segmentation parameters and the distance, the segmentation parameters corresponding to the distance are determined. The segmentation parameters include at least one of the number of text lines included in the vertical region and the number of horizontal regions divided by the vertical region. The correspondence is configured such that the larger the area of ​​the analysis region obtained by the segmentation is, the farther the distance. Furthermore, the process of dividing the page content into multiple analysis regions includes: dividing the page content into multiple analysis regions according to the segmentation parameters.

4. The search method as described in any one of claims 1-2, characterized in that, The analysis area for locating user gaze includes: Determine the direction of the user's gaze relative to the facial plane; Determine facial contours based on the user's facial image; Based on the change of the facial contour relative to the calibrated facial contour, a first angle of the facial plane relative to the reference coordinate system is determined, wherein the calibrated facial contour is the facial contour when the user's facial plane is at a preset angle relative to the reference coordinate system. Determine a second angle of the screen relative to the reference coordinate system; The position where the user's line of sight intersects with the screen is determined based on the direction of the user's line of sight relative to the facial plane, the first angle of the facial plane relative to the reference coordinate system, and the second angle of the screen relative to the reference coordinate system. The analysis area that the user is looking at is located based on the position where the user's line of sight intersects with the screen and the position of the analysis area on the screen.

5. The search method as described in claim 4, characterized in that, Determine the direction of the user's gaze relative to the facial plane, including: Identify the position of the pupil in the eye based on the user's eye image; The direction of the user's gaze relative to the facial plane is determined based on the offset of the identified pupil position relative to the calibrated pupil position. Furthermore, determining the first angle of the facial plane relative to the reference coordinate system based on the change of the facial contour relative to the calibrated facial contour includes: Based on the user's facial image and preset facial segmentation lines, determine the upper and lower facial contours; Determine the areas of the upper face and the lower face; Based on the change in the area of ​​the upper face and the lower face relative to the calibrated area of ​​the upper face and the lower face, a first angle of the facial plane relative to the reference coordinate system is determined, wherein the calibrated area of ​​the upper face and the lower face is the area of ​​the upper face and the lower face when the user's facial plane is at the preset angle relative to the reference coordinate system.

6. The search method as described in any one of claims 1-2, characterized in that, Also includes: In response to any determination of a search candidate word corresponding to the gaze, the determined search candidate word is placed into and displayed in the search input box; or In response to detecting user input for initiating a search, the system obtains the search candidate words corresponding to the last gaze before the user input, places the obtained search candidate words into the search input box, and displays them.

7. A search device, characterized in that, include: The acquisition module is used to acquire the page content displayed on the screen, wherein the page content includes text; A segmentation module is used to determine the text paragraphs contained in the page content; For at least one text paragraph, the area containing the text paragraph is divided into multiple vertical regions, where each vertical region includes one or more lines of text; For at least one vertical region, determine the sentence separators of the text within that vertical region; Using the position of the sentence separator within the vertical region as the horizontal segmentation position, the vertical region is divided into at least two horizontal regions, resulting in multiple analysis regions corresponding to the text paragraph. The positioning module is used to locate the analysis area that the user is looking at during the display of the page content; The determination module is used to determine the search candidate words corresponding to the gaze based on the text in the analysis area of ​​the gaze.

8. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

Images