A method and device for sending a request and a device for sending a request

Abstract

The embodiment of the present application discloses a request sending method, device and device for sending a request. The embodiment of the method includes: obtaining the user's eye movement information during the user input process; inputting the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area; and determining whether to send a request to the server based on the position of the target observation area. This implementation can effectively reduce the sending of useless pre-fetch requests, thereby saving computing resources.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of computer technology, and in particular to a method and device for sending a request, and a device for sending a request. Background Art

[0002] When a user inputs content through an input method client, it is usually necessary to send some requests to the server so as to provide better functions for the input method client with the help of the powerful computing resources of the server. For example, during the user input process, a request can be sent to the server in real time based on the user's input string to obtain candidate items related to the input string.

[0003] In the prior art, due to different input preferences of users (such as word-by-word input, word-by-word input, sentence-by-sentence input, etc.) and different input methods (such as full spelling input, abbreviated spelling input, last word abbreviated spelling input, etc.), it is difficult to determine the end of the input string during the user input process. In order to obtain the results of the server in time, a request is usually sent to the server after each character is input by the user. However, the results obtained by this request sending method are mostly useless results, resulting in too many useless requests being sent, which greatly consumes computing resources. Summary of the invention

[0004] The embodiments of the present application propose a request sending method, device and device for sending requests to solve the technical problem in the prior art of wasting computing resources due to sending too many useless requests.

[0005] In a first aspect, an embodiment of the present application provides a request sending method, the method comprising: obtaining the user's eye movement information during the user input process; inputting the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area; and determining whether to send a request to a server based on the position of the target observation area.

[0006] In some embodiments, the observation area prediction model is trained through the following steps: obtaining the user's historical eye movement information and historical operation areas during the historical input process; establishing a correspondence between the historical eye movement information and the historical operation areas; based on the correspondence, using a machine learning method to train a pre-constructed deep neural network to obtain an observation area prediction model.

[0007] In some embodiments, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in a candidate area in an input method interface, sending a candidate acquisition request to the server, wherein the candidate acquisition request includes the user's current input information.

[0008] In some embodiments, determining whether to send a request to a server based on the location of the target observation area includes: obtaining a first probability that the target observation area is located in a candidate area in an input method interface, and determining whether the target observation area is located in the candidate area based on the first probability; in a case where it is determined that the target observation area is located in the candidate area, determining a target delay time based on the first probability and a preset delay time for sending a request; when the target delay time meets a preset condition, sending a candidate acquisition request to the server, the candidate acquisition request including the user's current input information.

[0009] In some embodiments, determining whether to send a request to the server based on the position of the target observation area also includes: determining not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0010] In some embodiments, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in the keyboard area in the input method interface, sending a candidate prefetch request to the server, the candidate prefetch request including a predicted input string, the predicted input string being generated based on the user's current input information and the target key observed by the user.

[0011] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, obtaining a second probability that the user observes each candidate key; and determining the target key observed by the user based on the second probability.

[0012] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability of the user observing each candidate key; inputting the user's current input information into a pre-trained syllable prediction model, and obtaining a third probability of each candidate key being associated with the current input information; and determining the target key observed by the user based on the second probability and the third probability.

[0013] In some embodiments, determining whether to send a request to the server based on the position of the target observation area also includes: determining not to send the candidate pre-fetch request to the server when the target observation area is located in any of the following areas: a candidate area in the input method interface, a function key area in the input method interface, and an area outside the input method interface.

[0014] In the second aspect, an embodiment of the present application provides a request sending device, which includes: an acquisition unit, configured to acquire the user's eye movement information during the user input process; a prediction unit, configured to input the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area; a determination unit, configured to determine whether to send a request to a server based on the position of the target observation area.

[0015] In some embodiments, the observation area prediction model is trained through the following steps: obtaining the user's historical eye movement information and historical operation areas during the historical input process; establishing a correspondence between the historical eye movement information and the historical operation areas; based on the correspondence, using a machine learning method to train a pre-constructed deep neural network to obtain an observation area prediction model.

[0016] In some embodiments, the determination unit is further configured to: when the target observation area is located in a candidate area in the input method interface, send a candidate acquisition request to a server, wherein the candidate acquisition request includes the current input information of the user.

[0017] In some embodiments, the determination unit is further configured to: obtain a first probability that the target observation area is located in a candidate area in the input method interface, and determine whether the target observation area is located in the candidate area based on the first probability; when it is determined that the target observation area is located in the candidate area, determine a target delay time based on the first probability and a preset delay time for sending a request; when the target delay time meets a preset condition, send a candidate acquisition request to the server, and the candidate acquisition request includes the user's current input information.

[0018] In some embodiments, the determination unit is further configured to: determine not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0019] In some embodiments, the determination unit is further configured to: when the target observation area is located in the keyboard area in the input method interface, send a candidate prefetch request to the server, the candidate prefetch request including a predicted input string, and the predicted input string is generated based on the user's current input information and the target key observed by the user.

[0020] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, obtaining a second probability that the user observes each candidate key; and determining the target key observed by the user based on the second probability.

[0021] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability of the user observing each candidate key; inputting the user's current input string into a pre-trained syllable prediction model to obtain a third probability of each candidate key being associated with the current input information; and determining the target key observed by the user based on the second probability and the third probability.

[0022] In some embodiments, the determination unit is further configured to: determine not to send the candidate pre-fetch request to the server when the target observation area is located in any of the following areas: the candidate area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0023] In the third aspect, an embodiment of the present application provides a device for sending a request, comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: during the user input process, obtaining the user's eye movement information; inputting the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area; and determining whether to send a request to a server based on the position of the target observation area.

[0024] In some embodiments, the observation area prediction model is trained through the following steps: obtaining the user's historical eye movement information and historical operation areas during the historical input process; establishing a correspondence between the historical eye movement information and the historical operation areas; based on the correspondence, using a machine learning method to train a pre-constructed deep neural network to obtain an observation area prediction model.

[0025] In some embodiments, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in a candidate area in an input method interface, sending a candidate acquisition request to the server, wherein the candidate acquisition request includes the user's current input information.

[0026] In some embodiments, determining whether to send a request to a server based on the location of the target observation area includes: obtaining a first probability that the target observation area is located in a candidate area in an input method interface, and determining whether the target observation area is located in the candidate area based on the first probability; in a case where it is determined that the target observation area is located in the candidate area, determining a target delay time based on the first probability and a preset delay time for sending a request; when the target delay time meets a preset condition, sending a candidate acquisition request to the server, the candidate acquisition request including the user's current input information.

[0027] In some embodiments, determining whether to send a request to the server based on the position of the target observation area also includes: determining not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0028] In some embodiments, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in the keyboard area in the input method interface, sending a candidate prefetch request to the server, the candidate prefetch request including a predicted input string, the predicted input string being generated based on the user's current input information and the target key observed by the user.

[0029] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, obtaining a second probability that the user observes each candidate key; and determining the target key observed by the user based on the second probability.

[0030] In some embodiments, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability of the user observing each candidate key; inputting the user's current input information into a pre-trained syllable prediction model, and obtaining a third probability of each candidate key being associated with the current input information; and determining the target key observed by the user based on the second probability and the third probability.

[0031] In some embodiments, determining whether to send a request to the server based on the position of the target observation area also includes: determining not to send the candidate pre-fetch request to the server when the target observation area is located in any of the following areas: a candidate area in the input method interface, a function key area in the input method interface, and an area outside the input method interface.

[0032] In a fourth aspect, an embodiment of the present application provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect above.

[0033] The request sending method, device and device for sending requests provided in the embodiments of the present application obtain the user's eye movement information during the user input process, and then input the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area. Finally, based on the position of the target observation area, it is determined whether to send a request to the server. Thus, a request can be sent when the position of the target observation area meets certain conditions, avoiding sending a request to the server after each character entered by the user, reducing the amount of useless requests sent, and saving computing resources of the server and local devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 is a flowchart of an embodiment of a request sending method according to the present application;

[0036] Figure 2 is a flowchart of a training process of an observation area prediction model according to the request sending method of the present application;

[0037] Figure 3 is a structural diagram of an embodiment of a request sending device according to the present application;

[0038] Figure 4 is a structural schematic diagram of a device for sending a request according to the present application;

[0039] Figure 5 It is a schematic diagram of the structure of the server in some embodiments of the present application. DETAILED DESCRIPTION

[0040] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It should also be noted that, for ease of description, only the parts related to the relevant invention are shown in the accompanying drawings.

[0041] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] Please refer to Figure 1, which shows a process 100 of an embodiment of a request sending method according to the present application. The request sending method can be run on various electronic devices, including but not limited to: servers, smart phones, tablet computers, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptop computers, car computers, desktop computers, set-top boxes, smart TVs, wearable devices, etc.

[0043] The input method application mentioned in the embodiment of the present application can support multiple input methods. Among them, the input method can be a coding method used to input various symbols into electronic devices such as computers and mobile phones. Users can use the input method application to conveniently input the required characters or strings into electronic devices. It should be noted that in the embodiment of the present application, in addition to supporting common Chinese input methods (such as pinyin input method, Wubi input method, Zhuyin input method, voice input method, handwriting input method, etc.), the input method can also support input methods of other languages ​​(such as English input method, Japanese Hiragana input method, Korean input method, etc.), and no limitation is made on the input method and the language type of the input method.

[0044] The request sending method in this embodiment may include the following steps:

[0045] Step 101: during the user input process, obtain the user's eye movement information.

[0046] In this embodiment, the user can input information in any input method, such as pinyin, Wubi, strokes, etc. During the user input process, the execution subject of the request sending method (such as the above-mentioned electronic device) can obtain the user's eye movement information. Among them, the eye movement information may include the eye image during the user input process. The above-mentioned execution subject can be installed with an image acquisition device (such as a camera) to collect the user's eye image in real time through the image acquisition device, so as to summarize the eye movement information.

[0047] Step 102: input the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area.

[0048] In this embodiment, the execution subject may store a pre-trained observation area prediction model. The observation area prediction model may analyze the eye movement information (such as extracting features, processing features, etc.) to predict the user's target observation area. The predicted target observation area is the user's next operation area.

[0049] The observation area prediction model can be pre-trained by a machine learning method (such as a supervised learning method). The above-mentioned execution subject can input the eye movement information into the trained observation area prediction model to obtain the output result of the observation area prediction model. Based on the output result, the user's target observation area is determined, such as the keyboard area of ​​the input method interface, the candidate area, the area outside the input method interface, etc. When the target observation area is the keyboard area of ​​the input method interface, it can further be one or more keys in the keyboard area.

[0050] Here, the output result of the observation area prediction model may include, but is not limited to: the probability of the user observing the keyboard area of ​​the input method interface, the probability of the user observing the candidate area of ​​the input method interface. In addition, it may further include the probability of the user observing each key in the keyboard area. Based on the output result of the observation area prediction model, the user's target observation area can be determined.

[0051] It should be noted that the observation area prediction model can be deployed locally on the above-mentioned execution subject or on the server. When the observation area prediction model is deployed on the server, the above-mentioned execution subject can send eye movement information to the server and obtain the prediction result of the target observation area returned by the server.

[0052] In some optional implementations of this embodiment, the above observation area prediction model can be obtained by training through the following sub-steps S11 to S13:

[0053] Sub-step S11, obtaining the user's historical eye movement information and historical operation area during the historical input process.

[0054] Here, historical eye movement information can be obtained during the user's historical input process (such as real-time collection of the user's eye image). The method for obtaining the historical eye movement information is basically the same as the method for obtaining the eye movement information in step 101, and will not be repeated here.

[0055] At the same time, the user's operation behavior can be monitored during the user's historical input process. The area targeted by each operation performed by the user is the historical operation area, thereby determining the historical operation area of ​​the user during the historical input process.

[0056] The historical operation area may include but is not limited to the keyboard area, the candidate area, etc. Among them, the keyboard area may further include each key in the keyboard. When a user inputs a character, he usually needs to observe the key corresponding to the keyboard area first, and then move his finger to click the key to complete the input of the character. At the same time, when the user displays the candidate items on the screen, he usually needs to observe the candidate items in the candidate area to complete the display of the required candidate items. Therefore, the historical operation area is the target observation area after the user's eye moves.

[0057] As an example, if the user needs to put "I" on the screen, the character string "wo" can be first input through the full keyboard provided by the input method application (i.e., the input method client), and then the candidate "I" in the candidate area can be selected to put the character "I" on the screen. In the process of the user inputting the character string "wo", the eyeball first moves to the upper left, toward the position of the "w" key on the full keyboard, and then the user moves the finger to click the "w" key on the keyboard. In this process, an eye movement behavior and an operation behavior are generated, and the eye movement information (such as the eye image) corresponding to this eye movement behavior can be obtained, and the historical operation area (such as the keyboard area, specifically the "w" key in the keyboard area) targeted by this operation behavior is recorded. After that, the eyeball moves to the upper right, toward the position of the "o" key on the full keyboard. Then, move the finger to click the "o" key on the keyboard. In this process, a second eye movement behavior and a second operation behavior are generated, and the eye movement information (such as the eye image) corresponding to this eye movement behavior can be obtained, and the historical operation area (such as the keyboard area, specifically the "o" key in the keyboard area) targeted by this operation behavior is recorded. Through the above operations, the input of the character string "wo" can be completed.

[0058] After inputting the character string "wo", the user can move his eyes to view the candidate area (such as the candidate bar) of the input method interface, and then click the candidate "I" to put the character "I" on the screen. In this process, the third eye movement behavior and the third operation behavior are generated, and the eye movement information (such as the eye image) corresponding to this eye movement behavior can be obtained, and the historical operation area (such as the candidate area) targeted by this operation behavior is recorded.

[0059] Sub-step S12, establishing a correspondence between historical eye movement information and historical operation areas.

[0060] Usually, before a user performs an operation (such as clicking a key or clicking a candidate item), he usually needs to observe the corresponding area (such as a key in the keyboard area or a candidate item in the candidate area), so the eye movement information has a corresponding relationship with the historical operation area. The corresponding relationship between the historical eye movement information and the historical operation area can be established to obtain training data.

[0061] The historical eye movement information involved in each input process of the user can be regarded as a training sample. Based on the correspondence between the historical eye movement information and the historical operation area, the historical eye movement information in each training sample can be labeled.

[0062] Sub-step S13, based on the corresponding relationship, a pre-built deep neural network is trained using a machine learning method to obtain an observation area prediction model.

[0063] Here, the pre-built deep neural network may be a neural network structure such as CNN (Convolutional Neural Networks).

[0064] During the training process, the historical eye movement information in the training sample can be input into the deep neural network to obtain the prediction results output by the deep neural network. The prediction results may include the probability of the user observing each area. For example, it may include but is not limited to the probability of the user observing the keyboard area, the probability of the user observing the candidate area, the probability of the user observing the area outside the input method interface, the probability of the user observing each key in the keyboard area, etc. Then, the loss value can be determined based on the annotation corresponding to the prediction result and the input training sample. The above loss value is the value of the loss function, which is a non-negative real-valued function that can be used to characterize the difference between the predicted result and the true result. In general, the smaller the loss value, the better the robustness of the model. The loss function can be set according to actual needs. Afterwards, the loss value can be used to update the parameters of the deep neural network. Thus, each time a training sample is input, the parameters of the deep neural network can be updated based on the loss value corresponding to the training sample until the training is completed.

[0065] It should be noted that since users usually need to observe a certain area before operating it, the user's operation area is the user's observation area. Therefore, the model trained using historical eye movement information and historical observation areas can be used to predict the user's target observation area.

[0066] Step 103: Determine whether to send a request to the server based on the location of the target observation area.

[0067] In this embodiment, the execution subject may determine whether to send a request to the server based on the location of the target observation area. Here, multiple implementations may be used to determine whether to send a request to the server.

[0068] In some optional implementations of this embodiment, when the predicted target observation area is located in the candidate area in the input method interface, the execution subject may send a request to the server. The request may be a candidate acquisition request, which may include the current input information of the user. The current input information may include but is not limited to at least one of the following: the current input string, the above information, the information sent by the opposite user, etc. Thus, the candidate items corresponding to the current input information returned by the server may be obtained.

[0069] It is understandable that due to different input preferences of users, such as some users prefer to input word by word, some users prefer to input word by word, and some users prefer to input sentence by sentence, it is usually difficult to determine the end of the input string. Therefore, the prior art sends a request to the server after each character entered by the user. For example, the user needs to screen "Wudaokou", and when the user enters the pinyin "w", a request is sent, and the candidate "I" returned by the server is obtained. When the user enters the pinyin "wu", a request is sent again, and the candidate "none" returned by the server is obtained. When the user continues to enter "wud", a request is sent again, and the candidate "invincible" returned by the server is obtained. These request results are not the results ultimately required by the user, so the sending timing of these requests is not the sending timing expected by the user. These requests are useless requests, which not only cannot help the user input, but also consume the computing resources of the server and the local device. This implementation method can determine whether the user has the need to click on the candidate items by predicting the user's target observation area. When the user is predicted to observe the candidate area, it means that the user has the need to click on the candidate items. In this case, sending a candidate acquisition request can greatly reduce the amount of useless requests sent, saving computing resources on the server and local devices. At the same time, sending a candidate acquisition request when it is predicted that the user is observing the candidate area can send the candidate acquisition request earlier than sending it when the user actually flips through the candidate area, which helps to reduce the display delay of the candidate items.

[0070] In some optional implementations of this embodiment, the area observed by the user can be determined based on probability. Specifically, the prediction results output by the observation area prediction model may include the probability of the user observing each area. Here, the probability of the user observing the candidate area can be referred to as the first probability, and the above-mentioned execution subject can determine whether the target observation area is located in the candidate area based on the first probability (for example, when the first probability is greater than the preset value, the target observation area can be regarded as being located in the candidate area). In the case where the target observation area is located in the candidate area, the target delay time can be determined based on the first probability and the preset delay time of sending the request (such as 200 milliseconds). For example, the preset delay time can be subtracted from the product of the first probability and the preset coefficient to obtain the target delay time. Among them, the preset coefficient can be calculated in advance based on a large amount of data and the target delay time can be calculated based on a large amount of data (such as timeout rate and arrival rate, etc.), which is not specifically limited here. Finally, when the target delay time meets the preset condition (such as when it is greater than a preset threshold), a request can be sent to the server. Among them, the request can be specifically a candidate acquisition request, and the candidate acquisition request can include the current input information of the above-mentioned user. Thus, the candidate items corresponding to the current input information returned by the server can be obtained.

[0071] By combining the probability of observing candidate areas predicted by the model with the request delay, it is possible to avoid sending useless requests due to request delays, further saving computing resources on the server and local devices.

[0072] In some optional implementations of the present embodiment, when the predicted target observation area is located in any of the following areas, it can be determined not to send the above-mentioned candidate acquisition request to the server: the keyboard area in the input method interface, the function button area (such as keyboard switch button, voice input button, emoticon button, mini-program button) in the input method interface, and the area outside the input method interface. Specifically, when the user observes the keyboard area in the input method interface, it means that the input is not finished. At this time, sending a candidate acquisition request cannot obtain the candidate items expected by the user, and thus the candidate acquisition request is not sent. When the user observes the function button area in the input method interface and the area outside the input method interface, it means that the user has not yet decided on the on-screen candidate, and there is no need to send a candidate acquisition request at this time.

[0073] In some optional implementations of this embodiment, the execution entity may send a corresponding request according to the type of area observed by the user. For example, when the target observation area is a candidate area, a candidate acquisition request may be sent to obtain a candidate corresponding to the current input information. When the target observation area is an expression area, an expression acquisition request may be sent to obtain richer expressions. When the target observation area is a search area, a search request may be sent to display a search interface. These will not be described one by one here.

[0074] In some optional implementations of the present embodiment, when the predicted target observation area is located in the keyboard area in the input method interface, a request can be sent to the server. The request sent at this time can be a candidate pre-fetch request. The candidate pre-fetch request may include a predicted input string, thereby obtaining the pre-fetch candidate corresponding to the predicted input string returned by the server. Among them, the predicted input string can be generated based on the user's current input information and the target key observed by the user. For example, it can be generated by splicing the current input information and the target key. The target key observed by the user can be selected from the keys contained in the target observation area. For example, when the target observation area is predicted to be the upper left corner of the full keyboard, the target observation area may include but is not limited to the key "q", the key "a" and the key "e", and the above-mentioned execution subject can select a preset number (such as 2) of keys as target keys.

[0075] By sending a candidate pre-fetch request based on the target key observed by the user when the user observes the keyboard area, on the one hand, the amount of calculation on the server side can be reduced, and computing resources can be saved, so as to achieve the purpose of quickly pre-fetching the server-side results at a relatively low cost. On the other hand, since the candidate pre-fetch request is sent before the user clicks the required key, the sending time is earlier, so it can be avoided that the result is difficult to return quickly due to large network delay. On the other hand, after sending the candidate pre-fetch request, the candidate result required by the user can be obtained in advance, and the candidate result can be directly displayed after the user clicks the key. For example, a candidate pre-fetch request containing a predicted input string "womq" is sent to the server, and the result returned by the server may include a candidate corresponding to the predicted input string "womq". If the user clicks the key "q" in the next step, the candidate corresponding to "womq" obtained by the previous request can be directly presented. As a result, the display efficiency of the candidate items and the fluency of the input process are improved, which helps to improve the input efficiency of the user.

[0076] In some optional implementations of the present embodiment, the prediction results output by the observation area prediction model may include the probability of the user observing each key (which may be referred to as the second probability). When the target observation area is located in the keyboard area, the above-mentioned execution subject may use each key contained in the target observation area as a candidate key to obtain the second probability of the user observing each candidate key. Then, based on the second probability, predict the target key observed by the user. As an example, the candidate keys include key "q", key "a" and key "e". The probabilities of the user observing key "q", key "a", and key "e" are recorded as p(q), p(a), and p(e) respectively. Among them, p(q) = 0.4, p(a) = 0.3, and p(e) = 0.2. At this time, two candidate keys with higher probabilities (such as "q" and key "a") can be selected as target keys.

[0077] In some optional implementations of this embodiment, the above execution subject may also combine the syllable prediction model to predict the target key. Specifically, it may be performed according to the following steps:

[0078] First, each key included in the target observation area is taken as a candidate key, and the second probability of the user observing each candidate key is obtained. This step can be referred to in the above implementation method, and will not be repeated here.

[0079] Then, the user's current input information is input into the pre-trained syllable prediction model to obtain the probability of each candidate key being associated with the current input information (which may be referred to as the third probability). The syllable prediction model may be a statistical model, and after the current input information is input into the model, the probability of each key being associated with the current input information can be obtained, so that the probability of each candidate key being associated with the current input information can be read therefrom.

[0080] Here, for a candidate key, the probability of the candidate key being associated with the current input information may specifically refer to the probability of the candidate key being connected to the input string in the current input information. The greater the probability, the greater the probability that the user will select the candidate key in the next step.

[0081] Continuing with the above example, the current input information is the input string "wom", and the candidate keys include key "q", key "a", and key "e". The probabilities of keys "q", "a", and "e" being associated with the input string "wom" are recorded as p(q|wom), p(a|wom), and p(e|wom), respectively. Among them, p(q|wom) = 0.1, p(a|wom) = 0.5, and p(e|wom) = 0.3.

[0082] Finally, based on the second probability and the third probability, the target key selected by the user is predicted. For example, for each candidate key, the second probability of the user observing the key and the third probability of the key being associated with the current input information can be multiplied, and the candidate keys with a preset number (such as 2) with a larger calculation result are used as target keys. Continuing with the above example, the preset number is 2, and the target keys are key "a" and key "e".

[0083] In some optional implementations of the present embodiment, when the predicted target observation area is located in any of the following areas, the execution subject may determine not to send the candidate pre-fetch request to the server: the candidate area in the input method interface, the function button area in the input method interface, and the area outside the input method interface. Specifically, when the user observes the candidate area in the input method interface, it means that the input has been completed and the candidate is ready to be displayed on the screen. At this time, there is no next key input, so there is no need to send a candidate pre-fetch request. When the user observes the function button area (such as keyboard switch button, voice input button, emoticon button, mini-program button) in the input method interface and the area outside the input method interface, it means that the user needs to use other functions and input the terminal at this time, so there is no need to send a candidate pre-fetch request.

[0084] The method provided by the above-mentioned embodiment of the present application obtains the user's eye movement information during the user input process, and then inputs the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area, and finally determines whether to send a request to the server based on the position of the target observation area. Thus, a request can be sent when the position of the target observation area meets certain conditions, avoiding sending a request to the server after each character entered by the user, reducing the amount of useless requests sent, and saving computing resources of the server and local devices.

[0085] Further references Figure 3 As an implementation of the methods shown in the above figures, the present application provides an embodiment of a request sending device, and the device embodiment is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0086] like Figure 3 As shown, the request sending device 300 in this embodiment includes: an acquisition unit 301, configured to acquire the eye movement information of the user during the user input process; a prediction unit 302, configured to input the eye movement information into a pre-trained observation area prediction model to predict the target observation area of ​​the user; a determination unit 303, configured to determine whether to send a request to the server based on the position of the target observation area.

[0087] In some optional implementations of the present embodiment, the above-mentioned observation area prediction model is trained through the following steps: obtaining the historical eye movement information and historical operation areas of the user during the historical input process; establishing a correspondence between the above-mentioned historical eye movement information and the above-mentioned historical operation areas; based on the above-mentioned correspondence, using a machine learning method to train a pre-constructed deep neural network to obtain the observation area prediction model.

[0088] In some optional implementations of this embodiment, the above-mentioned determination unit 303 is further configured to: when the above-mentioned target observation area is located in the candidate area in the input method interface, send a candidate acquisition request to the server, and the above-mentioned candidate acquisition request includes the current input information of the above-mentioned user.

[0089] In some optional implementations of the present embodiment, the determination unit 303 is further configured to: obtain a first probability that the target observation area is located in a candidate area in the input method interface, and determine whether the target observation area is located in the candidate area based on the first probability; when it is determined that the target observation area is located in the candidate area, determine a target delay time based on the first probability and a preset delay time for sending a request; when the target delay time meets a preset condition, send a candidate acquisition request to the server, and the candidate acquisition request includes the current input information of the user.

[0090] In some optional implementations of the present embodiment, the determination unit 303 is further configured to determine not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0091] In some optional implementations of the present embodiment, the determination unit 303 is further configured to: when the target observation area is located in the keyboard area in the input method interface, send a candidate prefetch request to the server, wherein the candidate prefetch request includes a predicted input string, and the predicted input string is generated based on the current input information of the user and the target key observed by the user.

[0092] In some optional implementations of the present embodiment, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, obtaining a second probability that the user observes each candidate key; and determining the target key observed by the user based on the second probability.

[0093] In some optional implementations of the present embodiment, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability of each candidate key observed by the user; inputting the user's current input string into a pre-trained syllable prediction model, and obtaining a third probability of each candidate key being associated with the current input information; and determining the target key observed by the user based on the second probability and the third probability.

[0094] In some optional implementations of the present embodiment, the determination unit 303 is further configured to determine not to send the candidate pre-fetch request to the server when the target observation area is located in any of the following areas: the candidate area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0095] The device provided by the above-mentioned embodiment of the present application obtains the user's eye movement information during the user input process, and then inputs the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area, and finally determines whether to send a request to the server based on the position of the target observation area. Thus, a request can be sent when the position of the target observation area meets certain conditions, avoiding sending a request to the server after each character entered by the user, reducing the amount of useless requests sent, and saving computing resources of the server and local devices.

[0096] Figure 4 4 is a block diagram of an input device 400 according to an exemplary embodiment, and the device 400 may be a smart terminal or a server. For example, the device 400 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0097] Reference Figure 4 , the device 400 may include one or more of the following components: a processing component 402 , a memory 404 , a power component 406 , a multimedia component 408 , an audio component 410 , an input / output (I / O) interface 412 , a sensor component 414 , and a communication component 416 .

[0098] The processing component 402 generally controls the overall operation of the device 400, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 402 may include one or more processors 420 to execute instructions to complete all or part of the steps of the above-described method. In addition, the processing component 402 may include one or more modules to facilitate interaction between the processing component 402 and other components. For example, the processing component 402 may include a multimedia module to facilitate interaction between the multimedia component 408 and the processing component 402.

[0099] The memory 404 is configured to store various types of data to support operations on the device 400. Examples of such data include instructions for any application or method operating on the device 400, contact data, phone book data, messages, pictures, videos, etc. The memory 404 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0100] The power supply component 406 provides power to the various components of the device 400. The power supply component 406 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 400.

[0101] The multimedia component 408 includes a screen that provides an output interface between the above-mentioned device 400 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The above-mentioned touch sensor can not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the above-mentioned touch or slide operation. In some embodiments, the multimedia component 408 includes a front camera and / or a rear camera. When the device 400 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0102] The audio component 410 is configured to output and / or input audio signals. For example, the audio component 410 includes a microphone (MIC), and when the device 400 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 404 or sent via the communication component 416. In some embodiments, the audio component 410 also includes a speaker for outputting audio signals.

[0103] I / O interface 412 provides an interface between processing component 402 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0104] The sensor assembly 414 includes one or more sensors for providing various aspects of status assessment for the device 400. For example, the sensor assembly 414 can detect the open / closed state of the device 400, the relative positioning of the components, such as the display and keypad of the device 400, the sensor assembly 414 can also detect the position change of the device 400 or a component of the device 400, the presence or absence of user contact with the device 400, the orientation or acceleration / deceleration of the device 400 and the temperature change of the device 400. The sensor assembly 414 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 414 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 414 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0105] The communication component 416 is configured to facilitate wired or wireless communication between the device 400 and other devices. The device 400 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 416 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 416 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0106] In an exemplary embodiment, the apparatus 400 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.

[0107] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 404 including instructions, and the instructions can be executed by a processor 420 of the device 400 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0108] Figure 55 is a schematic diagram of the structure of the server in some embodiments of the present application. The server 500 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 522 (for example, one or more processors) and a memory 532, and one or more storage media 530 (for example, one or more mass storage devices) storing application programs 542 or data 544. Among them, the memory 532 and the storage medium 530 can be short-term storage or permanent storage. The program stored in the storage medium 530 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 522 can be configured to communicate with the storage medium 530 and execute a series of instruction operations in the storage medium 530 on the server 500.

[0109] The server 500 may also include one or more power supplies 526, one or more wired or wireless network interfaces 550, one or more input and output interfaces 558, one or more keyboards 556, and / or one or more operating systems 541, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0110] A non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by a processor of a device (smart terminal or server), the device is able to execute a request sending method. The method includes: obtaining the user's eye movement information during the user input process; inputting the eye movement information into a pre-trained observation area prediction model to predict the user's target observation area; and determining whether to send a request to a server based on the position of the target observation area.

[0111] Optionally, the observation area prediction model is trained through the following steps: obtaining the user's historical eye movement information and historical operation areas during the historical input process; establishing a correspondence between the historical eye movement information and the historical operation areas; based on the correspondence, using a machine learning method to train a pre-constructed deep neural network to obtain the observation area prediction model.

[0112] Optionally, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in a candidate area in an input method interface, sending a candidate acquisition request to the server, the candidate acquisition request including current input information of the user.

[0113] Optionally, determining whether to send a request to a server based on the position of the target observation area includes: obtaining a first probability that the target observation area is located in a candidate area in an input method interface, and determining whether the target observation area is located in the candidate area based on the first probability; in a case where it is determined that the target observation area is located in the candidate area, determining a target delay time based on the first probability and a preset delay time for sending the request; when the target delay time meets a preset condition, sending a candidate acquisition request to the server, the candidate acquisition request including the user's current input information.

[0114] Optionally, determining whether to send a request to a server based on the position of the target observation area further includes: determining not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

[0115] Optionally, determining whether to send a request to a server based on the position of the target observation area includes: when the target observation area is located in the keyboard area in the input method interface, sending a candidate prefetch request to the server, the candidate prefetch request including a predicted input string, the predicted input string being generated based on the user's current input information and the target key observed by the user.

[0116] Optionally, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability that the user observes each candidate key; and determining the target key observed by the user based on the second probability.

[0117] Optionally, the target key observed by the user is determined by the following steps: taking each key included in the target observation area as a candidate key, and obtaining a second probability of the user observing each candidate key; inputting the user's current input information into a pre-trained syllable prediction model to obtain a third probability of each candidate key being associated with the current input information; and determining the target key observed by the user based on the second probability and the third probability.

[0118] Optionally, determining whether to send a request to the server based on the position of the target observation area also includes: determining not to send the candidate pre-fetch request to the server when the target observation area is located in any of the following areas: a candidate area in the input method interface, a function key area in the input method interface, and an area outside the input method interface.

[0119] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0120] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

[0121] The above is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0122] The above is a detailed introduction to a request sending method, device and a device for sending a request provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A request sending method, characterized in that: The method comprises: During the user input process, obtaining the user's eye movement information; The eye movement information is input into a pre-trained observation area prediction model to predict the target observation area of ​​the user; the pre-trained observation area prediction model is obtained by training a pre-constructed deep neural network using a machine learning method based on the correspondence between historical eye movement information and historical operation areas; Based on the location of the target observation area, determining whether to send a request to the server; Wherein, determining whether to send a request to a server based on the position of the target observation area includes: When the target observation area is located in the candidate area in the input method interface, sending a candidate acquisition request to the server, wherein the candidate acquisition request includes the current input information of the user; When the target observation area is located in any of the following areas, determining not to send the candidate acquisition request to the server: a keyboard area in the input method interface, a function key area in the input method interface, or an area outside the input method interface; or, The determining whether to send a request to a server based on the position of the target observation area includes: Acquire a first probability that the target observation area is located in a candidate area in the input method interface, and determine whether the target observation area is located in the candidate area based on the first probability; In the case where it is determined that the target observation area is located in the candidate area, determining a target delay time based on the first probability and a preset delay time for sending a request; When the target delay time meets a preset condition, sending a candidate acquisition request to a server, wherein the candidate acquisition request includes the current input information of the user; When the target observation area is located in any of the following areas, it is determined not to send the candidate acquisition request to the server: a keyboard area in the input method interface, a function key area in the input method interface, and an area outside the input method interface.

2. The method according to claim 1, characterized in that The observation area prediction model is trained by the following steps: Obtain the user's historical eye movement information and historical operation area during the historical input process; Establishing a correspondence between the historical eye movement information and the historical operation area; Based on the corresponding relationship, a pre-built deep neural network is trained using a machine learning method to obtain an observation area prediction model.

3. The method according to claim 1, characterized in that The determining whether to send a request to a server based on the position of the target observation area includes: When the target observation area is located in the keyboard area in the input method interface, a candidate prefetch request is sent to the server, the candidate prefetch request includes a predicted input string, and the predicted input string is generated based on the current input information of the user and the target key observed by the user.

4. The method according to claim 3, characterized in that The target key observed by the user is determined by the following steps: Taking each key included in the target observation area as a candidate key, and obtaining a second probability that the user observes each candidate key; Based on the second probability, a target key observed by the user is determined.

5. A request sending device, characterized in that: The device comprises: an acquisition unit, configured to acquire eye movement information of the user during the user input process; A prediction unit is configured to input the eye movement information into a pre-trained observation area prediction model to predict the target observation area of ​​the user; the pre-trained observation area prediction model is obtained by training a pre-built deep neural network using a machine learning method based on the correspondence between historical eye movement information and historical operation areas; a determination unit configured to determine whether to send a request to a server based on the location of the target observation area; The determination unit is further configured to send a candidate acquisition request to a server when the target observation area is located in a candidate area in the input method interface, wherein the candidate acquisition request includes the current input information of the user; determine not to send the candidate acquisition request to the server when the target observation area is located in any of the following areas: a keyboard area in the input method interface, a function key area in the input method interface, or an area outside the input method interface; or The determination unit is further configured to obtain a first probability that the target observation area is located in a candidate area in the input method interface, and determine whether the target observation area is located in the candidate area based on the first probability; when it is determined that the target observation area is located in the candidate area, determine a target delay time based on the first probability and a preset delay time for sending a request; when the target delay time meets a preset condition, send a candidate acquisition request to the server, the candidate acquisition request including the user's current input information; when the target observation area is located in any of the following areas, determine not to send the candidate acquisition request to the server: the keyboard area in the input method interface, the function key area in the input method interface, and the area outside the input method interface.

6. A device for sending a request, characterized in that The system comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors, wherein the one or more programs include instructions for performing the following operations: During the user input process, obtaining the user's eye movement information; The eye movement information is input into a pre-trained observation area prediction model to predict the target observation area of ​​the user; the pre-trained observation area prediction model is obtained by training a pre-constructed deep neural network using a machine learning method based on the correspondence between historical eye movement information and historical operation areas; Based on the location of the target observation area, determining whether to send a request to the server; Wherein, determining whether to send a request to a server based on the position of the target observation area includes: When the target observation area is located in the candidate area in the input method interface, sending a candidate acquisition request to the server, wherein the candidate acquisition request includes the current input information of the user; When the target observation area is located in any of the following areas, determining not to send the candidate acquisition request to the server: a keyboard area in the input method interface, a function key area in the input method interface, or an area outside the input method interface; or, The determining whether to send a request to a server based on the position of the target observation area includes: Acquire a first probability that the target observation area is located in a candidate area in the input method interface, and determine whether the target observation area is located in the candidate area based on the first probability; In the case where it is determined that the target observation area is located in the candidate area, determining a target delay time based on the first probability and a preset delay time for sending a request; When the target delay time meets a preset condition, sending a candidate acquisition request to a server, wherein the candidate acquisition request includes the current input information of the user; When the target observation area is located in any of the following areas, it is determined not to send the candidate acquisition request to the server: a keyboard area in the input method interface, a function key area in the input method interface, and an area outside the input method interface.

7. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images