Information processing device and information processing method

Abstract

To provide an information processing device and method for supporting an input that meets user's needs.SOLUTION: In an information processing system, an information processing device includes: a support level specification part for specifying a support level for supporting a user in operation of a terminal; a screen information acquisition part for acquiring screen information showing a configuration of contents displayed on the screen according to detection of a screen transition in the terminal; an operation information generation part for generating operation information on the basis of input support information and the support level obtained by inputting the screen information to an input support information estimation system; a support execution part for supporting input operation on the screen of the terminal on the basis of the operation information; and the input support information estimation system for outputting the input support information together with a probability according to an input of the screen information. The input support information includes an input position in the screen and input items showing kinds of input information, and the operation information includes input information candidates being candidates for input information corresponding to at least either one between the support level or the probability and an input position.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus and an information processing method.

Background Art

[0002] In the provision of services via the Internet, in order to improve the convenience of users, a system that supports the input of various types of information required when receiving a service is known (see, for example, Patent Document 1). In addition, there is a technology called RPA (Robotic Process Automation) that supports input to a computer.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional technologies such as RPA, all inputs were automatically performed regardless of the user's needs for input support. When inputting information when receiving a service, input support that meets the user's needs has been desired.

[0005] Therefore, an object of the present invention is to provide input support that meets the user's needs.

Means for Solving the Problems

[0006] To solve the above problems, an information processing apparatus according to one aspect of the present disclosure includes: a support level specifying unit that specifies a support level indicating the degree of assisting a user in operating a terminal; a screen information acquisition unit that acquires screen information indicating the configuration of the content displayed on the screen in response to detecting a screen transition in the terminal operated by the user; and an operation information generation unit that generates operation information based on the input support information obtained by inputting the screen information into an input support information estimation system and the support level, wherein the input support information estimation system outputs the input support information together with a probability in response to the input of the screen information, the input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information, the operation information includes an input information candidate which is a candidate of the input information corresponding to at least one of the support level and the probability and an input position, and a support execution unit that executes support for an input operation on the screen of the terminal based on the operation information.

[0007] According to the above aspect, operation information corresponding to at least one of the probability of the input support information and the support level is generated based on the input support information. The probability indicates the likelihood of the input support information, and the support level is based on the operation information in which information reflecting the degree of support specified for each user is reflected. Since the support for the input operation is executed, it is possible to provide input support that meets the needs of the user.

Effects of the Invention

[0008] It is possible to provide input support that meets the needs of the user.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Mode for Carrying Out the Invention

[0010] An embodiment of an information processing apparatus according to the present invention will be described with reference to the drawings. In addition, when possible, the same parts are denoted by the same reference numerals, and redundant explanations are omitted.

[0011] FIG. 1 is a block diagram showing the device configuration of an information processing system including the information processing apparatus according to the present embodiment and the functional configuration of the information processing apparatus. The information processing system 1 includes an information processing apparatus 10 and a terminal T. The terminal T is configured to be able to access the information processing apparatus 10 via a network N. The user receives service provision via the terminal T and the Internet. The information processing apparatus 10 supports the input of various types of information required when receiving service provision in providing services via the Internet.

[0012] As shown in FIG. 1, the information processing apparatus 10 functionally includes a support level specifying unit 11, a screen information acquisition unit 12, an operation information generation unit 13, and a support execution unit 14. Further, as will be described later, the operation information generation unit 13 generates operation information using an input support information estimation system 19. The input support information estimation system 19 may be configured in the information processing apparatus 10 as shown in FIG. 1, or may be configured in another apparatus accessible from the information processing apparatus 10. Details of the input support information estimation system 19 will be described later. In the example shown in FIG. 1, the functional units 11 to 14 are configured in one information processing apparatus 10, but may be distributed and configured in a plurality of apparatuses.

[0013] The terminal T is a device operated by a user, receives input of information, transmits and receives information to and from the information processing apparatus 10, and presents information by a display means (display). In the example shown in FIG. 1, three generations of the terminal T are shown, but the number of terminals T is not limited.

[0014] Each functional unit of the information processing apparatus 10 is configured to be able to access a user attribute storage unit 21 and a product information storage unit 22. The user attribute storage unit 21 is a storage means (storage) that stores user attribute information. The user attribute storage unit 21 may be configured in the information processing apparatus 10 as shown in FIG. 1, or may be configured in another apparatus accessible from the information processing apparatus 10. The user attribute storage unit 21 is a non-essential configuration in the information processing apparatus 10 of the present embodiment.

[0015] FIG. 2 is a diagram showing an example of the configuration of user attribute information stored in the user attribute storage unit 21. As shown in FIG. 2, the user attribute storage unit 21 stores attribute information of various items such as name, date of birth, gender, and credit card number in association with a user ID for identifying the user.

[0016] In addition, in this embodiment, the information processing apparatus 10 has a mode of managing the attribute information of a plurality of users. However, each terminal T may manage the attribute information of the user of the terminal T. In that case, the functional unit of the information processing apparatus 10 receives the attribute information from the terminal T.

[0017] The product information storage unit 22 is a storage means (storage) that stores product information related to products on the e-commerce site. As shown in FIG. 1, the product information storage unit 22 may be configured in the information processing apparatus 10, or may be configured in another apparatus accessible from the information processing apparatus 10. The product information storage unit 22 is a non-essential configuration in the information processing apparatus 10 of this embodiment.

[0018] FIG. 3 is a diagram showing an example of the configuration of the product information stored in the product information storage unit 22. In the example shown in FIG. 3, the product information storage unit 22 stores, for example, recommended products for each product category on the e-commerce site. Specifically, the product information storage unit 22 stores product information indicating that the product ranked first in the order of recommendation in the product category "mineral water" is "mw001", and the second and third products are "mw004" and "mw002".

[0019] In addition, in this embodiment, the information processing apparatus 10 has a mode of managing product information. However, the product information may be stored and managed in another apparatus accessible from the information processing apparatus 10. In that case, the functional unit of the information processing apparatus 10 receives the attribute information from the other apparatus.

[0020] Next, each functional unit of the information processing apparatus 10 will be described. The support level specifying unit 11 specifies a support level indicating the degree of assisting the user in the operation of the terminal T. Specifically, the support level specifying unit 11 may acquire information for specifying one support level among a plurality of support levels, for example, information for specifying one support level among five levels of support levels 1 to 5. The support level specifying unit 11 may specify the support level based on, for example, an input by the user, or may acquire the support level input in the terminal T.

[0021] Further, the support level specifying unit 11 may specify the support level based on the estimation information output from a system that estimates information related to the support level based on at least one of the user's speech and action history.

[0022] The support level specifying unit 11 may specify the support level based on the estimation information indicating the user's values output from a system that estimates the user's values based on at least one of the user's speech and action history. When the estimation information indicating the values corresponds to the support level in advance, the support level specifying unit 11 may acquire the estimation information indicating the values as the support level. Further, the support level specifying unit 11 may specify the support level based on the estimation information with reference to the information defining the association between the estimation information indicating the values and the support level.

[0023] Further, the support level specifying unit 11 may specify the support level based on the estimation information indicating the proficiency level of the user's operation of the terminal T output from a system that estimates the proficiency level of the user's operation of the terminal T based on at least one of the user's speech and action history. Specifically, the support level specifying unit 11 may acquire the estimation information indicating the proficiency level of the user's operation of the terminal T as the support level, or may specify the support level based on the estimation information with reference to the information defining the association between the estimation information and the support level.

[0024] The screen information acquisition unit 12 acquires screen information indicating the configuration of the content displayed on the screen in response to detecting a screen transition on the terminal T operated by the user. In the present embodiment, the screen information acquisition unit 12 acquires screen information as information indicating that the user has started an operation for receiving a service via the Internet on the terminal T.

[0025] Specifically, when a screen transition is detected on the terminal T, the terminal T may transmit screen information representing the screen after the screen transition to the information processing apparatus 10. The screen information may be information obtained by saving the display screen on the terminal T as an image, or so-called, a screenshot on the terminal T.

[0026] FIG. 4 and FIG. 5 are diagrams showing examples of the screen information acquired by the screen information acquisition unit 12. In the example shown in FIG. 4, the screen information D1 represents a screen for receiving a final tax return service, and includes a field f11 for entering a name and a field f12 for entering a date of birth. In the example shown in FIG. 5, the screen information D2 represents a screen for selecting a product on an e-commerce site, and includes a field f21 for specifying the selected product.

[0027] The operation information generation unit 13 generates operation information in response to detecting a screen transition and acquiring screen information. The operation information is information including input information candidates that are candidates for input information on the screen and input positions. Specifically, the operation information generation unit 13 generates operation information based on the input support information and the support level obtained by inputting the screen information into the input support information estimation system 19.

[0028] The input support information estimation system 19 outputs input support information together with a probability in response to input of the screen information. The input support information includes an input position (position of an input field) for inputting input information in the screen indicated by the screen information and an input item indicating the type of input information.

[0029] Specifically, the input support information estimation system 19 may be a machine learning model that has learned the relationship between screen information and input support information, and for example, may be configured to include a neural network.

[0030] The input support information estimation system 19, which is a machine learning model including a learned neural network, can be regarded as a program that is read or referenced by a computer, causes the computer to execute a predetermined process, and realizes a predetermined function on the computer.

[0031] That is, the learned input support information estimation system 19 of the present embodiment is used in a computer including a processor and a memory. Specifically, the processor of the computer operates according to a command from the learned input support information estimation system 19 stored in the memory, and performs an operation on input data (explanatory variable) input to the input layer of the neural network based on learned weight coefficients (parameters) corresponding to each layer and a response function or the like, and outputs a result (probability, target variable) from the output layer.

[0032] The input support information estimation system 19 is constructed by machine learning using learning data consisting of a pair of an explanatory variable consisting of screen information and a target variable consisting of an input position and an input item on the screen. Therefore, the input support information estimation system 19 outputs the input support information together with a probability representing its likelihood. And the operation information includes input information candidates and input positions, which are candidates for input information on the screen, corresponding to at least one of the support level and the probability.

[0033] FIG. 6 and FIG. 7 are diagrams schematically showing examples of the learning data of the input support information estimation system 19. As shown in FIG. 6, learning data for constructing the input support information estimation system 19 is constituted by a pair of screen information tda1 (explanatory variable) representing a screen for receiving the provision of the tax return service and correct answer data tdb1 (target variable) consisting of an input position td12 and an input item td13 "name" on the screen td11.

[0034] Also, as shown in FIG. 7, learning data for constructing the input support information estimation system 19 is constituted by a pair of screen information tda2 (explanatory variable) representing a screen for selecting a product on an e-commerce site and correct answer data td21 (objective variable) consisting of an input position td22 and an input item td23 “purchased product” on screen td21.

[0035] The operation information generation unit 13 may generate operation information based on the input support information output from the input support information estimation system 19 and the probability output together with the input support information. That is, the operation information generation unit 13 may generate operation information including input information candidates corresponding to the probability.

[0036] Specifically, when the probability is equal to or higher than a first probability, the operation information generation unit 13 may generate operation information for inputting an input information candidate at the input position, and when the probability is less than a second probability that is less than the first probability, the operation information generation unit 13 may generate operation information for presenting the input information candidate in association with the input position.

[0037] FIG. 8 is a diagram schematically showing an example of a screen on which support for an input operation is executed based on operation information. FIG. 8(a) shows an example in which support for an input operation is executed based on operation information generated when the probability output together with the input support information by the input support information estimation system 19 is equal to or higher than a first probability that is a preset threshold value, and FIG. 8(b) shows an example in which support for an input operation is executed based on operation information generated when the probability output together with the input support information is less than a second level.

[0038] In the example shown in FIG. 8, it is assumed that the input support information estimation system 19 outputs input support information including an input position f31 and an input item “name” indicating the type of input information in response to the input of screen information D1 (see FIG. 4) representing a screen for receiving a service of final tax return.

[0039] In the case shown in FIG. 8(a), when the probability output together with the input support information is equal to or higher than a first probability which is a preset threshold value, there is a high probability that it is appropriate to input the information indicated as the input item at the input position included in the input support information. In such a case, the operation information generation unit 13 generates operation information such that the input information candidate i31 of the input item "Name" is input at the input position f31 on the screen D3 of the terminal T.

[0040] The support execution unit 14 executes support for an input operation on the screen of the terminal T based on the operation information. Specifically, the support execution unit 14 refers to the user attribute storage unit 21 and acquires the name "AAA" of the user corresponding to the input item "Name" included in the operation information as an input information candidate. Then, the support execution unit 14 inputs the input information candidate i31 "AAA" at the input position f31 on the screen D3.

[0041] On the other hand, in the case shown in FIG. 8(b), when the probability output together with the input support information is less than a second probability which is a preset threshold value, there is a high probability that it is not appropriate to input the information indicated as the input item at the input position included in the input support information. In such a case, the operation information generation unit 13 generates operation information such that the input information candidate i31 of the input item "Name" is presented in association with the input position f31 on the screen D3 of the terminal T.

[0042] The support execution unit 14 executes support for an input operation on the screen of the terminal T based on the operation information. Specifically, the support execution unit 14 refers to the user attribute storage unit 21 and acquires the name "AAA" of the user corresponding to the input item "Name" included in the operation information as an input information candidate. Then, the support execution unit 14 presents the input information candidate i32 "AAA" in association with the input position f31 on the screen D3. Specifically, the support execution unit 14 displays the input information candidate i32 as a menu selectable as the input information for the input position f31.

[0043] In FIG. 8, an example in which different operation information is generated according to the comparison with the threshold value of the probability output together with the input support information has been described. However, different operation information may be generated according to the result of the comparison with the respective threshold values of the probability and the support level.

[0044] Further, the operation information generation unit 13 may generate operation information based on the input support information and the support level output from the input support information estimation system 19. That is, the operation information generation unit 13 may generate operation information including input information candidates corresponding to the support level.

[0045] Specifically, when the support level is equal to or higher than the first level, the operation information generation unit 13 generates operation information for inputting input information candidates at the input position, and when the support level is less than the second level which is less than the first level, operation information for presenting input information candidates in association with the input position may be generated.

[0046] FIG. 9 is a diagram schematically showing an example of a screen in which support for an input operation is executed based on operation information. FIG. 9(a) shows an example in which support for an input operation is executed based on operation information generated when the support level specified by the support level specifying unit 11 is equal to or higher than a preset threshold value, the first level, and FIG. 9(b) shows an example in which support for an input operation is executed based on operation information generated when the support level is less than the second level.

[0047] In the example shown in FIG. 9, it is assumed that the input support information estimation system 19 outputs input support information including the input position f41 and the input item "purchased product" indicating the type of input information in response to the input of the screen information D2 (see FIG. 5) representing a screen for selecting a product on an e-commerce site.

[0048] In the case shown in FIG. 9(a), when the support level is equal to or higher than a first level which is a preset threshold value, it is highly probable that the user receiving the service requires a high level of input support. In such a case, the operation information generation unit 13 generates operation information such that input information candidate i41 of the input item "purchased product" is input at the input position f41 on the screen D4 of the terminal T.

[0049] The support execution unit 14 executes support for the input operation on the screen of the terminal T based on the operation information. Specifically, the support execution unit 14 refers to the product information storage unit 22 and acquires recommended products of "mineral water", which is the category of the product to be purchased on the screen D4 (see FIG. 3). Then, the support execution unit 14 inputs the product "mw001" with the highest recommendation order among the recommended products to the input position f41. As a result, the user can complete the product purchase process only by performing an operation to determine the purchase on the terminal.

[0050] On the other hand, in the case shown in FIG. 9(b), when the support level is less than a second level which is a preset threshold value, it is highly probable that the user receiving the service does not require a high level of input support. In such a case, the operation information generation unit 13 generates operation information such that the input information candidate i42 of the input item "purchased product" is presented in association with the input position f41 on the screen D4 of the terminal T.

[0051] The support execution unit 14 executes support for input operations on the screen of the terminal T based on the operation information. Specifically, the support execution unit 14 refers to the product information storage unit 22 and acquires recommended products of "mineral water", which is the category of the product to be purchased on the screen D4 (see Fig. 3). Then, the support execution unit 14 presents the recommended products "mw001", "mw004", and "mw002" arranged in the order of recommendation in association with the input position f41. Specifically, the support execution unit 14 displays the input information candidate i42 as a selectable menu as the input information to the input position f41. Thereby, the user can obtain an opportunity to select the information to be input, that is, an opportunity to select the product to be purchased.

[0052] Note that in Fig. 9, an example in which different operation information is generated according to the comparison with the threshold value of the support level has been described. However, different operation information may be generated according to the comparison result of the probability output together with the input support information and each threshold value of the support level.

[0053] Fig. 10 is a flowchart showing the processing contents of the information processing method in the information processing apparatus 10.

[0054] In step S1, the support level specifying unit 11 specifies a support level indicating the degree of supporting the user in the operation of the terminal T.

[0055] In step S2, the screen information acquisition unit 12 determines whether or not a screen transition has been detected in the terminal T operated by the user. If a screen transition is detected, the process proceeds to step S3. If a screen transition is not detected, the process of step S2 is repeated and an attempt is made to detect a screen transition.

[0056] In step S3, the screen information acquisition unit 12 acquires screen information indicating the configuration of the content displayed on the screen.

[0057] In step S4, the operation information generation unit 13 inputs the screen information into the input support information estimation system 19. In step S5, the operation information generation unit 13 acquires the input support information together with the probability.

[0058] In step S6, the operation information generation unit 13 generates operation information according to the support level and the probability. In step S7, the support execution unit 14 performs support for the implantation operation based on the operation information.

[0059] Next, with reference to FIG. 11, an information processing program for causing a computer to function as the information processing apparatus 10 of the present embodiment will be described. FIG. 11 is a diagram showing the configuration of the information processing program. The information processing program P1 includes a main module m10 that comprehensively controls the information processing in the information processing apparatus 10, a support level specifying module m11, a screen information acquisition module m12, an operation information generation module m13, and a support execution module m14. And each function for each functional unit 11 - 14 is realized by each of the modules m11 - m14.

[0060] Note that the information processing program P1 may be transmitted via a transmission medium such as a communication line, or may be stored in the recording medium M1 as shown in FIG. 11.

[0061] According to the information processing system 1, the information processing apparatus 10, the information processing method, and the information processing program P1 of the present embodiment described above, operation information corresponding to at least one of the probability of the input support information and the support level is generated based on the input support information. The probability indicates the likelihood of the input support information, and the support level reflects the degree of support specified for each user. Based on the operation information, support for the input operation is executed, so that it is possible to provide input support that meets the user's needs.

[0062] The information processing apparatus and the information processing method according to the present disclosure may have the following configurations. Also, the actions and effects in each configuration are described as follows.

[0063] An information processing apparatus according to an aspect of the present disclosure includes: a support level specifying unit that specifies a support level indicating the degree of assisting a user in operating a terminal; a screen information acquisition unit that acquires screen information indicating the configuration of the content displayed on the screen in response to detection of a screen transition in the terminal operated by the user; and an operation information generation unit that generates operation information based on the input support information and the support level obtained by inputting the screen information into an input support information estimation system. The input support information estimation system outputs input support information together with a probability in response to input of the screen information. The input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information. The operation information includes an input information candidate, which is a candidate for the input information corresponding to at least one of the support level and the probability, and an input position. The information processing apparatus further includes a support execution unit that executes support for an input operation on the screen of the terminal based on the operation information.

[0064] An information processing method according to an aspect of the present disclosure includes: a support level specifying step of specifying, by a processor, a support level indicating the degree of assisting a user in operating a terminal; a screen information acquisition step of acquiring, in response to detection of a screen transition in the terminal operated by the user, screen information indicating the configuration of the content displayed on the screen; an operation information generation step of generating operation information based on the input support information and the support level obtained by inputting the screen information into an input support information estimation system. The input support information estimation system outputs input support information together with a probability in response to input of the screen information. The input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information. The operation information includes an input information candidate, which is a candidate for the input information corresponding to the support level and the probability, and an input position. The information processing method further includes a support execution step of executing support for an input operation on the screen of the terminal based on the operation information.

[0065] According to the above aspect, based on the input support information, operation information corresponding to at least one of the probability of the input support information and the support level is generated. Since the probability indicates the likelihood of the input support information, and the support level reflects the degree of support specified for each user, the input operation is supported based on the operation information, so it is possible to provide input support that meets the user's needs.

[0066] Also, in the information processing apparatus according to another aspect, when the support level is equal to or higher than the first level, the operation information generation unit may generate operation information for inputting an input information candidate to the input position, and when the support level is lower than the first level and lower than the second level, the operation information generation unit may generate operation information for presenting the input information candidate in association with the input position.

[0067] According to the above aspect, when the support level is high, a screen in which the input information candidate is input to the input position is provided, so that input support that meets the needs of users who require advanced support is provided. On the other hand, when the support level is low, a screen in which the input information candidate is only presented in association with the input position is provided, so that it is possible to provide appropriate input support for users who do not require advanced support.

[0068] Also, in the information processing apparatus according to another aspect, when the probability is equal to or higher than the first probability, the operation information generation unit may generate operation information for inputting an input information candidate to the input position, and when the probability is lower than the first probability and lower than the second probability, the operation information generation unit may generate operation information for presenting the input information candidate in association with the input position.

[0069] According to the above aspect, when the probability of the input support information is high, a screen in which the input information candidate is input to the input position is provided, so that the labor of input is saved. On the other hand, when the probability of the input support information is low, a screen in which the input information candidate is only presented in association with the input position is provided, so that the selection of the input information to be finally input is entrusted to the user, and thus it is possible to support the input of appropriate input information.

[0070] Also, in the information processing apparatus according to another aspect, when the input item included in the input support information relates to the user's attributes, the operation information generation unit may generate operation information including the user's attributes corresponding to the input item as input information candidates by referring to the user's attribute information.

[0071] According to the above aspect, by referring to the user's attribute information, operation information having the user's attributes as input information candidates is generated, so that it is possible to support the input of information related to the user's attributes.

[0072] Also, in the information processing apparatus according to another aspect, when the input item included in the input support information is product information indicating a product to be purchased on an e-commerce site, the operation information generation unit may generate operation information including the product information of the products recommended on the e-commerce site as input information candidates.

[0073] According to the above aspect, by referring to the recommended product information, operation information having the product information as input candidates is generated, so that it is possible to support the input of product information on the e-commerce site.

[0074] Also, in the information processing apparatus according to another aspect, the support level specifying unit may specify the support level based on the estimation information output from a system that estimates information related to the support level based on at least one of the input by the user or the user's speech and action history.

[0075] According to the above aspect, it is possible to specify an appropriate support level for the user to be supported.

[0076] Also, in the information processing apparatus according to another aspect, the input support information estimation system may be a machine learning model constructed by machine learning using learning data consisting of a pair of an explanatory variable consisting of screen information and an objective variable consisting of an input position and an input item.

[0077] According to the above aspect, by using the input support information estimation system constructed as a machine learning model, it is possible to obtain highly accurate input support information including the input position and input items on the screen.

[0078] Further, in the information processing apparatus according to another aspect, the screen information may be information obtained by saving the display screen on the terminal as an image.

[0079] According to the above aspect, it is possible to easily acquire the screen information used for the input of the input support information estimation system, triggered by detecting the transition of the screen on the terminal.

[0080] Note that the block diagram shown in FIG. 1 shows the blocks in terms of functions. These functional blocks (components) are realized by any combination of at least one of hardware and software. Also, the realization method of each functional block is not particularly limited. That is, each functional block may be realized using one physically or logically combined device, or two or more physically or logically separated devices may be directly or indirectly (for example, using wired, wireless, etc.) connected and realized using these multiple devices. The functional block may be realized by combining software with the above one device or the above multiple devices.

[0081] Functions include, but are not limited to, judgment, decision, determination, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, solution, selection, selection, establishment, comparison, assumption, expectation, regarded as, notification (broadcasting), notification (notifying), communication (communicating), transfer (forwarding), configuration (configuring), reconfiguration (reconfiguring), allocation (allocating, mapping), assignment (assigning), etc. For example, a functional block (component) that functions as transmission is called a transmission unit or a transmitter. As described above, the realization method is not particularly limited.

[0082] For example, the information processing apparatus 10 in one embodiment of the present invention may function as a computer. FIG. 12 is a diagram showing an example of the hardware configuration of the information processing apparatus 10 according to the present embodiment. Physically, the information processing apparatus 10 may be configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

[0083] In the following description, the term "apparatus" can be read as a circuit, a device, a unit, or the like. The hardware configuration of the information processing apparatus 10 may be configured to include one or more of each apparatus shown in FIG. 12, or may be configured without including some of the apparatuses.

[0084] Each function in the information processing apparatus 10 is realized by causing the processor 1001 to perform operations by loading a predetermined software (program) onto hardware such as the processor 1001 and the memory 1002, and controlling communication by the communication device 1004 and reading and / or writing data in the memory 1002 and the storage 1003.

[0085] The processor 1001 controls the entire computer by operating an operating system, for example. The processor 1001 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like. For example, each functional unit 11 to 14 shown in FIG. 1 may be realized by the processor 1001.

[0086] Also, the processor 1001 reads a program (program code), software module, and data from the storage 1003 and / or the communication device 1004 into the memory 1002, and executes various processes according to them. As the program, a program that causes a computer to execute at least a part of the operations described in the above embodiments is used. For example, each functional unit 11 to 14 of the information processing apparatus 10 may be stored in the memory 1002 and realized by a control program operating on the processor 1001. Although it has been described that the above various processes are executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented with one or more chips. Note that the program may be transmitted from a network via a telecommunication line.

[0087] The memory 1002 is a computer-readable recording medium and may be composed of at least one of, for example, ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. The memory 1002 may be referred to as a register, cache, main memory (main storage device), etc. The memory 1002 can store a program (program code), software module, etc. that can execute the information processing method according to an embodiment of the present invention.

[0088] Storage 1003 is a computer-readable recording medium and may be composed of at least one of, for example, optical discs such as CD-ROM (Compact Disc ROM), hard disk drives, flexible disks, magneto-optical disks (e.g., compact discs, digital versatile discs, Blu-ray (registered trademark) discs), smart cards, flash memories (e.g., cards, sticks, key drives), floppy (registered trademark) disks, magnetic strips, etc. Storage 1003 may also be referred to as an auxiliary storage device. The above-described storage medium may be, for example, a database including memory 1002 and / or storage 1003, a server, or other appropriate media.

[0089] Communication device 1004 is hardware (a transmission / reception device) for performing communication between computers via wired and / or wireless networks and is also referred to as, for example, a network device, a network controller, a network card, a communication module, etc.

[0090] Input device 1005 is an input device (e.g., a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives external input. Output device 1006 is an output device (e.g., a display, a speaker, an LED lamp, etc.) that performs output to the outside. Note that input device 1005 and output device 1006 may have an integrated configuration (e.g., a touch panel).

[0091] Also, each device such as processor 1001 and memory 1002 is connected by a bus 1007 for communicating information. Bus 1007 may be composed of a single bus or may be composed of different buses between devices.

[0092] Further, the information processing apparatus 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented by at least one of these hardware components.

[0093] The notification of information is not limited to the aspects / embodiments described in the present disclosure, and other methods may be used. For example, the notification of information may be performed by physical layer signaling (e.g., downlink control information (DCI), uplink control information (UCI)), upper layer signaling (e.g., radio resource control (RRC) signaling, medium access control (MAC) signaling, notification information (master information block (MIB), system information block (SIB))), other signals, or a combination thereof. Further, the RRC signaling may be referred to as an RRC message, and may be, for example, an RRC connection setup message, an RRC connection reconfiguration message, or the like.

[0094] Each aspect / embodiment described in the present disclosure may be applied to at least one of systems using LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), other suitable systems, and next-generation systems extended based on these. Further, a plurality of systems may be combined (for example, a combination of at least one of LTE and LTE-A and 5G, etc.) and applied.

[0095] The processing procedures, sequences, flowcharts, etc. of each aspect / embodiment described in the present disclosure may be rearranged as long as there is no contradiction. For example, regarding the methods described in the present disclosure, the elements of various steps are presented using an exemplary order and are not limited to the specific order presented.

[0096] The specific operations assumed to be performed by the base station in the present disclosure may, in some cases, be performed by its upper node. In a network consisting of one or more network nodes having a base station, various operations performed for communication with a terminal can clearly be performed by at least one of the base station and other network nodes other than the base station (for example, but not limited to, MME or S-GW, etc.). Although the case where there is one other network node other than the base station has been exemplified above, a combination of a plurality of other network nodes (for example, MME and S-GW) may also be possible.

[0097] Information and the like can be output from an upper layer (or lower layer) to a lower layer (or upper layer). It may also be input and output via a plurality of network nodes.

[0098] The input and output information and the like may be stored in a specific location (for example, memory), or may be managed by a management table. The input and output information and the like can be overwritten, updated, or appended. The output information and the like may be deleted. The input information and the like may be transmitted to other devices.

[0099] The determination may be made by a value represented by 1 bit (0 or 1), may be made by a Boolean value (true or false), or may be made by a numerical comparison (for example, comparison with a predetermined value).

[0100] Each aspect / embodiment described in the present disclosure may be used alone, may be used in combination, or may be switched and used during execution. Also, the notification of predetermined information (for example, the notification of "being X") is not limited to being explicitly performed, and may be performed implicitly (for example, by not performing the notification of the predetermined information).

[0101] As described in detail above, it is obvious to those skilled in the art that the present disclosure is not limited to the embodiments described in the present disclosure. The present disclosure can be implemented in modified and changed forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is for illustrative purposes only and does not have any limiting meaning for the present disclosure.

[0102] Software should be broadly construed to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether called software, firmware, middleware, microcode, hardware description language, or by any other name.

[0103] Also, software, instructions, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, server, or other remote source using wired technologies such as coaxial cables, optical fiber cables, twisted pairs, and digital subscriber lines (DSL) and / or wireless technologies such as infrared, wireless, and microwave, these wired and / or wireless technologies are included within the definition of the transmission medium.

[0104] The information, signals, etc. described in the present disclosure may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0105] In addition, terms described in the present disclosure and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

[0106] The terms "system" and "network" used in this disclosure are used interchangeably.

[0107] Also, the information, parameters, etc. described in this disclosure may be represented by absolute values, relative values from a predetermined value, or other corresponding information. For example, wireless resources may be indicated by indexes.

[0108] The names used for the above-described parameters are not limiting names in any way. Furthermore, mathematical formulas and the like using these parameters may be different from those explicitly disclosed in this disclosure. Since various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable names, the various names assigned to these various channels and information elements are not limiting names in any way.

[0109] The terms "determining" and "determination" as used in this disclosure may encompass a wide variety of actions. "Determining" and "determination" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up (e.g., searching in a table, database, or other data structure), ascertaining, and considering something as having been "determined" or "determined". Also, "determining" and "determination" may include considering something as having been "determined" or "determined" based on receiving (e.g., receiving information), transmitting (e.g., transmitting information), inputting, outputting, accessing (e.g., accessing data in memory), etc. Further, "determining" and "determination" may include considering something as having been "determined" or "determined" based on resolving, selecting, choosing, establishing, comparing, etc. That is, "determining" and "determination" may include considering something as having been "determined" or "determined" based on any action. Also, "determining (determination)" may be read as "assuming", "expecting", "considering", etc.

[0110] The description "based on" as used in this disclosure does not mean "only based on" unless otherwise specified. In other words, the description "based on" means both "only based on" and "at least based on".

[0111] When terms such as "first", "second", etc. are used in the present disclosure, any reference to those elements does not generally limit the quantity or order of those elements. These terms can be used in this specification as a convenient way to distinguish between two or more elements. Therefore, references to the first and second elements do not mean that only two elements can be adopted there, or that the first element must precede the second element in some form.

[0112] The "means" in the configuration of each of the above devices may be replaced with "part", "circuit", "device", etc.

[0113] As long as "include", "including", and their variants are used in this specification or the claims, these terms are intended to be inclusive, similar to the term "comprising". Furthermore, the term "or" used in this specification or the claims is not intended to be an exclusive disjunction.

[0114] In the present disclosure, for example, when articles are added by translation, such as a, an, and the in English, the present disclosure may include that the nouns following these articles are in the plural form.

[0115] In the present disclosure, the term "A is different from B" may mean that "A and B are different from each other". Note that the term may also mean that "A and B are each different from C". Terms such as "separate", "coupled", etc. may also be interpreted in the same way as "different".

[0116] The information processing apparatus 10 and the information processing method of the present disclosure may have the following configuration.

[0117] [1] An assistance level specifying unit that specifies an assistance level indicating the degree of assisting the user in the operation of the terminal, and A screen information acquisition unit that acquires screen information indicating the configuration of the content displayed on the screen in response to detection of screen transitions on a terminal operated by a user; An operation information generation unit that generates operation information based on the input support information obtained by inputting the screen information into an input support information estimation system and the support level, wherein the input support information estimation system outputs input support information together with a probability in response to input of the screen information, the input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information, and the operation information includes an input information candidate that is a candidate of the input information corresponding to at least one of the support level and the probability and the input position; A support execution unit that executes support for an input operation on the screen of the terminal based on the operation information; An information processing apparatus comprising the same. [2] When the support level is equal to or higher than a first level, the operation information generation unit generates the operation information for inputting the input information candidate at the input position, and when the support level is lower than a second level that is lower than the first level, the operation information generation unit generates the operation information for presenting the input information candidate in association with the input position. The information processing apparatus according to [1]. [3] When the probability is equal to or higher than a first probability, the operation information generation unit generates the operation information for inputting the input information candidate at the input position, and when the probability is lower than a second probability that is lower than the first probability, the operation information generation unit generates the operation information for presenting the input information candidate in association with the input position. The information processing apparatus according to [1] or [2]. [4] The operation information generation unit When the input item included in the input support information relates to the attributes of the user, Refers to the attribute information of the user and generates the operation information including the attribute of the user corresponding to the input item as the input information candidate. The information processing apparatus according to any one of [1] to [3]. [5] The operation information generation unit When the input item included in the input support information is product information indicating a product to be purchased on an electronic commerce site, generating the operation information including product information of products recommended on the electronic commerce site as the input information candidates; The information processing device according to any one of [1] to [3]. [6] the support level determination unit determines the support level based on estimated information output from a system that estimates information related to the support level based on at least one of an input by the user or a speech and a behavior history of the user; The information processing device according to any one of [1] to [5]. [7] the input assistance information estimation system is a machine learning model constructed by machine learning using learning data consisting of pairs of explanatory variables consisting of the screen information and objective variables consisting of the input position and the input item, The information processing device according to any one of [1] to [6]. [8] The information processing device according to any one of [1] to [7], wherein the screen information is information in which a display screen of a terminal is saved as an image. [9] an assistance level specification step of specifying an assistance level indicating a degree of assistance to a user in operating the terminal; a screen information acquiring step of acquiring screen information indicating a configuration of contents displayed on the screen in response to detection of a screen transition on the terminal operated by the user; An operation information generation step of generating operation information based on the input support information obtained by inputting the screen information into the input support information estimation system and the support level, wherein the input support information estimation system outputs input support information together with a probability in response to the input of the screen information, the input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information, and the operation information includes an input information candidate which is a candidate of the input information corresponding to the support level and the probability and the input position. A support execution step of executing support for an input operation on the screen of the terminal based on the operation information. An information processing method executed by a processor, which has the above steps.

Explanation of Signs

[0118] 1... Information processing system, 10... Information processing device, 11... Support level specifying unit, 12... Screen information acquisition unit, 13... Operation information generation unit, 14... Support execution unit, 19... Input support information estimation system, 21... User attribute storage unit, 22... Product information storage unit, M1... Recording medium, m10... Main module, m11... Support level specifying module, m12... Screen information acquisition module, m13... Operation information generation module, m14... Support execution module, P1... Information processing program, T... Terminal. 。

Claims

1. An assistance level specifying unit that specifies an assistance level indicating the degree of assisting a user in terminal operations, A screen information acquisition unit that acquires screen information indicating the configuration of the content displayed on the screen in response to detection of screen transitions in the terminal operated by the user, An operation information generation unit that generates operation information based on the input assistance information obtained by inputting the screen information into an input assistance information estimation system and the assistance level, wherein the input assistance information estimation system outputs input assistance information together with a probability in response to the input of the screen information, the input assistance information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information, and the operation information includes an input information candidate that is a candidate of the input information corresponding to at least one of the assistance level and the probability and the input position, An assistance execution unit that executes assistance for input operations on the screen of the terminal based on the operation information, An information processing apparatus comprising the above.

2. When the assistance level is equal to or higher than a first level, the operation information generation unit generates the operation information for inputting the input information candidate at the input position, and when the assistance level is lower than a second level that is lower than the first level, the operation information generation unit generates the operation information for presenting the input information candidate in association with the input position, The information processing apparatus according to Claim 1.

3. When the probability is equal to or higher than a first probability, the operation information generation unit generates the operation information for inputting the input information candidate at the input position, and when the probability is lower than a second probability that is lower than the first probability, the operation information generation unit generates the operation information for presenting the input information candidate in association with the input position, The information processing apparatus according to Claim 1.

4. The operation information generation unit, when the input item included in the input assistance information relates to the attributes of the user, refers to the attribute information of the user and generates the operation information including the attribute of the user corresponding to the input item as the input information candidate, The information processing apparatus according to Claim 1.

5. The operation information generation unit, when the input item included in the input assistance information is product information indicating a product to be purchased on an e-commerce site, generates the operation information including the product information of the product recommended on the e-commerce site as the input information candidate, The information processing apparatus according to Claim 1.

6. The support level specifying unit specifies the support level based on the estimation information output from a system that estimates information related to the support level based on at least one of the input by the user, or the speech and action history of the user. The information processing apparatus according to claim 1.

7. The input support information estimation system is a machine learning model constructed by machine learning using learning data consisting of pairs of explanatory variables consisting of the screen information and target variables consisting of the input position and the input item. The information processing apparatus according to claim 1.

8. The screen information is information obtained by saving a display screen on a terminal as an image. The information processing apparatus according to claim 1.

9. A support level specifying step of specifying a support level indicating the degree of assisting the user in operating the terminal; A screen information acquisition step of acquiring screen information indicating the configuration of the content displayed on the screen in response to detection of a screen transition on the terminal operated by the user; An operation information generation step of generating operation information based on the input support information obtained by inputting the screen information into the input support information estimation system and the support level, wherein the input support information estimation system outputs the input support information together with a probability in response to the input of the screen information, the input support information includes an input position for inputting input information in the screen indicated by the screen information and an input item indicating the type of the input information, and the operation information includes an input information candidate which is a candidate of the input information corresponding to the support level and the probability and the input position. A support execution step of executing support for an input operation on the screen of the terminal based on the operation information; An information processing method executed by a processor, comprising:

Images