Information processing device, information processing method, and information processing program

Abstract

To properly process information to be processed.SOLUTION: An information processing device comprises a generation unit and an identification output unit. The generation unit generates conversion information by converting a first object included in processing information, which is information to be processed, to a second object using a model which has learned so as to convert the first object to the second object. The identification output unit identifies a range including the second object among conversion information using a model which has learned so as to identify the range including the second object, and outputs the identified range as a range including the first object among processing information.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and an information processing program.

Background Art

[0002] Conventionally, techniques for identifying information to be processed have been provided. For example, a technique for extracting feature amounts of objects in an input image, which is information to be processed, and identifying the objects in the image from the feature amounts has been provided.

Prior Art Documents

Patent Documents

[0003]

Patent Document Document=Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is room for improvement in the above conventional technology. In the above conventional technology, in order to extract feature amounts, the input image is reduced, and the feature amounts are not extracted in the same size as the input image. Therefore, it is desired to appropriately perform the processing of the information to be processed.

[0005] The present application has been made in view of the above, and an object thereof is to provide an information processing apparatus, an information processing method, and an information processing program that appropriately perform the processing of information to be processed.

Means for Solving the Problems

[0006] To solve the above-mentioned problems and achieve the objective, the information processing device of the present invention is characterized by comprising: a generation unit that generates converted information obtained by converting the first object included in processing information, which is information to be processed, to a second object using a model that has been trained to convert the first object to a second object; and a specific output unit that identifies the range in which the second object is included from the converted information using a model that has been trained to identify the range in which the second object is included, and outputs the identified range as the range in which the first object is included from the processing information. [Effects of the Invention]

[0007] According to one embodiment, the effect is achieved that appropriate processing of the information to be processed can be achieved. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 shows an example of information processing according to the embodiment. [Figure 2] Figure 2 shows an example of the configuration of an information processing system according to the embodiment. [Figure 3] Figure 3 shows an example of the configuration of an information processing terminal device according to an embodiment. [Figure 4] Figure 4 shows an example of a model information storage unit according to the embodiment. [Figure 5] Figure 5 is a flowchart showing an example of the process according to this embodiment. [Figure 6] Figure 6 shows an example of a hardware configuration. [Modes for carrying out the invention]

[0009] The following describes in detail, with reference to the drawings, embodiments for implementing the information processing device, information processing method, and information processing program according to the present application (hereinafter referred to as "embodiments"). Note that these embodiments do not limit the information processing device, information processing method, and information processing program according to the present application. Furthermore, the same parts are denoted by the same reference numerals in the following embodiments, and redundant descriptions are omitted.

[0010] (Embodiment) [1. Information Processing] First, an example of information processing according to the embodiment will be described with reference to Figure 1. Figure 1 is a diagram showing an example of information processing according to the embodiment. In Figure 1, an example will be described in which the information processing system 1 generates conversion information CI by converting the first target SB1 included in the processing information I (also called "processing information I") held by user U1 into a second target SB2, identifies the range in which the second target SB2 of the conversion information CI is included, and outputs identification information II as the range in which the first target SB1 of the processing information I is included. Note that the information to be processed is not limited to what the user has, and may be acquired by any means, for example, from an external device. Also, character information is not limited to what the user inputs, and may be acquired by any means, for example, from an external device. Furthermore, in the following, the terminal device 10 among the devices included in the information processing system 1 will be described as the processing entity for specifying the range, but various processing such as the processing of specifying the range may be performed by any device of the information processing system 1, such as the server device 50, but this point will be described later.

[0011] [1-1. Configuration of the Information Processing System] Prior to explaining Figure 1, the configuration of the information processing system 1 that realizes the information processing shown below will be explained using Figure 2. Figure 2 is a diagram showing an example configuration of the information processing system according to the embodiment. As shown in Figure 2, the information processing system 1 includes a plurality of terminal devices 10 and a server device 50. The terminal devices 10 and the server device 50 are connected to each other via a predetermined network N, either by wire or wireless communication. Note that the information processing system 1 shown in Figure 2 may include a plurality of terminal devices 10 and a plurality of server devices 50.

[0012] Terminal device 10 is a computer (information processing device) used by the user. Terminal device 10 can be implemented as, for example, a desktop PC (Personal Computer), a notebook PC, a tablet device, a smartphone, a mobile phone, or a PDA (Personal Digital Assistant). Note that each terminal device 2 is not limited to the examples above, and may also be, for example, a smartwatch or a wearable device. The following example shows the case where terminal device 10 is a smartphone.

[0013] The terminal device 10 generates conversion information CI by converting the first target SB1 contained in the processing information I to the second target SB2, using a learning model (also called the "conversion information generation model") that has been trained to convert the first target SB1 to the second target SB2 based on the processing information I. The terminal device 10 also uses a learning model (also called the "range identification model") that has been trained to identify the range that contains the second target SB2 to identify the range that contains the second target SB2 within the conversion information CI. The terminal device 10 then outputs the identified range as the range that contains the first target SB1 within the processing information. This series of processes is sometimes referred to as the identification process.

[0014] Furthermore, the terminal device 10 displays various types of information. For example, the terminal device 10 displays various types of information through various applications. For example, the terminal device 10 displays content such as web content. The terminal device 10 outputs specific information II, which includes the specific range identified by the specific processing described above.

[0015] The terminal device 10 may use various prior arts related to content display as appropriate to execute various processing related to content display using control information, etc. The terminal device 10 may execute various processing related to content display using control information. The terminal device 10 may acquire a script to be executed on a predetermined application such as a web browser as control information and execute the acquired script. Such control information corresponds to a display program, etc. according to the embodiment, and is implemented, for example, by CSS (Cascading Style Sheets), JavaScript (registered trademark), HTML (HyperText Markup Language), or any language capable of describing the above-mentioned display processing, etc. The terminal device 10, etc. that executes the above-mentioned display processing, etc. according to the display program according to the embodiment will be described in detail below.

[0016] The server device 50 is a computer (information processing device) that provides various information used by the terminal device 10 for processing. For example, the server device 50 is a server managed by the administrator of the information processing system 1. The server device 50 may also perform specific processing, but this will be discussed later.

[0017] The server device 50 provides various information to the user. The server device 50 transmits various information to the terminal device 10 used by the user. For example, the server device 50 distributes various content such as web content. The server device 50 receives requests for the provision of various information from the terminal device 10 and transmits information corresponding to the received requests to the terminal device 10. For example, the server device 50 receives a request for the distribution of web content from the terminal device 10 and distributes the received web content to the terminal device 10.

[0018] For example, the server device 50 provides various types of information used in processing to the terminal device 10. The server device 50 may distribute various models such as a range identification model to the terminal device 10. Note that the distribution of the model may be performed by a device other than the server device 50.

[0019] 〔1-2. Overall Outline of Processing in Information Processing System〕 Hereafter, an example of the information processing performed by the information processing system 1 will be described using FIG. 1. Note that detailed descriptions of points similar to those of conventional processing related to specific processing will be omitted as appropriate for the processing performed in the information processing system 1.

[0020] In FIG. 1, the terminal device 10 acquires information I (processing information I) that is the processing target input by the user U1. The processing information I is, for example, a character string. However, as shown in FIG. 1, the processing information I may include an input image IM1 and a character string BW1. For example, the terminal device 10 acquires the input image IM1 possessed by the user U1 and character information "XXXXXX", which is a sentence for converting the first target SB1 included in the input image IM1, as the character string BW1. In FIG. 1, the image IM1 is an image of a plant and the first target SB1 is a flower, but it is not limited to plants or images. Also, in FIG. 1, it is shown as an abstract character string "XXXXXX", but the character string shall be character information indicating specific content. For example, the character information "XXXXXX" is a character string including specific content such as "Convert the flower into a human face", and includes, for example, character information representing an article and character information indicating the content to be displayed after conversion.

[0021] Furthermore, the terminal device 10 obtains a transformation information generation model M1 (also simply called "model M1") from the server device 50, which generates transformation information CI from processing information I. Model M1 is a transformation information generation model that takes processing information I as input and outputs transformation information CI corresponding to processing information I. For example, model M1 outputs transformation information CI appropriate as input to the range identification model M2, which will be described later, from the input processing information I. Specifically, model M1 transforms the processing information I from the input processing information I, for example, by adding or removing the content of the string BW1 (for example, information about a person's face) as noise information to the input image IM1, thereby generating and outputting transformation information CI. Note that when the terminal device 10 is training model M1, it may obtain data to be used in the training process of model M1 (training data) from the server device 50, etc., and train model M1 using the training data, but this point will be described later. Furthermore, Model M1 is not limited to generating transformed information CI by adding or removing noise information to processed information I, but may generate transformed information CI using any technique. Also, in Figure 1, the transformed image IM2 is shown as the transformed information CI.

[0022] The terminal device 10 inputs processing information I as input information to model M1 (step S1-1). In Figure 1, the terminal device 10 inputs the input image IM1 of a plant, input by user U1, and the string "XXXXXX" indicating the conversion content as input information to model M1. Model M1, having received the input information, outputs conversion information CI (step S1-2). In Figure 1, model M1, having received the input information, outputs conversion information CI in which the first target SB1 (flower) has been converted to the second target SB2 (human face). Thus, by using model M1, the terminal device 10 generates appropriate conversion information CI from processing information I as input to range identification model M2.

[0023] Then, the terminal device 10 takes the conversion information CI as input information and uses a range identification model M2 (also simply called "model M2") to identify and output the range corresponding to the conversion information CI, and outputs the identification information II with the range displayed. The terminal device 10 inputs the conversion information CI as input information to model M2 (step S2-1). In Figure 1, the terminal device 10 inputs the conversion information CI, in which the first target SB1, a flower, has been converted to the second target SB2, a human face, as input information to model M2.

[0024] Model M2, upon receiving the input information, generates and outputs a specific range corresponding to the input transformation information CI (step S2-2). In Figure 1, Model M2, upon receiving the transformation information CI, outputs specific information II corresponding to the processing information I input by user U1. In Figure 1, specific information II is shown in the output image IM3. For example, specific information II is information that reflects the input information entered by user U1, and since the frame indicating the specified range includes the second target SB2, it also includes the first target SB1. The frame of specific information II includes the second target SB2, and since the second target SB2 is a transformation of the first target SB1, the position of the second target SB2 includes the first target SB1. In other words, the specified range is output as the range in the processing information I that includes the first target SB1. Therefore, it can be said that the first target SB1 exists at the position of the second target SB2. In Figure 1, the identified area is shown by a rectangular frame, but the shape of the frame can be any shape, such as a circle or polygon, and it may also be indicated by coloring or arrows. In this way, the terminal device 10 can identify the area by converting the first target SB1 (flower in Figure 1) included in the information to be processed input by user U1 to the second target SB2 (human face in Figure 1) and identifying the area, thereby identifying the area as the area containing the first target SB1 (flower). When the terminal device 10 identifies the first target SB1, it may also display related information such as the type, name, and website of the first target SB1, or provide services related to the first target.

[0025] As described above, the terminal device 10 uses model M1 to generate conversion information CI, which is a conversion of processing information I into content suitable for input to model M2, and by inputting the generated conversion information CI to model M2, it can output the range that includes the first target SB1. In other words, by converting the first target SB1 to the second target SB2, the range of the first target SB1 can be identified. Therefore, even when it is difficult for the terminal device 10 to recognize the first target SB1, it becomes easier to recognize the first target SB1.

[0026] Here, we will further explain the first target SB1 and the second target SB2. In the example in Figure 1, the first target SB1 was a flower, but it is not limited to flowers and can be anything, such as a fish or a vehicle. In the example in Figure 1, the second target SB2 was a human face, but it is not limited to human faces and can be anything, such as a dog or a cat. However, it is preferable that the second target SB2 has a higher accuracy in identifying the area included in the processing information I than the first target SB1. In other words, it is preferable that the second target SB2 is easier to identify than the first target SB1. Therefore, Model M1 can be said to be a model that converts the first target SB1 into a second target SB2 that is easier to identify (detect) than the first target SB1. Also, Model 2 can be said to be a model that identifies various things (dogs, cats, etc.) as long as they have a higher accuracy in identifying than the first target SB1. For example, in imaging, humans are imaged more often than plants. Therefore, it can be said that humans (human faces) have a higher accuracy in identifying than plants. On the other hand, since plants are not imaged more often than humans, it can be said that plants have a lower accuracy in identifying than humans.

[0027] Furthermore, by making the second target SB2 easier to identify than the first target SB1, the terminal device 10 can identify the first target SB1 if it was unable to do so before. For example, if the first target SB1 is not associated and its location cannot be determined, the location of the first target SB1 is manually associated. In this case, as described above, the information processing system 1 uses model M1 to convert the first target SB1 into the second target SB2, which is easier to identify than the first target SB1, and generates conversion information CI. By inputting this conversion information CI into model M2, the range (location) of the first target SB1 can be determined, making it easier to identify the first target SB1. This leads to cost and time reductions. In addition, if further association of related information such as the name, type, and website of the first target SB1 is required, the association of related information can be done using other services such as crowdsourcing.

[0028] [1-3. System Configuration] The configuration of the information processing system 1 described above is merely an example, and the information processing system 1 can employ any device configuration and distribution of functions. In Figure 1, the case where the terminal device 10 is an information processing device that identifies the scope of the first target was explained as an example, but in the information processing system 1, the identification process may be performed by an information processing device other than the terminal device 10 used by the user. For example, in the information processing system 1, the identification process may be performed by a server device (information processing device) such as the server device 50.

[0029] In this case, in the information processing system 1, the server device 50 generates converted information CI from the input information entered by user U1 through the process shown in Figure 1, and uses the converted information CI as input to identify the range that includes the first target SB1. The server device 50 may then provide the identified specific information II to the user by transmitting the identified information to the terminal device 10 used by the user. The terminal device 10 used by the user may then display the identified specific information II received from the server device 50.

[0030] Thus, in the information processing system 1, the information processing device that displays specific information II (e.g., terminal device 10) and the information processing device that identifies the scope of the first target (e.g., server device 50) may be separate entities. For example, in the information processing system 1, the server device 50 may be a providing device (information processing device) that generates and provides information to the user using the terminal device 10, and the terminal device 10 may be a display device that displays the information provided by the server device 50. Note that the configuration of the information processing system 1 described above is merely an example, and the information processing system 1 is not limited to the above; any device configuration and distribution of functions can be adopted.

[0031] [1-4. Other examples] The processes described above are merely examples, and the information processing system 1 may generate information using various types of information, or provide the generated information to the user. Furthermore, processes described as being performed by the information processing system 1 may be performed by any of the devices (information processing devices) included in the information processing system 1, such as the terminal device 10 or the server device 50.

[0032] For example, information processing system 1 may create model M1 using a string. For example, it may take the string "Convert "A" to "B"" as input and generate model M1 that converts A to B. In this case, information processing system 1 generates model M1 by learning the characteristics of the relationships between strings of high importance, as will be described later. For example, from the string "Convert "A" to "B"", information processing system 1 may identify "A" and "B" as strings of high importance and learn the relationships between these strings (for example, A is a flower, B is a human face).

[0033] For example, the information processing system 1 may process information other than images. The terminal device 10 may process audio. For example, the terminal device 10 may convert music into different sounds, melodies, speech, etc., to identify characteristic parts of the music (e.g., chorus, intro, etc.).

[0034] For example, information processing system 1 may process speech and text. For example, terminal device 10 may translate language-specific expressions into English and specify their range.

[0035] For example, the information processing system 1 may identify the range of a specific item from among multiple items. For example, if the input image IM1 contains multiple items, the terminal device 10 may identify a specific item from among those multiple items. For example, if the input image IM1 contains a first item and a second item, and the first item is to be identified, model M1 converts the second item into another item, and model M2 identifies the range of the first item. Specifically, if the first item in the input image IM1 is a red flower and the second item is a blue flower, model M1 converts the blue flower into another item (e.g., a human face), and model M2 identifies the range of the red flower. In this case, it is preferable that model M2 is a model that can appropriately detect the first item. Note that the first and second items are not limited to plants, but may be arbitrary, such as a dog and a cat, a building and a car, etc.

[0036] [2. Configuration of the Information Processing Device] Next, the configuration of a terminal device 10, which is an example of an information processing device, will be described using Figure 3. Figure 3 is a diagram showing an example of the configuration of an information processing device according to the embodiment. As shown in Figure 3, the terminal device 10 has a communication unit 11, an input unit 12, an output unit 13, a storage unit 14, and a control unit 15. The terminal device 10 may also have a microphone (sound sensor) and a speaker that serve as an audio input / output interface. For example, the sound sensor and speaker of the terminal device 10 may be connected to the terminal device 10 so that they can communicate via external connection or the like.

[0037] (Communications Section 11) The communication unit 11 is implemented, for example, by a communication circuit. The communication unit 11 is connected by wire or wireless to a predetermined communication network (not shown) and transmits and receives information with an external information processing device. For example, the communication unit 11 is connected by wire or wireless to a predetermined network N (see Figure 2) and transmits and receives information with the server device 50.

[0038] (Input section 12) The input unit 12 receives various operations from the user. For example, the input unit 12 may accept various operations from the user via a display surface (e.g., output unit 13) using a touch panel function. Alternatively, the input unit 12 may accept various operations from buttons provided on the terminal device 10, or from a keyboard or mouse connected to the terminal device 10.

[0039] (Output section 13) The output unit 13 is a screen for displaying information. For example, the output unit 13 is a display screen for a tablet terminal, etc., which is implemented using a liquid crystal display or an organic EL (Electro-Luminescence) display, and is a display device for displaying various types of information. The output unit 13 may also function as a touch panel screen.

[0040] (Storage unit 14) The storage unit 14 is implemented by, for example, a semiconductor memory element such as RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or optical disc. The storage unit 14 stores, for example, information related to applications installed on the terminal device 10 (e.g., specific processing applications), such as programs. Furthermore, as shown in Figure 3, the storage unit 14 according to this embodiment has a model information storage unit 141, a specific information storage unit 142, and a user information storage unit 143. The storage unit 14 may also store various other types of information. For example, the storage unit 14 may store generated information or identified information. The storage unit 14 may also store information used by the terminal device 10 to display multiple images. The storage unit 14 may also store information used by the terminal device 10 to determine which image to display. For example, the storage unit 14 may store information indicating the display order of images.

[0041] (Model information storage unit 141) The model information storage unit 141 according to the embodiment stores information about the model. For example, the model information storage unit 141 stores information (model data) of a trained model (model) that has been trained (generated) through a training process. The model information storage unit 141 shown in Figure 4 stores the data used for training (training data) in association with the trained model (model). Figure 4 is a diagram showing an example of the model information storage unit according to the embodiment. In the example shown in Figure 4, the model information storage unit 141 includes items such as "model ID", "purpose", "model data", and "training data". In the example in Figure 4, the model information storage unit 141 stores the data used for training (training data) in association with the trained model (model).

[0042] The "Model ID" indicates identification information for identifying the model. The "Purpose" indicates the purpose of the corresponding model. The "Model Data" indicates the data of the model. Figure 4 shows an example where conceptual information such as "MDT1" is stored in "Model Data," but in reality, it includes various information that constitutes the model, such as information about the model's configuration (network configuration) and parameters. For example, "Model Data" includes information such as the nodes in each layer of the network, the functions adopted by each node, the connection relationships between nodes, and the connection coefficients set for the connections between nodes.

[0043] "Training data" refers to the data used to train a trained model (model). "Training data" stores information indicating the dataset used to train the corresponding model. For example, "training data" stores data (input information) and the corresponding correct answer information (output information) as training data (also called "training data"). Figure 4 shows an example where conceptual information such as "LDT1" is stored in "training data," but in reality, it includes various information about the data used to train the corresponding model, such as data (input information) and the corresponding correct answer information (output information).

[0044] Figure 4 shows that the model identified by model ID "M1" (model M1) is intended for "transformation information generation." In other words, model M1 is a model that transforms input information and outputs (generates) it. It also shows that the model data for model M1 is model data MDT1. Furthermore, it shows that the training data used to train model M1 is training data LDT1.

[0045] Furthermore, the model identified by model ID "M2" (model M2) indicates that its purpose is "range identification." That is, model M2 is a model that takes transformation information as input and identifies and outputs the range corresponding to that transformation information. It also indicates that the model data for model M2 is model data MDT2. Furthermore, it indicates that the training data used to train model M2 is training data LDT2.

[0046] (Identification information storage unit 142) The identification information storage unit 142 according to this embodiment stores various information related to specific processing. For example, the identification information storage unit 142 stores various information used for specific information II provided to the user.

[0047] The identification information storage unit 142 stores various information used to identify the range of processing information including images. The identification information storage unit 142 stores various information used to identify the range of processing information including predetermined music. For example, the identification information storage unit 142 stores the sound source (music data) of predetermined music to be attached to the processing information. The identification information storage unit 142 stores various information used to identify the range of processing information including special effects. For example, the identification information storage unit 142 stores the data (effect data) of special effects to be attached to the processing information.

[0048] The identification information storage unit 142 stores various information used to identify the range of processing information including the name of the specified target. The identification information storage unit 142 stores various information used to identify the range of processing information including the description of the specified target. The identification information storage unit 142 stores the identified range as the range of processing information I that includes the first target SB1.

[0049] For example, the specific information storage unit 142 stores information used to identify the range of processing information displayed on a display device such as the terminal device 10. For example, the specific information storage unit 142 stores image information used to identify the range of processing information. For example, the specific information storage unit 142 may store images in various formats such as JPEG (Joint Photographic Experts Group), GIF (Graphics Interchange Format), and PNG (Portable Network Graphics) as image information used to identify the range of processing information. In addition, the specific information storage unit 142 may store information in the format of CSS, JavaScript, or HTML as image information used to identify the range of processing information.

[0050] Furthermore, the specific information storage unit 142 is not limited to the above and may store various types of information depending on the purpose.

[0051] (User information storage unit 143) The user information storage unit 143 according to this embodiment stores various information about the user. The user information storage unit 143 stores various information about the user who uses the terminal device 10. For example, the user information storage unit 143 stores various information about user attributes and various information about user behavior.

[0052] The user information storage unit 143 stores information about the user's future actions, such as planned activities. For example, the user information storage unit 143 stores information about planned activities (schedules) that the user has registered in a calendar application (also simply called "calendar"). The user information storage unit 143 stores schedule information related to the user's plans (schedules), such as the user's calendar.

[0053] The user information storage unit 143 stores information about the user's past behavior, such as their activity history. The user information storage unit 143 stores various types of behavioral information, such as the user's activity on the internet. The user information storage unit 143 stores various attribute information about the user, such as their age, gender, address, and place of employment. For example, the user information storage unit 143 stores information about the user's preferences, such as the objects of their interest.

[0054] The user information storage unit 143 is not limited to the above and may store various types of information depending on the purpose. For example, the user information storage unit 143 may store other demographic attribute information or psychographic attribute information. For example, the user information storage unit 143 may store information such as name, family structure, income, and lifestyle. The user information storage unit 143 may also store information (terminal ID) that identifies the terminal device 10 used by the user.

[0055] (Acquisition part 151) The acquisition unit 151 acquires various types of information. For example, the acquisition unit 151 acquires various types of information from the storage unit 14. For example, the acquisition unit 151 acquires various types of information from the model information storage unit 141, the specific information storage unit 142, the user information storage unit 143, etc. The acquisition unit 151 may also acquire various types of information from an external information processing device.

[0056] For example, the acquisition unit 151 acquires various information from the server device 50. For example, the acquisition unit 151 acquires various information used for processing from the server device 50. For example, the acquisition unit 151 acquires models such as model M1 and M2 from the server device 50.

[0057] The acquisition unit 151 receives various types of information. The acquisition unit 151 receives various operations from the user. For example, the acquisition unit 151 receives various operations from the user via the input unit 12. The acquisition unit 151 receives user operations. The acquisition unit 151 receives operations where the user moves their finger that is touching the screen. The acquisition unit 151 receives operations where the user selects the displayed content.

[0058] The acquisition unit 151 acquires processing information I. The acquisition unit 151 acquires a string that is processing information I. The acquisition unit 151 acquires an image that is processing information I. The acquisition unit 151 acquires a string to convert the image. The acquisition unit 151 acquires a string that contains a string related to the item.

[0059] (Learning Section 152) The learning unit 152 executes a learning process to learn a learning model (model). Note that the terminal device 10 does not need to have a learning unit 152 if it obtains learning models such as model M1 and M2 from the server device 50.

[0060] For example, the learning unit 152 performs learning processing based on various information acquired by the acquisition unit 151. The learning unit 152 performs learning processing based on information from an external information processing device and information stored in the storage unit 14. The learning unit 152 performs learning processing based on information stored in the model information storage unit 141. The learning unit 152 stores the model generated by learning in the model information storage unit 141.

[0061] The learning unit 152 performs learning processing. The learning unit 152 performs various types of learning. The learning unit 152 learns various types of information based on the information acquired by the acquisition unit 151. The learning unit 152 learns (generates) a model. The learning unit 152 learns various types of information such as the model. The learning unit 152 generates a model through learning. The learning unit 152 learns the model using various machine learning techniques. For example, the learning unit 152 learns the parameters of the model (network). The learning unit 152 learns the model using various machine learning techniques.

[0062] The learning unit 152 generates various learning models such as model M1. The learning unit 152 learns the network parameters. For example, the learning unit 152 learns the network parameters of various learning models such as model M1. The learning unit 152 generates various learning models such as model M1 by performing learning processing using the learning data stored in the model information storage unit 141. For example, the learning unit 152 generates a model used to generate transformation information CI. The learning unit 152 generates various learning models such as model M1 by learning the network parameters of various learning models such as model M1.

[0063] The learning unit 152 performs learning processing based on the learning data (training data) stored in the model information storage unit 141. The learning unit 152 generates various learning models such as model M1 by performing learning processing using the learning data stored in the model information storage unit 141. For example, the learning unit 152 learns model M1, which is a transformation information generation model that learns the characteristics of the relationship between processing information I and strings with high importance in that processing information I. The learning unit 152 learns model M1 that extracts character information from processing information I (strings) and outputs transformation information CI by transforming the first target SB1 included in the processing information into a second target SB2 according to the content of the character information. The learning unit 152 learns model M1 that extracts character information with high importance from the character information and outputs transformation information CI.

[0064] For example, the learning unit 152 trains model M1 using training data that associates processing information I with correct information, which is the desired conversion information CI that is output when the processing information I is input to model M1. The correct information may also be conversion information CI that is appropriate as input to model M2. For example, the correct information may also be conversion information CI that generates a desired specific range when input to model M2. For example, the correct information may also be conversion information CI that generates a desired specific range when input to model M2 as specific information II (range) corresponding to character information. In other words, the correct information is conversion information CI that is appropriate as input information to a range identification model.

[0065] For example, the learning unit 152 performs learning processing using methods such as backpropagation so that the transformation information CI output by model M1 approaches the correct information associated with the character information (input information) input to model M1. For example, the learning unit 152 performs learning processing so that the change information output by model M1, which is input information that is character information indicating the transformation content, approaches the correct information associated with that input information.

[0066] For example, the learning unit 152 adjusts the values ​​of the weights (i.e., connection coefficients) that are considered when values ​​are transmitted between nodes during the learning process. In this way, the learning unit 152 learns model M1 by processing such as backpropagation to correct the parameters (connection coefficients) so that the error between the output of model M1 and the correct information corresponding to the input is reduced. For example, the learning unit 152 generates model M1 by processing such as backpropagation to minimize a predetermined loss function. This allows the learning unit 152 to perform a learning process to learn the parameters of model M1.

[0067] Furthermore, the model training method is not limited to the methods described above, and any publicly known technique can be applied. The generation of each model may be performed using various conventional machine learning techniques as appropriate. For example, model generation may be performed using supervised machine learning techniques such as SVM (Support Vector Machine). Alternatively, model generation may be performed using unsupervised machine learning techniques. For example, model generation may be performed using deep learning techniques. For example, model generation may be performed using various deep learning techniques such as DNN (Deep Natural Network), RNN (Recurrent Neural Network), and CNN (Convolutional Neural Network) as appropriate. The above description of model generation is illustrative, and model generation may be performed using a training method appropriately selected according to the available information. In other words, the training unit 152 may generate model M1 by any method as long as it can train model M1 to output information corresponding to the correct answer information when input information included in the training data is input.

[0068] As described above, the learning method used by the learning unit 152 is not particularly limited, but for example, learning data may be prepared by linking data (input information) with correct answer information (output information), and this learning data may be input into a computational model based on a multilayer neural network for learning. Alternatively, methods based on DNNs such as CNNs and 3D-CNNs may be used. When dealing with time-series data such as audio, the learning unit 152 may use methods based on recurrent neural networks (RNNs) or LSTMs (Long Short-Term Memory units), which are extensions of RNNs.

[0069] (Generation unit 153) The generation unit 153 generates various types of information. The generation unit 153 generates various types of information based on the information acquired by the acquisition unit 151. The generation unit 153 generates various types of information based on the information stored in the storage unit 14. The generation unit 153 generates various types of information based on the information stored in the model information storage unit 141, the specific information storage unit 142, the user information storage unit 143, etc. The generation unit 153 performs a decision process to determine various types of information. The generation unit 153 performs an estimation process to estimate various types of information. The generation unit 153 performs a decision extraction process to extract various types of information. The generation unit 153 performs a selection process to select various types of information.

[0070] The generation unit 153 generates conversion information CI by converting the first target SB1 included in the processing information I into the second target SB2 based on the processing information. The generation unit 153 generates conversion information CI by converting the processing information I. The generation unit 153 generates conversion information CI by converting to the second target SB2, which has a higher accuracy in identifying the region included in the processing information I than the first target SB1.

[0071] The generation unit 153 generates transformation information CI using a transformation information generation model M1, which is a learning model that has been trained to transform the first target SB1 into the second target SB2, based on the processing information I. When processing information I is input to the generation unit 153, it generates transformation information CI using a transformation information generation model M1 that transforms and outputs the processing information I based on the processing information I. The generation unit 153 inputs the processing information I to the transformation information generation model M1 and generates transformation information CI using the output information output by the transformation information generation model M1.

[0072] (Specific output unit 154) The specific output unit 154 identifies various types of information. The specific output unit 154 identifies various types of information based on the information acquired by the acquisition unit 151. The specific output unit 154 identifies various types of information based on the information stored in the storage unit 14. The specific output unit 154 identifies various types of information based on the information generated by the generation unit 153. The specific output unit 154 identifies various types of information based on the information stored in the model information storage unit 141, the specific information storage unit 142, the user information storage unit 143, etc. The specific output unit 154 performs a decision process to determine various types of information. The specific output unit 154 performs an estimation process to estimate various types of information. The specific output unit 154 performs a decision extraction process to extract various types of information. The specific output unit 154 performs a selection process to select various types of information.

[0073] The specific output unit 154 uses a range identification model M2, which is a learned model trained to identify a range from the conversion information CI, to identify the range of the second target SB2 corresponding to the conversion information CI. When conversion information CI is input, the specific output unit 154 uses the range identification model M2, which outputs a range corresponding to the conversion information CI, to identify the range of the second target SB2 within the conversion information CI. The specific output unit 154 also outputs the identified range as the range in the processing information I that includes the first target SB1.

[0074] The specific output unit 154 executes a process to generate specific information II to be provided to users and other services (such as crowdsourcing). The specific output unit 154 generates specific information II to be displayed on the screen (output unit 13). For example, the specific output unit 154 generates specific information II (image) from the range identification model M2, with a frame displayed around the second target SB2.

[0075] (Output control unit 155) The output control unit 155 outputs the specific information II identified (generated) by the specific output unit 154. For example, the output control unit 155 outputs the specific information II to the display unit 156. Alternatively, the output control unit 155 may provide the specific information II to other services. For example, the output control unit 155 may provide the specific information II to a crowdsourcing service.

[0076] (Display section 156) The display unit 156 functions as a provider unit that provides specific information II, whose range is specified, generated by the specific output unit 154. The display unit 156 displays various types of information. For example, the display unit 156 displays various types of information according to user operations input by the input unit 12.

[0077] For example, the display unit 156 displays various information via the output unit 13. The display unit 156 displays various information based on the information acquired by the acquisition unit 151. The display unit 156 displays various information based on the information stored in the storage unit 14. The display unit 156 displays various information based on the information stored in the model information storage unit 141, the specific information storage unit 142, the user information storage unit 143, etc. The display unit 156 displays various information processed by the generation unit 153 and the specific output unit 154.

[0078] The display unit 156 provides specific information II identified by the specific output unit 154. The display unit 156 provides specific information II including the output image IM3.

[0079] The display unit 156 displays the output image IM3. The display unit 156 also displays specific information II, which includes the output image IM3.

[0080] (Transmitter 157) The transmitting unit 157 functions as a provider unit that provides specific information II identified by the specific output unit 154. The transmitting unit 157 transmits various types of information. For example, the transmitting unit 157 transmits various types of information to an external information processing device in accordance with user operations input by the input unit 12. The transmitting unit 157 also transmits request information to the external information processing device requesting various types of information in response to user operations.

[0081] The transmitting unit 157 provides specific information II identified by the specific output unit 154. The transmitting unit 157 provides specific information II including the output image IM3.

[0082] The transmitting unit 157 transmits specific information II to an external device. The transmitting unit 157 transmits specific information II, including the output image IM3.

[0083] The transmitting unit 157 transmits request information to the server device 50 requesting information. The transmitting unit 157 transmits request information requesting content delivery. The transmitting unit 157 transmits user action information to the server device 50. The transmitting unit 157 transmits action information indicating user operations.

[0084] Furthermore, if the information processing and other operations performed by the control unit 15 described above are carried out by a predetermined application, each part of the control unit 15 may be implemented by, for example, the predetermined application. For example, the information processing and other operations performed by the control unit 15 may be implemented by control information including JavaScript (registered trademark). Also, if the information processing and other operations described above are carried out by a dedicated application, the control unit 15 may have, for example, an application control unit that controls a predetermined application (e.g., a range-specific application) or a dedicated application.

[0085] [3. Information Processing Flow] Next, the procedure for information processing by the terminal device 10 according to this embodiment will be explained using Figure 5. Figure 5 is a flowchart showing an example of information processing according to this embodiment.

[0086] As shown in Figure 5, the terminal device 10 acquires the information I to be processed (step S101). Then, based on the information to be processed, the terminal device 10 generates conversion information CI by converting the first target SB1 contained in the information to be processed into the second target SB2 (step S102). Then, the terminal device 10 takes the conversion information CI as input and identifies the range in which the second target SB2 is included (step S103). Then, the terminal device 10 outputs identification information II that identifies the range of the second target SB2 (step S104).

[0087] [4. Effects] As described above, the information processing device according to the embodiment (terminal device 10 in the embodiment) includes a generation unit (generation unit 153 in the embodiment), a specific output unit (specific output unit 154 in the embodiment), and an output control unit (output control unit 155 in the embodiment). The generation unit generates converted information by converting the first object included in the processing information, which is the information to be processed, to the second object, using a model that has been trained to convert the first object to the second object. The specific output unit identifies the range in the converted information that includes the second object, using a model that has been trained to identify the range that includes the second object, and outputs the identified range as the range in the information that includes the first object.

[0088] Thus, the information processing device according to the embodiment can perform appropriate processing on the information to be processed by using a model that has been trained to convert a first object to a second object to generate converted information in which the first object contained in the information to be processed is converted to the second object, and using a model that has been trained to identify the range in which the second object is included to identify the range in which the second object is included to identify the range in which the second object is included from the converted information.

[0089] Furthermore, in the information processing apparatus according to this embodiment, the second target is a target with higher accuracy in identifying the region included in the processing information than the first target.

[0090] Thus, the information processing device according to this embodiment can improve the accuracy of range identification by making the second target a target with higher accuracy in identifying the area contained in the information than the first target. Furthermore, it can prevent situations where the information processing device cannot recognize the location of the first target, for example, if the first target is not labeled. In other words, it can perform appropriate processing of the information to be processed.

[0091] Furthermore, in the information processing apparatus according to the embodiment, the processing information is a string. In this way, by using a string for the information, the information processing apparatus according to the embodiment can convert the first object contained in the string into a second object, and can perform appropriate processing on the information to be processed.

[0092] Furthermore, in the information processing apparatus according to the embodiment, the information to be processed is an image. In this way, by using an image for the information, the information processing apparatus according to the embodiment can convert a first object contained in the image into a second object, and can perform appropriate processing on the information to be processed.

[0093] Furthermore, in the information processing device according to this embodiment, the second object is a human face. By making the second object a human face in the information processing device according to this embodiment, it becomes easier to identify than the first object, and the accuracy of range identification can be increased. In addition, the information processing device can prevent situations where the first object is not recognized, for example, because it is not labeled. That is, it can process the information to be processed appropriately.

[0094] [5. Program] The processing performed by the terminal device 10 and server device 50 described above is realized by the information processing program according to the present invention. For example, the generation unit 153 and the specific output unit 154 of the terminal device 10 are realized by the CPU or MPU of the terminal device 10, for example, by the information processing program included in a range-specific application using RAM as a working area, and the information processing procedures related to the information processing program are executed.

[0095] Furthermore, the processing performed by the terminal device 10 and server device 50 according to this application does not necessarily have to be entirely implemented by an information processing program. For example, information outside the terminal device 10 may be obtained by the OS (Operating System) of the terminal device 10. In other words, the information processing program itself may not execute the processing performed by the terminal device 10 as described above, but rather may implement the processing of the terminal device 10 as described above by receiving data obtained by the OS (for example, data used to display (play) content such as images).

[0096] [6. Hardware Configuration] The information processing device 10 and other division processing devices according to the above-described embodiment are implemented by a computer 1000, for example, as shown in Figure 6. Figure 6 is a diagram showing an example of a hardware configuration. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, communication interface (I / F) 1500, input / output interface (I / F) 1600, and media interface (I / F) 1700.

[0097] The arithmetic unit 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, as well as programs read from the input device 1020, and executes various processes. The arithmetic unit 1030 can be implemented using, for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array).

[0098] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, controlling various components. The ROM 1300 stores boot programs executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0099] The HDD1400 stores programs executed by the CPU1100, as well as data used by such programs. The communication interface1500 receives data from other devices via a predetermined network N and sends it to the CPU1100, and transmits data generated by the CPU1100 to other devices via the predetermined network N.

[0100] The CPU 1100 controls output devices such as displays and printers, and input devices such as keyboards and mice, via the input / output interface 1600. The CPU 1100 acquires data from input devices via the input / output interface 1600. The CPU 1100 also outputs the generated data to output devices via the input / output interface 1600.

[0101] The media interface 1700 reads a program or data stored in the storage medium 1800 and provides it to the CPU 1100 via the RAM 1200. The CPU 1100 loads the program from the storage medium 1800 onto the RAM 1200 via the media interface 1700 and executes the loaded program. The storage medium 1800 can be, for example, an optical storage medium such as a DVD (Digital Versatile Disc) or PD (Phase Change Rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0102] For example, when the computer 1000 functions as a terminal device 10 according to the embodiment, the CPU 1100 of the computer 1000 realizes the functions of the control unit 15 by executing a program loaded on the RAM 1200. The CPU 1100 of the computer 1000 reads and executes these programs from the storage medium 1800, but as another example, these programs may be obtained from other devices via a predetermined network N.

[0103] Although embodiments of the present application have been described in detail based on the drawings, these are illustrative examples, and the present invention can be implemented in various other forms with modifications and improvements based on the knowledge of those skilled in the art, starting with the embodiments described in the disclosure lines of the invention.

[0104] [7. Other] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, specific names, and information including various data and parameters shown in the above document and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each figure is not limited to the information shown.

[0105] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads and usage conditions.

[0106] Furthermore, the embodiments and modifications described above can be combined as appropriate, provided that the processing content is not inconsistent.

[0107] Furthermore, the "section, module, unit" mentioned above is a "means" It can be reinterpreted as "or circuit," etc. For example, the acquisition unit can be reinterpreted as acquisition means or acquisition circuit. [Explanation of Symbols]

[0108] 1. Information Processing System 10. Terminal device (information processing device) 14 Storage section 141 Model Information Storage Unit 143 User Information Storage Unit 15 Control Unit 151 Acquisition Department 152 Learning Department 153 Generation part 154 Specific output section 155 Output Control Unit 156 Display section (provider section) 157 Transmitting Section (Providing Section) 50 Server Devices N Network

Claims

1. A generation unit generates converted information in which the first object included in the processing information, which is the information to be processed, is converted to the second object, using a model that has been trained to convert the first object to the second object. A specific output unit that uses a model trained to identify the range that includes the second target to identify the range that includes the second target from the conversion information, and outputs the identified range as the range that includes the first target from the processing information, An information processing device characterized by comprising:

2. The second target is one in which the accuracy of identifying the region included in the processing information is higher than that of the first target. The information processing apparatus according to feature 1.

3. The processing information is a string. The information processing apparatus according to claim 1 or 2.

4. The processing information is an image. The information processing apparatus according to claim 1 or 2.

5. The second object mentioned above is a human face. The information processing apparatus according to claim 1 or 2.

6. An information processing method performed by an information processing device, A generation step of generating transformed information by transforming the first object contained in the processing information, which is the information to be processed, into the second object, using a model that has been trained to transform the first object into the second object, A specific output step which involves using a model trained to identify the range that includes the second target to identify the range that includes the second target from the conversion information, and outputting the identified range as the range that includes the first target from the information, An information processing method characterized by including

7. A generation procedure that generates transformed information by transforming the first object contained in the processing information, which is the information to be processed, into the second object, using a model that has been trained to transform the first object into the second object, A specific output procedure that uses a model trained to identify the range that includes the second target to identify the range that includes the second target from the transformed information, and outputs the identified range as the range that includes the first target from the information, An information processing program that causes a computer to execute something.

Images