Information processing device and information processing method

Abstract

This information processing device comprises: a first acquisition unit that acquires concentration degree information indicating the concentration degree of a viewer when the viewer views content; a determination unit that determines a policy for modifying the content in order to improve the concentration degree indicated by the concentration degree information; and a provision unit that provides the viewer with the content modified in accordance with the policy.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present invention relates to a technology for providing improved content according to the degree of concentration.

[0002] Technologies for measuring a user's degree of concentration are known. For example, Patent Document 1 discloses an invention that measures the degree of concentration based on an image of a face region.

[0003] Japanese Unexamined Patent Application Publication No. 2021-089594

[0004] The invention described in Patent Document 1 merely measures the degree of concentration, and does not modify content such as moving images in accordance with the degree of concentration.

[0005] In response to this, the present invention provides a technology for providing improved content according to the degree of concentration.

[0006] An information processing apparatus according to one aspect of the present disclosure includes a first acquisition unit that acquires concentration degree information indicating the concentration degree of a viewer when the viewer views content, a determination unit that determines a policy for modifying the content in order to improve the concentration degree indicated by the concentration degree information, and a provision unit that provides the viewer with the content modified in accordance with the policy.

[0007] An information processing method according to another aspect of the present disclosure includes steps of causing a computer to acquire concentration degree information indicating the concentration degree of a viewer when the viewer views content, determine a policy for modifying the content in order to improve the concentration degree indicated by the concentration degree information, and provide the viewer with the content modified in accordance with the policy.

[0008] According to the present invention, it is possible to provide improved content according to the degree of concentration.

[0009] A diagram illustrating the system configuration of an information processing system 1 according to one embodiment. A diagram illustrating the functional configuration of the information processing system 1. A diagram illustrating the hardware configuration of the information processing device 10. A sequence chart illustrating a method for providing modified content. A diagram illustrating a user information database 500. A diagram illustrating a concentration graph (part 1) 600. A diagram illustrating a policy database 700. A diagram illustrating a prompt. A diagram illustrating pre-modification content 900. A diagram illustrating modified content 1000. A diagram illustrating a concentration graph (part 2) 1100.

[0010] 1. Diagram 1 illustrates the system configuration of an information processing system 1 according to one embodiment. In this example, the information processing system 1 (or simply referred to as the system) is a system that modifies content displayed on a user terminal 30 based on the level of concentration of the viewer viewing the content, and provides the modified content to the user terminal 30. The content is, for example, a video. The video is, for example, a video used for workplace training.

[0011] Generally, watching videos presents the challenge of maintaining concentration. Even without any particular reason, human concentration levels decline over time. One way to recover lost concentration is for people to stop watching content and take a break, but break times are limited, and the speed and timing of concentration decline vary from person to person. Furthermore, as the number of times the same content is watched increases, people tend to become accustomed to watching the same content repeatedly, and their concentration tends to decline even more easily. However, it is possible that viewers' concentration levels can be restored by providing some kind of stimulus. The challenge has been to generate or process videos that enhance concentration according to each individual viewer. In this embodiment, content is modified using deepfake technology based on the viewer's level of concentration, and the modified content is provided to the viewer.

[0012] The information processing system 1 includes an information processing device 10, a user terminal 30, and a generating AI 40. In this example, the information processing device 10, the user terminal 30, and the generating AI 40 are connected via a network 20 as shown in Figure 1. In this example, the network 20 is a computer network such as the Internet or a mobile network.

[0013] The information processing device 10 is an information processing device or server device in the information processing system 1. The information processing device 10 stores content for streaming distribution to the user terminal 30 in advance. The information processing device 10 also stores the modified content output from the generation AI 40. In this example, the information processing device 10 obtains at least a portion of the viewer's attention level information and attribute information from the user terminal 30. Attention level information is information about the viewer's level of attention to the content. Attention level information may be the level of attention itself, or it may be information for determining the level of attention (for example, the number of views or viewing time). Attribute information is information about the attributes related to the viewer's viewing of the content. Based on the attention level information and attribute information, the information processing device 10 determines a policy for modifying the content. The information processing device 10 inputs a prompt for modifying the content, generated according to the determined policy, to the generation AI 40 that modifies the content. The information processing device 10 provides the modified content according to the prompt to the viewer (user terminal 30 in this example).

[0014] Generative AI 40 is, as an example, a video generation AI that utilizes deepfake technology. Generative AI 40 has already been trained on video data. Furthermore, Generative AI 40 has been trained on knowledge of, for example, patterns of human facial expressions, knowledge of image processing and generation, knowledge of music, and knowledge of video editing.

[0015] The user terminal 30 is a terminal used by a user (viewer in this example) in the information processing system 1. The user terminal 30 includes, for example, a smartphone, tablet, or personal computer. The user terminal 30 displays content provided by the information processing device 10. The user terminal 30 also outputs viewer concentration information and attribute information to the information processing device 10. The user terminal 30 also retrieves content modified by the information processing device 10.

[0016] Figure 2 is a diagram illustrating the functional configuration of the information processing system 1. In this embodiment, the information processing device 10 has functional blocks (or components) consisting of a first acquisition unit 11, a determination unit 12, a provision unit 13, a second acquisition unit 14, a third acquisition unit 15, a fourth acquisition unit 16, a control unit 17, a storage unit 18, and an input unit 19. The storage unit 18 stores various data, programs, and software, including, for example, a database. The storage unit 18 also stores various content to be provided to the user terminal 30. The control unit 17 performs various controls.

[0017] The first acquisition unit 11 acquires viewer concentration level information from the database when viewers view content. Concentration level information includes, for example, information about the viewer's viewing history or information about the viewer's actions while viewing. Information about the viewer's viewing history includes, for example, the number of times the content has been viewed, the duration of viewing the content, or the viewer's satisfaction level with past viewings of the content. Information about the viewer's actions while viewing includes, for example, the number of times their gaze averted from the content, the number of times they yawned, or the number of times they rubbed their eyes.

[0018] The second acquisition unit 14 acquires attribute information from the database that indicates attributes related to viewing the content. Attribute information includes, for example, the viewer's gender or age. The third acquisition unit 15 acquires effect information from the database that indicates the effect of the modification. Effect information includes, for example, information indicating the degree of improvement in concentration.

[0019] The decision unit 12 determines a policy to modify the content in order to improve the concentration level indicated by the concentration level information. The decision unit 12 also determines a policy to modify the content according to the attribute information. Furthermore, the decision unit 12 determines a policy to modify the content according to the effect information.

[0020] The input unit 19 generates prompts according to the policy and inputs them to the content-modifying AI 40. The fourth acquisition unit 16 acquires the modified content from the generation AI 40 according to the prompts. The provision unit 13 provides the modified content acquired from the generation AI 40 according to the policy to the viewer.

[0021] Figure 3 illustrates the hardware configuration of the information processing device 10. Physically, the information processing device 10 is configured as a computer including a processor 101, memory 102, storage 103, communication device 104, input device (optional), display device (optional), and a bus connecting these. Each of these devices operates on power supplied from a battery (not shown). In the following description, the term "device" can be read as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in Figure 3, or it may be configured without some of the devices. Alternatively, multiple devices with different enclosures may be connected via communication to constitute the information processing device 10.

[0022] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 101 and memory 102, which allows the processor 101 to perform calculations, control communication by the communication device 104, and control at least one of the reading and writing of data in the memory 102 and storage 103. The processor 101 also modifies content using an AI module for modifying content.

[0023] The processor 101 controls the entire computer, for example, by running the operating system. The processor 101 may be composed of a central processing unit (CPU) that includes interfaces with peripheral devices, control devices, arithmetic units, registers, etc. Also, for example, a baseband signal processing unit or a call processing unit may be implemented by the processor 101.

[0024] The processor 101 reads programs (program code), software modules, data, etc., from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described later. Functional blocks of the information processing device 10 may be stored in the memory 102 and implemented by control programs that run on the processor 101. Various processes may be executed by one processor 101, or they may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The program may also be transmitted to the information processing device 10 via a telecommunications line.

[0025] Memory 102 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may also be called a register, cache, main memory, etc. Memory 102 can store executable programs (program code), software modules, etc., for carrying out the method according to this embodiment. For example, memory 102 stores an AI module for modifying content.

[0026] The storage 103 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 103 may also be called an auxiliary storage device.

[0027] The communication device 104 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0028] Each device, such as the processor 101 and memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used for each device.

[0029] The information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by this hardware. For example, the processor 101 may be implemented using at least one of these hardware components.

[0030] In this example, the program stored in the storage 103 includes a program (hereinafter referred to as the "server program") that causes the computer to function as a server in the information processing system 1. When the processor 101 is executing the server program, the processor 101, memory 102, storage 103, and communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of a determination unit 12, a third acquisition unit 15, a fourth acquisition unit 16, a control unit 17, and an input unit 19. At least one of the memory 102 and the storage 103 is an example of a storage unit 18. The communication device 104 is an example of a first acquisition unit 11, a provision unit 13, and a second acquisition unit 14.

[0031] Although a detailed explanation will be omitted, the user terminal 30 is a computer having a processor, memory, storage, communication device, display device, input device, and output device, specifically, for example, a smartphone, tablet terminal, or personal computer. In this example, the program stored in the user terminal 30 includes a program that causes the computer to function as a client in the information processing system 1 (hereinafter referred to as the "client program"). The configuration of the information processing system 1 has now been described. Next, the operation of the information processing system 1 will be described.

[0032] 2. Operation diagram 4 is a sequence chart illustrating a method for providing modified content. The process in Figure 4 is initiated, for example, when a content viewing application is launched on the user terminal 30. In step S401, the user terminal 30 requests the information processing device 10 to provide the content. This request includes the user ID of the viewer who requested the content, and the content ID. The user ID is identification information for uniquely identifying the viewer, and will be described later in Figure 5. The content ID is identification information for uniquely identifying the content. The information processing device 10 receives the request for content provision from the user terminal 30.

[0033] In step S402, the information processing device 10 refers to the user information database 500 and obtains user information corresponding to the user ID of the viewer who requested the content. In this example, the user information is information about the viewer and is stored in the user information database 500 in advance.

[0034] Figure 5 illustrates a user information database 500. In this example, the user information database 500 is stored in the information processing device 10. The user information database 500 stores information about viewers who use the information processing system 1. The user information database 500 stores user information for each content ID. User information includes, as an example, user ID, attribute (gender), attribute (age), concentration level, and number of views. Attribute (gender) and attribute (age) are examples of attribute information, and concentration level is an example of concentration level information. The user ID is identification information for uniquely identifying a viewer and is predetermined for each viewer. Attribute (gender) is information about the viewer's gender. Attribute (gender) is stored in the user information database 500 when, for example, the viewer inputs "male" or "female" through an operation via the user terminal 30. Attribute (age) is information about the viewer's age. Attribute (age) is stored in the user information database 500 when, for example, the viewer inputs a number greater than or equal to 0 through an operation via the user terminal 30. In this example, the level of concentration represents the viewer's level of concentration. In this example, a decrease in the viewer's concentration corresponds to a decrease in the viewer's ability to concentrate. That is, the higher the viewer's concentration, the more attentively they are expected to be able to watch the content. In this example, the level of concentration stored in the user information database 500 is determined, for example, based on the number of times the content has been viewed. For example, as the number of times the content has been viewed increases, it is assumed that viewers will become bored with the content, and their level of concentration will decrease. More specifically, for example, for a given piece of content, the level of concentration of a viewer who has never watched that content before is 0, the level of concentration of a viewer who has watched that content once in the past is -1, the level of concentration of a viewer who has watched that content twice in the past is -2, and the level of concentration of a viewer who has watched that content three times in the past is -3. Therefore, in this example, the value stored in the level of concentration is updated each time the number of times the content has been viewed increases. The number of views is information about the number of times a viewer has watched the content, and for example, it is the number of times it has been watched. Therefore, in this example, the value stored in the number of views is updated each time the number of times the content has been viewed increases.Taking the user information database 500 as an example, it shows that a viewer with user ID "001" (hereinafter referred to as user 001) is male, 28 years old, has a level of engagement with this content of "-2", and has viewed it twice. Attribute information is updated, for example, when a viewer requests a change. Additionally, the viewer's date of birth may be stored as attribute information, and attribute (age) 503 may be automatically updated when the viewer's birthday arrives.

[0035] Let's return to the explanation of Figure 4. In step S403, the information processing device 10 detects that the content modification conditions have been met. The modification conditions are the conditions that trigger the modification of the content. For example, the modification conditions are that the content has not been modified even once since the content was requested, or that the level of attention of viewers watching the content has fallen below the modification threshold. The modification threshold is the level of attention that serves as the criterion for whether or not the information processing device 10 modifies the content.

[0036] Figure 6 is an example of a concentration graph (1) 600. The information processing device 10, as an example, stores data for the concentration graph (1) 600 in advance. The concentration graph (1) 600 is an example that shows how much a viewer's concentration decreases over time from the time they start watching the content. In the concentration graph (1) 600, the vertical axis is the user's concentration level, and the horizontal axis is the content viewing time. Generally, human concentration decreases over time. Also, when a viewer watches the same content repeatedly, the overall concentration level of the viewer tends to decrease as the number of viewings increases. As an example, the concentration graph (1) 600 shows that when a viewer watches a certain content for the first time, their concentration level is "0" at 0 minutes of viewing, and it decreases by 1 after 30 minutes from the start of viewing. Furthermore, as another example, the concentration graph (1) 600 shows that when a viewer is watching a certain content for the second time, the concentration level of a viewer who is "-1" at the start of viewing time will decrease by 1 after 20 minutes from the start of viewing. Also, as yet another example, the concentration graph (1) 600 shows that when a viewer is watching a certain content for the third time, the concentration level of a viewer who is "-2" at the start of viewing time will decrease by 1 after 15 minutes from the start of viewing. Although not shown in the illustration, the concentration graph (1) 600 also stores data on the change in concentration over time when the concentration level at the start of viewing is greater than 0, or when the concentration level at the start of viewing is less than -2.

[0037] For example, if the target content has not been modified even once since the content was requested, the viewer's attention level when the viewer's viewing time is 0 minutes is the attention level stored in the user information database 500. As another example, if the target content has already been modified at least once since the content was requested, the content viewing time is reset each time the content is modified. The modification threshold value in the most recent modification is then considered as the attention level at the start of viewing in the next modification, and a new modification threshold value and the time to reach that threshold value are established. For example, if the modification threshold value in the most recent modification is "-1", then the attention level at the start of viewing in the next modification is "-1", the corresponding modification threshold value is "-2", and the time to reach the modification threshold value is "20 minutes".

[0038] Let's return to the explanation of Figure 4. In step S404, the information processing device 10 determines a policy upon detecting that the modification conditions have been met. The policy is the policy for modifying the content that was requested to be provided in step S401. The policy is determined, for example, according to the policy database 700 shown in Figure 7.

[0039] Figure 7 illustrates a policy database 700. In this example, the policy database 700 is stored in the information processing device 10. The policy database 700 stores information regarding policies for modifying content. The policy database 700 includes, as an example, user information and character information. User information is information about the viewer. User information includes, as an example, attention level, attribute (gender), and attribute (age). The attention level in the policy database 700 is an example of attention level information, similar to the attention level in the user information database 500, and in this example, it is the attention level of the viewer. The attribute (gender) in the policy database 700 is information about the viewer's gender, similar to the attribute (gender) in the user information database 500. The attribute (gender) is, for example, "male" or "female". The attribute (age) in the policy database 700 is information about the viewer's age, similar to the attribute (age) in the user information database 500. The attribute (age) is, for example, "under 40" or "over 41".

[0040] The policy database 700 stores information about multiple elements that make up the content. Character information is an example of information about multiple elements that make up the content, and is information about characters included in the content. Characters are, for example, displayed together with the content and produce output through at least one of display and sound. More specifically, for example, characters display themselves, guide the flow and content of the content, explain the content of the content, or ask questions to the viewer. Character information includes, as an example, appearance (gender), appearance (expression), voice (volume), and voice (pitch). Appearance (gender) is information about the gender of the character. Appearance (gender) is information about the gender of the character, for example, male or female. Appearance (expression) is information about the expression of the character, for example, expressionless, smiling, or grinning. Voice (volume) is information about the volume of the character's voice. For example, voice (volume) is indicated by a number, and the larger the number, the louder the character's voice. Voice (pitch) is information about the pitch of the character's voice. For example, the voice (pitch) is represented by a number, and the higher the number, the higher the character's voice. The actual volume of the voice output, corresponding to the voice (volume) value, is stored in the information processing device 10 beforehand, for example. Similarly, the actual pitch of the voice output, corresponding to the voice (pitch) value, is also stored in the information processing device 10 beforehand, for example.

[0041] For example, character information corresponding to a viewer with a concentration level of "-1", an attribute (gender) of "female", and an age of 32 (i.e., an attribute (age) of "under 40") would be: appearance (gender) of "female", appearance (expression) of "smiling", voice (volume) of "16", and voice (pitch) of "2". Furthermore, "determining a policy" means, as an example, that the information processing device 10 extracts character information from the same record as the user information corresponding to the user who requested the content. In this example, the information extracted as a policy would be "appearance (gender) = female", "appearance (expression) = smiling", "voice (volume) = 16", and "voice (pitch) = 2".

[0042] Further, in this example, the degree of concentration used for policy determination varies depending on the modification condition. For example, in step S403, when the modification condition that the content has not been modified even once since the request for content provision is satisfied, the degree of concentration used for policy determination is the degree of concentration stored in the user information database 500. For example, when the viewer is user 001, the degree of concentration used for policy determination is "-2". Also, for example, in step S403, when the modification condition that the degree of concentration of the viewer viewing the content is below the modification reference value of "-3" is satisfied, the degree of concentration used for policy determination is the same value as the modification reference value. For example, when the viewer is user 001, the degree of concentration used for policy determination is "-3".

[0043] Returning again to the description of FIG. 4. In step S405, the information processing apparatus 10 generates a prompt to be input to the generative AI 40. In this example, the prompt is generated based on the policy determined in step S403.

[0044] FIG. 8 is a diagram illustrating an example of the prompt. As an example, the prompt includes an instruction and conditions. In this example, the instruction is "Please modify the content with the content ID 'aaa' to satisfy the following conditions.", and the conditions are "Appearance (Gender): Female", "Appearance (Facial expression): Smiling", "Voice (Volume): 16", and "Voice (Pitch): 2".

[0045] Returning again to the description of FIG. 4. In step S406, the information processing apparatus 10 inputs the prompt generated in step S404 to the generative AI 40. When the prompt is input to the generative AI 40, the original content stored in the storage unit 18 is modified. For example, for the original content (corresponding to FIG. 9 described later) in which the character's appearance (gender) is "female", the appearance (facial expression) is "smile", the voice (volume) is "14", the voice (pitch) is "2", and the content ID is "aaa", when a prompt (in this example, a prompt corresponding to FIG. 8) indicating that the character's appearance (gender) is "female", the appearance (facial expression) is "grin", the voice (volume) is "16", and the voice (pitch) is "2" is input to the generative AI 40, modified content (corresponding to FIG. 10 described later) is generated.

[0046] FIG. 9 is a diagram illustrating pre-modification content 900. The pre-modification content 900 is, as an example, an image obtained by extracting one scene of the content before being modified in step S406. The gender of the character is "female", and the facial expression of the character is "smile". Although not displayed in the pre-modification content 900, the volume of the character's voice is "14", and the pitch of the character's voice is "2".

[0047] FIG. 10 is a diagram illustrating post-modification content 1000. The post-modification content 1000 is, as an example, an image obtained by extracting one scene of the content after being modified in step S406. The gender of the character is "female", and the facial expression of the character is "grin". Although not displayed in the post-modification content 1000, the volume of the character's voice is "16", and the pitch of the character's voice is "2".

[0048] Returning again to the description of FIG. 4. In step S407, the information processing apparatus 10 acquires the modified content output from the generative AI 40. The information processing apparatus 10 stores the modified content.

[0049] In step S408, the information processing device 10 provides the modified content to the user terminal 30. The user terminal 30 obtains the modified content from the information processing device 10. Having obtained the modified content, the user terminal 30 plays the modified content. The process from steps S403 to S408 is, for example, repeatedly executed each time it is detected that the modification conditions have been met.

[0050] In this example, the content is streamed as a video from the information processing device 10 to the user terminal 30. For example, when the user terminal 30 acquires modified content, it switches from the originally playing content to the modified content based on the content time (corresponding to the video's time axis) to ensure that no content is lost. The content time is the time information inherent in the content itself and is different from the viewing time. For example, suppose the original content, which has not been modified at all, is played for 30 minutes, then the content generated by the first modification is played for 20 minutes, and then the content generated by the second modification is played for 15 minutes. In this case, the content time at the point when the original content, which has not been modified at all, has been played for 30 minutes is 30 minutes, the content time at the point when the content generated by the first modification is played for 20 minutes is 50 minutes, and the content time at the point when the content generated by the second modification is played for 15 minutes is 65 minutes. The user terminal 30 switches and plays the content so that the content time at the point when the playback of the originally playing content ends and the content time at the point when the playback of the next content to be played begins are the same.

[0051] Figure 11 illustrates the concentration graph (part 2) 1100. In the concentration graph (part 2) 1100, the vertical axis represents the user's concentration level, and the horizontal axis represents the content duration. The concentration graph (part 2) 1100 is an example showing the difference in how the viewer's concentration level changes depending on whether or not the content is modified, when watching 65 minutes of content from beginning to end. In the concentration graph (part 2) 1100, the vertical axis represents the user's concentration level, and the horizontal axis represents the content duration. When the content duration is 0 minutes, the concentration level is "0", and the content is delivered without any modifications. In this example, the concentration level changes as shown in the graph for modification 0. On the other hand, when the content duration is 0 minutes, the concentration level is "0", and the content is modified and delivered each time the concentration level reaches the modification threshold (i.e., each time the concentration level decreases by 1). In this example, when the content duration is between 0 minutes and 30 minutes, the concentration level changes as shown in the graph for modification 0. If the first modification occurs when the content time reaches 30 minutes, the concentration level will change as shown in the graph for the first modification when the content time is between 30 and 50 minutes. Similarly, if the second modification occurs when the content time reaches 50 minutes, the concentration level will change as shown in the graph for the second modification when the content time is between 50 and 65 minutes.

[0052] When viewers repeatedly watch the same content, their concentration may decrease with each viewing. Furthermore, even for first-time viewers, concentration may decline as viewing time increases. Modifying the content, such as altering at least one of the character's appearance and / or voice, to match the viewer's level of concentration is expected to restore or mitigate the decline in concentration.

[0053] 3. Modifications The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the matters described below may be combined and applied.

[0054] (1) Information Processing System 1 The hardware configuration and network configuration of the information processing system 1 are not limited to those illustrated in the embodiments. The information processing system 1 may have any hardware configuration and network configuration as long as it can realize the required functions. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, at least a part of the functions of the information processing device 10 may be implemented on a server (not shown) separate from the information processing device 10. For example, that separate server may have a function related to the information processing device 10 that determines the policy for modifying content.

[0055] (2) Information Processing Device 10 Some of the functions of the information processing device 10 may be implemented on other servers. These servers may be, for example, physical servers or virtual servers (including so-called clouds). Furthermore, the correspondence between functional elements and hardware is not limited to those illustrated in the embodiments. For example, at least some of the functions described in the embodiments as being implemented on the information processing device 10 may be implemented on other devices or systems, or conversely, at least some of the functions described as being implemented on other devices or systems may be implemented on the information processing device 10. In this example, at least some of the information processing device 10 may be implemented on the user terminal 30.

[0056] (3) User terminal The user terminal 30 is not limited to those illustrated in the embodiment. The user terminal 30 may perform the above-described processing using any display screen, input device, external device, or various UI.

[0057] (4) User information database 500 The user information database 500 is not limited to being created for each piece of content, but may be created in common for multiple pieces of content.

[0058] The attribute (gender) is not limited to "male" or "female," but may also be "unanswered" or "other."

[0059] The attribute (age) is not limited to a specific age (e.g., 28 years old), but may be an age range (e.g., 25 years old or older and under 30 years old).

[0060] The attribute information contained in the user information database 500 is not limited to attributes (gender) and attributes (age), but may include any information that indicates attributes related to viewing the content. For example, attribute information may include race, preferences, address, or occupation. More specifically, if the attribute information includes preferences, and those preferences are related to a celebrity or anime character that the viewer likes, the content may be modified based on that celebrity or anime character.

[0061] (5) Policy Database 700 The attribute (gender) is not limited to "male" or "female," but may also be "unanswered" or "other."

[0062] Instead of physical appearance (gender), information about the appearance of a specific person, such as an anime character or a celebrity, may be stored. In that case, a photograph of that person may be stored, linked to the information about their appearance. The content may be generated using a character created based on that photograph.

[0063] Character information is not limited to information about the character's gender, facial expressions, voice volume, and voice pitch; any information that describes the character's characteristics is acceptable. This information may include, for example, information about speaking speed or intonation.

[0064] The policy database 700 may be created for each piece of content, or it may be created for multiple pieces of content.

[0065] (6) Information on concentration level and attribute information The information used to determine the level of concentration is not limited to the number of times the content has been viewed, but may be any information that can contribute to determining the level of concentration. For example, the information used to determine the level of concentration may be the likelihood of falling asleep, the frequency of smartphone operation while viewing, information on the viewing time of the content, information showing the results of a survey conducted when the content was viewed in the past, or information on the viewer's facial expression while viewing. As an example, the level of concentration in the user information database 500 may be changed based on the results of a survey conducted after viewing the content. For example, if the survey result is "interesting," 1 may be added to the level of concentration based on the number of views (for example, if the number of views is 2, the level of concentration is -2) (i.e., the final level of concentration is -1). Also, for example, if the survey result is "average," neither addition nor subtraction may be made to the level of concentration based on the number of views. Also, for example, if the survey result is "boring," 1 may be subtracted from the level of concentration based on the number of views (for example, if the number of views is 2, the level of concentration is -2) (i.e., the final level of concentration is -3).

[0066] The concentration level information and attribute information are not limited to being provided to the information processing device 10 from the user terminal 30 or generated in the information processing device 10; they may also be directly input to the information processing device or provided to the information processing device 10 from other terminals.

[0067] Content modification is not limited to being carried out based on the concentration level in the user information database 500 or a policy based on modification criteria. For example, the information processing device 10 may measure the viewer's concentration level in real time. More specifically, the front camera of the user terminal 30 playing the content may photograph the viewer, and the information processing device 10 may measure the concentration level using a predetermined algorithm from the viewer's facial expressions or eye movements included in the captured video. The information processing device 10 may then modify the content as appropriate based on the concentration level measured in real time.

[0068] (7) The policy for modifying the effectiveness information content may be determined in accordance with the effectiveness information that demonstrates the effect of the modification. Effective information includes, for example, information that shows the reactions or actions of viewers. More specifically, effective information includes, for example, information that shows changes in viewers' facial expressions, changes in the amount of time viewers spend looking at the screen, or the results of viewer responses to satisfaction surveys.

[0069] (8) Content The video, which is an example of content, is not limited to videos used for workplace training, etc., but may also be, for example, online education, e-learning, video advertisements, or entertainment content (for example, movies, dramas, or games).

[0070] The content is not limited to video; it may also be images, text, or audio. For example, if the content is text, the appearance of the text (color, size, font, etc.) may be changed depending on the level of attention, the presence or absence of summaries may be changed, or the summaries themselves may be changed. Also, for example, if the content is audio, the speed of the audio may be changed, or the intonation of the audio may be changed depending on the level of attention. Even if the content is video, if the video has text or audio, the text or audio may be modified as described above.

[0071] (9) In the example of content modification, as an example of content modification, elements related to characters were changed, but other elements may also be changed. For example, if the content is a video, the color tone of the video, the brightness of the video, the video effects, the type of music, or the speed of the music may be changed. Only one of the elements related to characters and / or other elements may be changed, or both may be changed. Even if the content does not contain characters, elements other than those related to characters may be changed as described above.

[0072] The content is not necessarily limited to being modified. For example, if a user terminal 30 requests the provision of content that a viewer is viewing for the first time, that content may be provided to the user terminal 30 in an unmodified state.

[0073] The content used to modify the content during viewing is not limited to the original content pre-stored in the memory unit 18, but may also be, for example, the content modified in step S408.

[0074] The modification threshold value in the most recent modification is not necessarily considered the level of concentration at the start of viewing in the next modification. It is also possible that concentration may recover as a result of content modification and content switching. The level of concentration in the next modification may be a higher value than the modification threshold value in the most recent modification. For example, if the modification threshold value in the most recent modification is "-2", the level of concentration at the start of viewing in the next modification may be "-1.5".

[0075] The method for calculating the concentration level or modification criterion value exemplified is merely one example, and the method for calculating the concentration level or modification criterion value may be flexibly changed by the administrator of the information processing system 1 or the like.

[0076] (10) Timing of content modification Content modification is not limited to being made before viewing the content or during viewing it, but may be made at either of these times.

[0077] The conditions for modification are not limited to the fact that the content has not been modified even once since the content was requested, or that the viewer's level of concentration while viewing the content has fallen below the modification threshold, but rather any time when it is assumed that the viewer's level of concentration has decreased. For example, while a viewer is viewing the content, the content may be modified when the viewer is photographed by the user terminal 30 used for viewing, and a specific facial expression or action of the viewer is detected. A specific facial expression or action of the viewer may be, for example, looking away from the screen, frequent blinking, yawning, resting the chin on the hand, or dozing off.

[0078] The modification threshold is not limited to being 1 lower than the concentration level when viewing time is 0 minutes. For example, the modification threshold could be a specific value such as "-4", regardless of the concentration level when viewing time is 0 minutes.

[0079] The modification threshold is not limited to concentration level; for example, it could be viewing time. Specifically, if the modification threshold is 30 minutes, the content may be modified 30 minutes after viewing begins.

[0080] (11) Prompts are not limited to being generated based on the policy database 700, but may also be, for example, the information contained in the policy database 700 itself.

[0081] The input prompt is not limited to text. For example, the input prompt may be an image, audio, video, 3D model, parameter, or music, or a combination of these. For instance, if the prompt is an image, instructions for changing the facial expressions of the content's characters may be more detailed than if the prompt were text. Also, if the prompt is music, the AI ​​may analyze the mood of the music and modify the content to match that mood.

[0082] (12) Other programs executed by the processor 101 may be provided by downloading them via a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0083] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining software with the one or more of the above devices.

[0084] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmission unit or transmitter. In all cases, as mentioned above, the method of implementation is not particularly limited.

[0085] For example, the information processing device 10 in one embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0086] Each aspect or embodiment described in this disclosure may be applied to at least one of the following: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA®, GSM®, CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi®), IEEE 802.16 (WiMAX®), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth®, and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0087] The processing procedures, sequences, flowcharts, etc., of each aspect or embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements in an exemplary order and are not limited to the specific order presented.

[0088] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0089] The determination may be made by a value represented by one bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0090] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0091] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name. Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0092] The information, signals, etc., described herein may be represented using any of the following different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Terms used herein and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.

[0093] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0094] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0095] Any reference to elements using the designations “First,” “Second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the First and Second elements do not imply that only two elements may be employed, or that the First element must precede the Second element in any way.

[0096] In the above-described configuration of each device, the term "part" may be replaced with "means," "circuit," "device," etc.

[0097] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0098] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0099] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0100] 1... Information processing system, 10... Information processing device, 11... First acquisition unit, 12... Decision unit, 13... Provision unit, 14... Second acquisition unit, 15... Third acquisition unit, 16... Fourth acquisition unit, 17... Control unit, 18... Storage unit, 19... Input unit, 20... Network, 30... User terminal, 40... Generation AI, 101... Processor, 102... Memory, 103... Storage, 104... Communication device, 500... User information database, 600... Concentration graph (part 1), 700... Policy database, 900... Content before modification, 1000... Content after modification, 1100... Concentration graph (part 2)

Claims

1. An information processing device comprising: a first acquisition unit that acquires concentration level information indicating the level of concentration of a viewer when viewing content; a decision unit that determines a policy for modifying the content in order to improve the level of concentration indicated by the concentration level information; and a provision unit that provides the viewer with the content modified in accordance with the policy.

2. The information processing apparatus according to claim 1, further comprising a second acquisition unit that acquires attribute information indicating the attributes of the viewer relating to the viewing of the content, wherein the determination unit determines the policy according to the attribute information.

3. The information processing apparatus according to claim 2, wherein the content includes a video, and the determination unit determines the timing of the modification in the time axis of the video according to the attribute information.

4. The information processing apparatus according to claim 2, wherein the attribute information includes the viewing history of the viewer viewing the content, and the decision unit determines the policy according to the viewing history.

5. The information processing apparatus according to claim 2, wherein the attribute information includes preference information indicating the preferences of the viewer, and the decision unit determines the policy according to the preference information.

6. The information processing apparatus according to claim 1, further comprising a third acquisition unit for acquiring effect information indicating the effect of the modification, wherein the determination unit determines the policy according to the effect information.

7. The information processing apparatus according to claim 1, wherein the content comprises a plurality of elements, the policy includes identifying the element to be modified among the plurality of elements, and the providing unit provides the content, in which the identified element has been modified, to the viewer.

8. The information processing apparatus according to claim 1, comprising: an input unit that inputs a prompt generated in accordance with the above policy to an AI that modifies the content, the prompt to be modified, to an AI; and a fourth acquisition unit that acquires the content modified in accordance with the prompt from the AI, wherein the provision unit provides the content acquired from the AI ​​to the viewer.

9. The information processing apparatus according to claim 1, wherein the content includes a video, the concentration level information includes the real-time concentration level of the viewer watching the content, and the determination unit determines the timing of the modification in the time axis of the video according to the real-time concentration level.

10. An information processing method comprising the steps of: obtaining concentration level information indicating the level of attention a viewer has while viewing content; determining a policy to modify the content in order to improve the level of attention indicated by the concentration level information; and providing the viewer with the content modified in accordance with the policy.

Images