Moving image analysis system

Abstract

[Problem] To objectively evaluate online communications, which have become mainstream, in order to carry out communication more efficiently. [Solution] A system according to the present disclosure is a video analysis system comprising: a video acquisition unit that acquires video obtained by imaging a consumer during an online session; an analysis unit that analyzes changes in biological reactions of the consumer on the basis of the video acquired by the video acquisition unit; a storage unit that stores a consumer database in which changes in the biological reactions of consumers are analyzed; a keyword reception unit that receives the input of a keyword from a new consumer; and an extraction unit that, on the basis of the keyword, extracts frames from the consumer database for which a positive biological response pertaining to the keyword is exhibited.

Description

Technical Field

[0001] The present invention relates to a moving image analysis system that analyzes the reactions of participants based on moving images obtained by photographing the participants in an environment where an online session is held by a plurality of participants, regardless of whether the participants are displayed on the screen during the online session.

Background Art

[0002] Techniques for analyzing the emotions received by others in response to the speech of a speaker are known (see, for example, Patent Document 1). In addition, techniques for analyzing the changes in the facial expressions of a subject over a long period of time in a time series and estimating the emotions held during that period are also known (see, for example, Patent Document 2). Furthermore, techniques for identifying the factors that most influenced the change in emotions are also known (see, for example, Patent Documents 3 to 5). Still further, techniques for comparing the normal facial expression of a subject with the current facial expression and issuing an alert when the facial expression is gloomy are also known (see, for example, Patent Document 6). In addition, techniques for comparing the facial expression of a subject in a normal state (when expressionless) with the current facial expression and determining the degree of the subject's emotion have also been known (see, for example, Patent Documents 7 to 9). Furthermore, techniques for analyzing the emotions of an organization and the atmosphere within a group felt by an individual are also known (see, for example, Patent Documents 10 and 11).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

Patent Document 5

Patent Document 6

[0004] All of the above-mentioned technologies are merely secondary functions in a situation where communication in the real space is mainly involved. That is, they were not born in a situation where communication such as business and classes is mainly conducted online due to the recent DX (Digital Transformation) of business and the worldwide spread of infectious diseases.

[0005] An object of the present invention is to objectively evaluate these communications in order to perform more efficient communication in a situation where online communication such as meetings and lectures is the main form. [Means for Solving the Problems]

[0006] According to the present invention, a moving image acquisition unit that acquires a moving image obtained by photographing a purchaser user during an online session; an analysis unit that analyzes changes in the biological reaction of the purchaser user based on the moving image acquired by the moving image acquisition unit; a storage unit that stores a purchaser database in which changes in the biological reaction of the purchaser user are analyzed; a keyword reception unit that receives an input of a keyword from a new purchaser user; An extraction unit that extracts, from the purchaser database, a frame showing a positive biological reaction with respect to the keyword based on the keyword; comprising a moving image analysis system is obtained.

Advantages of the Invention

[0007] According to the present disclosure, by analyzing and evaluating a moving image of a video session, it is possible to objectively evaluate, in particular, an evaluation regarding the content.

[0008] In particular, according to the present invention, in a situation mainly involving online communication, it is possible to objectively evaluate the communicated communication in order to perform more efficient communication.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Embodiments for Carrying Out the Invention

[0010] The content of the embodiment of the present disclosure will be listed and described. The present disclosure has the following configuration. [Item 1] A moving image acquisition unit that acquires a moving image obtained by photographing a purchaser user during an online session, An analysis unit that analyzes changes in the biological reaction of the purchaser user based on the moving image acquired by the moving image acquisition unit, A storage unit that stores a purchaser database in which changes in the biological reaction of the purchaser user are analyzed, A keyword reception unit that receives an input of a keyword from a new purchaser user, An extraction unit that extracts a frame showing a positive biological reaction regarding the keyword from the purchaser database based on the keyword, Comprising Moving image analysis system. [Item 2] A moving image analysis apparatus having the configuration of the moving image analysis system according to Item 1. [Item 3] A moving image analysis program that causes a moving image analysis apparatus to function with the configuration of the moving image analysis system according to Item 1. [Item 4] A moving image analysis method that executes the configuration of the moving image analysis system according to Item 1 as steps.

[0011] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0012] <Basic Function> The video session evaluation system of this embodiment is a system that analyzes and evaluates specific emotions (feelings that occur in response to one's own or others' words and actions, such as pleasure, discomfort, or the degree thereof) that are different from those of others for the person to be analyzed among a plurality of people in an environment where a video session (hereinafter referred to as an online session, including one-way and two-way) is conducted by multiple people. The online session is, for example, an online meeting, an online class, an online chat, etc., in which terminals installed in multiple locations are connected to a server via a communication network such as the Internet, and moving images can be exchanged between the multiple terminals through the server. The moving images handled in the online session include the face images and voices of the users using the terminals. The moving images also include images such as materials shared and viewed by multiple users. It is possible to switch between the face image and the material image on the screen of each terminal and display only one of them, or to divide the display area and display the face image and the material image simultaneously. It is also possible to display the image of one person among multiple people in full screen, or to divide and display the images of some or all of the users in small screens. Among the multiple users participating in the online session using the terminals, it is possible to specify one or more of them as the person to be analyzed. For example, the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the organizer) specifies one of the users as the person to be analyzed. The organizer of the online session is, for example, a teacher in an online class, a chairperson or facilitator in an online meeting, a coach in a session for coaching purposes, etc. The organizer of the online session is usually one of the multiple users participating in the online session, but may also be someone else who does not participate in the online session. Note that it is also possible not to specify the person to be analyzed and analyze all the participants as the objects. Also, the leader, facilitator, or administrator of the online session (hereinafter collectively referred to as the organizer) can specify one of the users as the person to be analyzed. The organizer of the online session is, for example, a teacher in an online class, a chairperson or facilitator in an online meeting, a coach in a session for coaching purposes, etc.The organizer of an online session is usually one of the multiple users participating in the online session, but it may also be someone else who does not participate in the online session.

[0013] When a video session is established among multiple terminals, the video session evaluation system according to this embodiment displays at least a moving image acquired from the video session. The displayed moving image is acquired by the terminal, and at least a face image included in the moving image is identified for each predetermined frame unit. Then, an evaluation value regarding the identified face image is calculated. The evaluation value is shared as needed. In particular, in this embodiment, the acquired moving image is stored in the terminal, analyzed and evaluated on the terminal, and the result is provided to the user of the terminal. Therefore, for example, even for a video session including personal information or a video session including confidential information, it can be analyzed and evaluated without providing the video itself to an external evaluation institution or the like. Also, if necessary, by providing only the evaluation result (evaluation value) to an external terminal, the result can be visualized or cross-analyzed.

[0014] As shown in FIG. 1, the video session evaluation system according to this embodiment includes user terminals 10 and 20 having at least input units such as a camera unit and a microphone unit, a display unit such as a display, and an output unit such as a speaker, a video session service terminal 30 that provides a two-way video session to the user terminals 10 and 20, and an evaluation terminal 40 that performs part of the evaluation regarding the video session.

[0015] <Hardware Configuration Example> Each functional block, functional unit, and functional module described below can be configured by, for example, any of hardware, DSP (Digital Signal Processor), and software provided in a computer. For example, when configured by software, it is actually configured with a computer's CPU, RAM, ROM, etc., and is realized by the operation of a program stored in a recording medium such as RAM, ROM, hard disk, or semiconductor memory. A series of processes by the system and terminal described in this specification may be realized using any of software, hardware, and a combination of software and hardware. It is possible to create a computer program for realizing each function of the information sharing support device 10 according to this embodiment and install it on a PC or the like. Further, it is also possible to provide a computer-readable recording medium storing such a computer program. The recording medium is, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

[0016] The evaluation terminal according to this embodiment acquires a moving image from a video session service terminal, identifies at least a face image included in the moving image for each predetermined frame unit, and calculates an evaluation value related to the face image (details will be described later).

[0017] <Method for acquiring video> As shown in FIG. 3, the video session service provided by the video session service terminal (hereinafter sometimes simply referred to as "this service") enables two-way communication by image and voice with user terminals 10 and 20. This service can display a moving image acquired by the camera unit of the other user terminal on the display of the user terminal, and output the voice acquired by the microphone unit of the other user terminal from the speaker. Also, this service is configured to be able to record (record) moving images and voices (collectively referred to as "moving images, etc.") in the storage unit on at least one of the user terminals by either or both of the user terminals. The recorded moving image information Vs (hereinafter referred to as "recording information") is cached in the user terminal that started the recording and is recorded only locally on one of the user terminals. The user can, if necessary, view the recording information by himself / herself within the scope of using this service, share it with others, etc.

[0018] <Functional Configuration Example 1> FIG. 4 is a block diagram showing a configuration example according to this embodiment. As shown in FIG. 4, the video session evaluation system of this embodiment is realized as a functional configuration of user terminal 10. That is, user terminal 10 includes, as its functions, a moving image acquisition unit 11, a biological reaction analysis unit 12, a specific determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16.

[0019] The moving image acquisition unit 11 acquires moving images obtained by photographing a plurality of people (a plurality of users) with the cameras provided in each terminal during an online session from each terminal. Whether the moving images acquired from each terminal are set to be displayed on the screen of each terminal or not is not a concern. That is, the moving image acquisition unit 11 acquires moving images from each terminal, including the moving images being displayed and not being displayed on each terminal.

[0020] Based on the moving image acquired by the moving image acquisition unit 11 (regardless of whether it is being displayed on the screen), the biological reaction analysis unit 12 analyzes the changes in the biological reactions of each of a plurality of people. In the present embodiment, the biological reaction analysis unit 12 separates the moving image acquired by the moving image acquisition unit 11 into a set of images (a collection of frame images) and audio, and analyzes the changes in the biological reactions from each of them.

[0021] For example, the biological reaction analysis unit 12 analyzes the facial image of the user by using the frame images separated from the moving image acquired by the moving image acquisition unit 11, and thereby analyzes the changes in the biological reaction related to at least one of the expression, eye line, pulse, and facial movement. In addition, the biological reaction analysis unit 12 analyzes the audio separated from the moving image acquired by the moving image acquisition unit 11, and thereby analyzes the changes in the biological reaction related to at least one of the speech content and voice quality of the user.

[0022] When a person's emotion changes, it appears as changes in biological reactions such as expression, eye line, pulse, facial movement, speech content, and voice quality. In the present embodiment, by analyzing the changes in the biological reactions of the user, the changes in the user's emotion are analyzed. The emotion analyzed in the present embodiment is, as an example, the degree of pleasure / displeasure. In the present embodiment, the biological reaction analysis unit 12 calculates a biological reaction index value that reflects the content of the change in the biological reaction by quantifying the change in the biological reaction according to a predetermined standard.

[0023] The analysis of the change in expression is performed, for example, as follows. That is, for each frame image, the facial region is specified from the frame image, and the facial expression specified according to the image analysis model that has been machine-learned in advance is classified into a plurality of types. Then, based on the classification result, it is analyzed whether a positive expression change has occurred between consecutive frame images, whether a negative expression change has occurred, and the magnitude of the expression change that has occurred, and an expression change index value corresponding to the analysis result is output.

[0024] Analysis of the change in the line of sight is performed as follows, for example. That is, for each frame image, the area of the eyes is identified from within the frame image, and by analyzing the directions of both eyes, it is analyzed where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, the shared material being displayed, outside the screen, etc. Also, it may be analyzed whether the movement of the line of sight is large or small, and whether the frequency of movement is high or low. The change in the line of sight is also related to the user's concentration. The biological reaction analysis unit 12 outputs a line-of-sight change index value according to the analysis result of the change in the line of sight.

[0025] Analysis of the change in the pulse is performed as follows, for example. That is, for each frame image, the area of the face is identified from within the frame image. Then, using a learned image analysis model that captures the numerical value of the facial color information (G of RGB), the change in the G color on the face surface is analyzed. By arranging the results along the time axis, a waveform representing the change in the color information is formed, and the pulse is identified from this waveform. A person's pulse becomes faster when they are nervous and slower when they are calm. The biological reaction analysis unit 12 outputs a pulse change index value according to the analysis result of the change in the pulse.

[0026] Analysis of the change in the movement of the face is performed as follows, for example. That is, for each frame image, the area of the face is identified from within the frame image, and by analyzing the direction of the face, it is analyzed where the user is looking. For example, it is analyzed whether the user is looking at the face of the speaker being displayed, the shared material being displayed, outside the screen, etc. Also, it may be analyzed whether the movement of the face is large or small, and whether the frequency of movement is high or low. The movement of the face and the movement of the line of sight may be analyzed together. For example, it may be analyzed whether the user is looking straight at the face of the speaker being displayed, looking with an upward or downward glance, or looking obliquely. The biological reaction analysis unit 12 outputs a face orientation change index value according to the analysis result of the change in the face orientation.

[0027] The analysis of the speech content is performed as follows, for example. That is, the biological reaction analysis unit 12 converts the speech into a character string by performing known speech recognition processing on the speech for a specified time (for example, a time of about 30 to 150 seconds), and removes words that are unnecessary for representing conversations such as particles and articles by performing morphological analysis on the character string. Then, the remaining words are vectorized, and it is analyzed whether a positive emotional change is occurring, whether a negative emotional change is occurring, and the magnitude of the emotional change that is occurring, and a speech content index value corresponding to the analysis result is output.

[0028] The analysis of the voice quality is performed as follows, for example. That is, the biological reaction analysis unit 12 identifies the acoustic characteristics of the voice by performing known voice analysis processing on the voice for a specified time (for example, a time of about 30 to 150 seconds). Then, based on the acoustic characteristics, it is analyzed whether a positive voice quality change is occurring, whether a negative voice quality change is occurring, and the magnitude of the voice quality change that is occurring, and a voice quality change index value corresponding to the analysis result is output.

[0029] The biological reaction analysis unit 12 calculates a biological reaction index value using at least one of the facial expression change index value, eye line change index value, pulse rate change index value, face orientation change index value, speech content index value, and voice quality change index value calculated as described above. For example, the biological reaction index value is calculated by performing weighted calculation on the facial expression change index value, eye line change index value, pulse rate change index value, face orientation change index value, speech content index value, and voice quality change index value.

[0030] The specific determination unit 13 determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to the change in the biological reaction analyzed for other persons other than the analysis target person. In the present embodiment, the specific determination unit 13 determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to others based on the biological reaction index values calculated for each of a plurality of users by the biological reaction analysis unit 12.

[0031] For example, the specificity determination unit 13 calculates the variance of the biological reaction index values calculated for each of a plurality of persons by the biological reaction analysis unit 12, and determines whether the change in the biological reaction analyzed for the analysis target person is specific compared to others by comparing the biological reaction index value calculated for the analysis target person with the variance.

[0032] As cases where the change in the biological reaction analyzed for the analysis target person is specific compared to others, the following three patterns can be considered. The first is the case where no particularly large change in the biological reaction has occurred for others, but a relatively large change in the biological reaction has occurred for the analysis target person. The second is the case where no particularly large change in the biological reaction has occurred for the analysis target person, but a relatively large change in the biological reaction has occurred for others. The third is the case where a relatively large change in the biological reaction has occurred for both the analysis target person and others, but the content of the change is different between the analysis target person and others.

[0033] The related event identification unit 14 identifies an event occurring with respect to at least one of the analysis target person, others, and the environment when a change in the biological reaction determined to be specific by the specificity determination unit 13 occurs. For example, the related event identification unit 14 identifies the speech and actions of the analysis target person himself / herself from a moving image when a specific change in the biological reaction occurs for the analysis target person. In addition, the related event identification unit 14 identifies the speech and actions of others from a moving image when a specific change in the biological reaction occurs for the analysis target person. In addition, the related event identification unit 14 identifies the environment from a moving image when a specific change in the biological reaction occurs for the analysis target person. The environment is, for example, shared materials being displayed on the screen, things reflected in the background of the analysis target person, and the like.

[0034] The clustering unit 15 analyzes the degree of correlation between the change in the biological reaction (for example, one or a combination of one or more of eye line, pulse, facial movement, speech content, and voice quality) determined to be specific by the abnormality determination unit 13 and the event (the event specified by the related event identification unit 14) that occurred when the specific change in the biological reaction occurred. When it is determined that the correlation is equal to or higher than a certain level, the analysis target person or event is clustered based on the analysis result of the correlation.

[0035] For example, when the change in the specific biological reaction corresponds to a negative emotional change and the event that occurred when the specific change in the biological reaction occurred is also a negative event, a correlation equal to or higher than a certain level is detected. The clustering unit 15 clusters the analysis target person or event into one of a plurality of pre-segmented classifications according to the content of the event, the degree of negativity, the magnitude of the correlation, and the like.

[0036] Similarly, when the change in the specific biological reaction corresponds to a positive emotional change and the event that occurred when the specific change in the biological reaction occurred is also a positive event, a correlation equal to or higher than a certain level is detected. The clustering unit 15 clusters the analysis target person or event into one of a plurality of pre-segmented classifications according to the content of the event, the degree of positivity, the magnitude of the correlation, and the like.

[0037] The analysis result notification unit 16 notifies at least one of the change in the biological reaction determined to be specific by the abnormality determination unit 13, the event specified by the related event identification unit 14, and the classification clustered by the clustering unit 15 to the designated person of the analysis target person (the analysis target person or the organizer of the online session).

[0038] For example, when a specific change in the bioreaction that is different from others occurs in the subject to be analyzed (any of the three patterns described above; the same applies hereinafter), the analysis result notification unit 16 notifies the subject to be analyzed of his or her own words and deeds as the events occurring at that time. Thereby, the subject to be analyzed can grasp that he or she has different feelings from others when performing a certain action. At this time, the specific change in the bioreaction identified for the subject to be analyzed may also be notified to the subject to be analyzed. Furthermore, the change in the bioreaction of the other person to be compared may also be notified to the subject to be analyzed.

[0039] For example, when there is a difference between the feelings received by others for the words and deeds performed by the subject to be analyzed without particular awareness with normal feelings or the words and deeds performed with particular awareness with a certain feeling, and the feelings held by the subject to be analyzed himself or herself during the words and deeds, the words and deeds of the subject to be analyzed himself or herself at that time are notified to the subject to be analyzed. Thereby, it is also possible to discover words and deeds that are well-received by others or words and deeds that are not well-received by others against one's own consciousness.

[0040] In addition, when a specific change in the bioreaction that is different from others occurs in the subject to be analyzed, the analysis result notification unit 16 notifies the organizer of the online session of the events occurring at that time together with the specific change in the bioreaction. Thereby, the organizer of the online session can know what events are affecting what changes in feelings as specific phenomena unique to the designated subject to be analyzed. And it becomes possible to take appropriate measures for the subject to be analyzed according to the grasped content.

[0041] In addition, when a specific change in the biological reaction different from others occurs in the person to be analyzed, the analysis result notification unit 16 notifies the event occurring at that time or the clustering result of the person to be analyzed to the organizer of the online session. As a result, the organizer of the online session can grasp the tendency of the behavior peculiar to the person to be analyzed according to the classification in which the specified person to be analyzed is clustered, and can predict the possible future behavior and state, etc. And it becomes possible to take appropriate measures against the person to be analyzed.

[0042] In the above embodiment, the biological reaction index value is calculated by quantifying the change in the biological reaction according to a predetermined standard, and based on the biological reaction index values calculated for each of a plurality of people, it is determined whether the change in the biological reaction analyzed for the person to be analyzed is specific compared to others. Although an example has been described, it is not limited to this example. For example, the following may be done.

[0043] That is, the biological reaction analysis unit 12 analyzes the movement of the line of sight for each of a plurality of people and generates a heat map indicating the direction of the line of sight. The specific determination unit 13 determines whether the change in the biological reaction analyzed for the person to be analyzed is specific compared to the change in the biological reaction analyzed for others by comparing the heat map generated for the person to be analyzed by the biological reaction analysis unit 12 with the heat map generated for others.

[0044] In this way, in the present embodiment, the moving image of the video session is stored in the local storage of the user terminal 10, and the above-described analysis is performed on the user terminal 10. Although there is a possibility of depending on the machine specifications of the user terminal 10, it is possible to perform the analysis without providing the information of the moving image to the outside.

[0045] <Functional configuration example 2> As shown in FIG. 5, the video session evaluation system of the present embodiment may include a moving image acquisition unit 11, a biological reaction analysis unit 12, and a reaction information presentation unit 13a as functional configurations.

[0046] The reaction information presenting unit 13a presents information indicating changes in the biological reaction analyzed by the biological reaction analysis unit 12a, including participants not displayed on the screen. For example, the reaction information presenting unit 13a presents information indicating changes in the biological reaction to the leader, progressor, or administrator of the online session (hereinafter collectively referred to as the host). The host of the online session is, for example, a lecturer of an online class, a chairperson or facilitator of an online meeting, a coach of a session for coaching purposes, etc. The host of the online session is usually one of the multiple users participating in the online session, but may also be someone else who does not participate in the online session.

[0047] By doing so, the host of the online session can also grasp the state of participants not displayed on the screen in an environment where the online session is held by multiple people.

[0048] <Functional Configuration Example 3> FIG. 6 is a block diagram showing a configuration example according to the present embodiment. As shown in FIG. 6, for the video session evaluation system of the present embodiment, for functions similar to those in the above-described Embodiment 1 in terms of functional configuration, the same reference numerals may be used and the description may be omitted.

[0049] The system according to the present embodiment includes a camera unit that acquires video of the video session and a microphone unit that acquires audio, an analysis unit that analyzes and evaluates the moving image, an object generation unit that generates a display object (described later) based on information obtained by evaluating the acquired moving image, and a display unit that displays both the moving image of the video session and the display object during the execution of the video session.

[0050] The analysis unit includes a moving image acquisition unit 11, a biological reaction analysis unit 12, a specific determination unit 13, a related event identification unit 14, a clustering unit 15, and an analysis result notification unit 16, as described above. The functions of each element are as described above.

[0051] As shown in FIG. 7, based on the result of analyzing the moving image acquired from the video session by the analysis unit, the object generation unit superimposes and displays, as necessary, an object 50 indicating the recognized face portion and information 100 indicating the above-described analyzed and evaluated content on the moving image. When there are multiple people's faces moving within the moving image, the object 50 may identify and display the faces of all multiple people.

[0052] Also, for example, when the camera function of the video session is stopped on the other party's terminal (that is, not physically covering the camera, but software-wise stopped within the video session application), and the other party's face was recognized by the other party's camera, the object 50 or the object 100 may be displayed at the portion where the other party's face is located. Thereby, even if the camera function is turned off, it becomes possible for both parties to confirm that the other party is in front of the terminal. In this case, for example, in the video session application, while making the information acquired from the camera non-displayed, only the object 50 and the object 100 corresponding to the face recognized by the analysis unit may be displayed. Also, the video information acquired from the video session and the information that can be recognized by the analysis unit may be divided into different display layers, and the layer related to the former information may be made non-displayed.

[0053] When there is an area for displaying multiple moving images, the object 50 and the object 100 may be displayed only in all areas or some areas. For example, as shown in FIG. 8, it may be displayed only in the moving image on the guest side.

[0054] In the embodiments of the invention described in the above-described basic configuration examples 1 to 3, the embodiments may be realized as a single device, or may be realized by a plurality of devices (for example, a cloud server) partially or entirely connected by a network. For example, the control unit 110 and the storage 130 of each terminal 10 may be realized by different servers connected to each other by a network. That is, this system includes user terminals 10 and 20, a video session service terminal 30 that provides a two-way video session to the user terminals 10 and 20, and an evaluation terminal 40 that evaluates the video session. In this case, the following variation combinations of configurations can be considered. (1) All are processed only by the user terminal As shown in FIG. 8, by performing the processing by the analysis unit in the terminal that is conducting the video session, (although a certain processing capacity is required), the analysis / evaluation results can be obtained in real time simultaneously with the time of the video session. (2) Processed by the user terminal and the evaluation terminal As shown in FIG. 9, it is also possible to provide the evaluation terminal connected by a network or the like with an analysis unit. In this case, the moving image acquired by the user terminal is shared with the evaluation terminal simultaneously with or after the video session, and after being analyzed and evaluated by the analysis unit in the evaluation terminal, the information of the object 50 and the object 100 is shared with the user terminal together with or separately from the moving image data (that is, at least the information including the analysis data) and displayed on the display unit.

[0055] Using each configuration of the above-described functional configuration examples 1 to 3 or a combination thereof, the following system is realized.

[0056] <Embodiment> The moving image analysis system according to an embodiment of the present invention (hereinafter simply referred to as "system") analyzes and examines the reactions of participants based on moving images obtained by photographing all or only specific participants in an environment where an online session is held by a plurality of participants. The analysis may be performed regardless of whether the participants are displayed on the screen during the online session.

[0057] In particular, the system according to the present embodiment extracts and provides a moving image that satisfies a predetermined keyword from the already acquired moving image. Specifically, the system includes a moving image acquisition unit that acquires a moving image obtained by photographing a purchaser user during an online session, an analysis unit that analyzes changes in the biological reaction of the purchaser user based on the moving image acquired by the moving image acquisition unit, and a storage unit that stores a purchaser database in which the changes in the biological reaction of the purchaser user are analyzed.

[0058] Furthermore, the system includes a keyword reception unit that receives an input of a keyword from a new purchaser user, and an extraction unit that extracts a frame showing a positive biological reaction with respect to the keyword from the purchaser database based on the keyword.

[0059] As shown in FIG. 10, the system according to the present embodiment extracts and provides a part of a video in the following steps. First, an input of a keyword is received from a new purchaser who is trying to newly purchase a certain product. The keyword may be anything that can identify something to be purchased, such as a product name, service name, company name, etc. Then, based on the keyword, the moving image is searched, and a part that includes the keyword during the speech is extracted. Then, the above-described analysis unit analyzes what kind of emotion the person who first presses is speaking words related to the keyword. As a result of the analysis, if positive content is spoken with respect to the keyword, that part is extracted and provided to the user, and if it is not positive with respect to the keyword, the extracted clip is not provided.

[0060] The processes described using flowcharts in this specification do not necessarily have to be executed in the order shown in the figures. Some process steps may be executed in parallel. Additionally, additional process steps may be adopted, and some process steps may be omitted.

[0061] It is also possible to implement the embodiments described above by appropriately combining them. Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects that are apparent to those skilled in the art from the description in this specification, together with or instead of the above effects.

Description of Reference Numerals

[0062] 10, 20 User terminals 30 Video session service terminals 40 Evaluation terminals

Claims

【Claim 1】 A moving image acquisition unit that acquires a moving image obtained by photographing a purchaser user during an online session; An analysis unit that analyzes changes in the biological reaction of the purchaser user based on the moving image acquired by the moving image acquisition unit; A storage unit that stores a purchaser database in which changes in the biological reaction of the purchaser user are analyzed; A keyword reception unit that receives input of a keyword from a new purchaser user; An extraction unit that extracts a frame showing a positive biological reaction with respect to the keyword from the purchaser database based on the keyword; comprising a moving image analysis system.

Images