Recording device and program

The recording device captures users' emotional reactions to images, addressing the subjective value of photographs, enabling easy retrieval and utilization by associating emotional responses with the images.

JP2024168518AActive Publication Date: 2024-12-05DAI NIPPON PRINTING CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023085268
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-24
Publication Date
2024-12-05
Estimated Expiration
2043-05-24

AI Technical Summary

Technical Problem

Conventional methods fail to consider the subjective importance of photographs, such as emotional excitement and memories, making it difficult to effectively search and utilize photos that evoke these feelings.

Method used

A recording device that acquires passive reaction data from users' emotions when viewing images, recording these reactions in association with the images, using biometric information or facial expressions, to facilitate easy retrieval based on subjective evaluations.

Benefits of technology

Enables effective utilization of photographs by allowing users to search and extract images based on emotional responses, enhancing the operation by eliminating the need for active input and capturing subtle emotional reactions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024168518000001_ABST
    Figure 2024168518000001_ABST
Patent Text Reader

Abstract

To provide a recording device, etc. that are to promote the use of photographs, etc.SOLUTION: A user terminal 2 includes display means 210 for displaying a photograph on a display, acquisition means 220 for acquiring passive reaction data of a user based on the emotion when the user views a photograph displayed on the display, and recording means 230 for recording a result of the user's reaction when viewing the photograph on the basis of the reaction data in association with the photograph. The reaction data is, for example, biometric information of the user, and the acquisition means 220 receives the reaction data from a wearable device 3 worn by the user.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a recording device or the like for promoting effective use of images such as photographs. [Background technology]

[0002] As photography becomes digitalized and the number of photos taken increases dramatically, the task of printing good photos and compiling them into a photo book becomes more tedious, and often the photos you have taken end up going unnoticed.

[0003] In response to this, Patent Document 1 discloses a system that makes it easy to search for "good photos" using objective measures such as the sharpness of the focus, the quality of the composition, and the degree to which the subject is smiling, helping to make effective use of the photos that have been taken. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-097090 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the photographs that make us feel excited, thrilled, or evoke memories when we actually see them do not necessarily match what would be considered a "good photograph" analyzed using the objective criteria mentioned above.

[0006] For example, we may feel a sense of dynamism in an out-of-focus, blurry photograph, or feel attached to a child's face that is visible in the image. A photograph of a crying child may evoke as much affection for the child's parents as a smiling face, and it is difficult to measure the importance of these photographs using objective criteria such as focus, composition, or the degree of the smile.

[0007] The value of a photograph lies in the fact that it allows us to remember the excitement, thrills, and memories we feel when we see it, and evokes them every time we look at it. However, with conventional technology it was not possible to effectively search for or extract photos that have this subjective importance, posing a challenge in terms of promoting the use of photographs.

[0008] The present invention has been made in consideration of the above problems, and has an object to provide a recording device etc. that leads to the promotion of the use of photographs etc. [Means for solving the problem]

[0009] The first invention for solving the above-mentioned problems is a recording device characterized by having an acquisition means for acquiring passive reaction data of a user based on emotions when viewing an image displayed on a display, and a recording means for recording the user's reaction results when viewing the image based on the reaction data, in association with the image.

[0010] In the present invention, passive reaction data based on the emotions of a person who sees an image such as a photograph is acquired, and the reaction results when the person sees the image are linked to the image and recorded. Even with the same image, the reaction varies depending on the person who sees it, and the reaction often differs from the evaluation of the image based on an objective scale. In the present invention, by adding the user's emotional (subjective) evaluation to the image, such as the image making the user excited, making the heart flutter, or evoking memories from the past, it becomes possible to search or extract images later based on these evaluations, which contributes to the effective use of images. In addition, the user only needs to look at the photo, and there is no need for active actions such as entering impressions in text or pressing the "Like" button, so the operation is simple.

[0011] It is desirable that the reaction data is biometric information of a user, and that the acquisition means receives the reaction data from a wearable device worn by the user. The reaction data may be biological information such as the user's body temperature or heart rate, and may also include reactions that the user is not particularly aware of.

[0012] It is preferable that the reaction data is a captured image of the user showing a facial expression of the user, and that the acquisition means acquires the captured image by photographing the user. The reaction data may be a captured image showing the user's facial expression. In this case, known image recognition technology or the like may be used to determine the type of emotion the user is feeling from the captured image and record it as the reaction result.

[0013] It is also preferable that the recording device is a mobile terminal equipped with the display. The recording device may be, for example, a mobile terminal equipped with a display, such as a smartphone, and may record the user's reaction to an image displayed on the smartphone or the like.

[0014] A second invention is a program for causing a computer to function as a recording device having an acquisition means for acquiring passive reaction data of a user based on emotions felt when viewing an image displayed on a display, and a recording means for recording the user's reaction results when viewing the image based on the reaction data, in association with the image. Effect of the Invention

[0015] According to the present invention, a recording device or the like that leads to the promotion of the use of photographs and the like can be provided. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 shows a recording system 1. [Diagram 2] FIG. 2 shows a configuration of a user terminal 2. [Diagram 3] FIG. 2 is a diagram showing the configuration of a wearable device 3. [Figure 4] FIG. 2 is a diagram showing the functions of the recording system 1. [Diagram 5] 4 is a flowchart showing the processing in the recording system 1. [Figure 6] Example system including Server 4. [Figure 7]13 is a flowchart for a case where a photographed image of a user is used as reaction data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[0018] (1. Recording System 1) Fig. 1 is a diagram showing a recording system 1 including a user terminal 2 according to an embodiment of the recording apparatus of the present invention. The recording system 1 records a user's reaction when viewing an image displayed on the display of the user terminal 2 in association with the image, and has a configuration in which the user terminal 2 and a wearable device 3 are communicatively connected as shown in Fig. 1.

[0019] The user terminal 2 is a terminal carried by a user, and is a computer (computer terminal) that performs various processes related to recording the reaction results. In this embodiment, a mobile terminal such as a smartphone or a tablet terminal is used as the user terminal 2.

[0020] Fig. 2 is a diagram showing the configuration of the user terminal 2. As shown in Fig. 2, the user terminal 2 is configured by connecting a control unit 21, a storage unit 22, a display 23, an input unit 24, a communication unit 25, a camera 26, an audio input / output unit 27, etc., via a bus or the like. However, the user terminal 2 is not limited to this and can have various appropriate configurations.

[0021] The control unit 21 is composed of a CPU, a ROM, a RAM, etc. The CPU loads programs related to the processing of the user terminal 2 stored in a storage medium such as the storage unit 22 or ROM into a work area on the RAM, executes the programs, and controls each part of the user terminal 2 to realize the processing described below. The ROM is a non-volatile memory, and permanently stores programs such as a boot program and a BIOS, data, etc. The RAM is a volatile memory, and temporarily stores programs and data loaded from the storage unit 22 or ROM, etc., and has a work area used by the control unit 21 to perform various processing.

[0022] The storage unit 22 is a flash memory or the like, and stores programs executed by the user terminal 2 during processing described below, data necessary for executing the programs, an OS (operating system), etc. These programs and data are read and executed by the control unit 21 as necessary.

[0023] The display 23 is, for example, a liquid crystal panel, and is realized as a touch-panel display that also serves as the input unit 24. The display 23 displays various screens related to the operation and selection of the user terminal 2, and accepts user instruction inputs, etc., by operating the touch panel on the screen.

[0024] The communication unit 25 is a communication interface that mediates communication such as wireless communication, and communicates with the wearable device 3 and the like.

[0025] The camera 26 is an area camera having an imaging element such as a CCD (Charged Coupled Device), and the audio input / output unit 27 includes a microphone for audio input and a speaker for audio output.

[0026] Returning to the explanation of Fig. 1, the wearable device 3 is a terminal worn by the user, for example a wristband-type or watch-type terminal. However, the wearable device 3 is not limited to this, and earphone-type, earring-type, bracelet-type, headgear-type, eyeglass-type terminals, etc. can also be used.

[0027] Fig. 3 is a diagram showing the configuration of the wearable device 3. As shown in Fig. 3, the wearable device 3 includes a main controller 31, a display 32, a body temperature sensor 33, a heart rate sensor 34, a wireless module 35, and the like.

[0028] The main controller 31 is a control unit that controls the entire wearable device 3, and includes a CPU, ROM, RAM, memory, etc. The CPU, ROM, and RAM are substantially the same as those described above. The memory stores various setting information for the wearable device 3, measurement data measured by the wearable device 3, etc.

[0029] The display 32 is a simple display such as a small liquid crystal panel or electronic paper.

[0030] The body temperature sensor 33 and the heart rate sensor 34 measure the body temperature and the heart rate, respectively, of the user as biometric information of the user wearing the wearable device 3. The body temperature sensor 33 and the heart rate sensor 34 may be known sensors.

[0031] The wireless module 35 is a communication unit for performing wireless communication with the user terminal 2 and transmitting measurement data from the above-mentioned sensors 33, 34 to the user terminal 2, and includes an antenna and the like.

[0032] (2. Functions of Recording System 1) Fig. 4 is a block diagram showing the functions of the recording system 1. As shown in Fig. 4, in the recording system 1, the user terminal 2 (recording device) has a display unit 210, an acquisition unit 220, a recording unit 230, and the like.

[0033] The display means 210 allows the control unit 21 of the user terminal 2 to display the photographs (still images) stored in the storage unit 22 on the display 23.

[0034] The acquisition means 220 is for the control unit 21 of the user terminal 2 to acquire passive reaction data of the user based on the emotions felt when viewing a photo displayed on the display 23. Passive reaction data is data indicating a reaction that does not involve active behavior such as inputting an impression in text or pressing a "Like" button. This reaction data is, for example, the biometric information of the user described above, but is not particularly limited as long as it is based on the user's emotions.

[0035] The recording means 230 is a means for the control unit 21 of the user terminal 2 to record the reaction result of the user when viewing a photograph based on the above-mentioned reaction data, in association with the photograph.

[0036] (3. Processing of Recording System 1) Next, the processing executed in the recording system 1 will be described with reference to Fig. 5 etc. Fig. 5 is a flowchart showing the processing in the recording system 1, and each processing in Fig. 5 is executed by the control unit 21 of the user terminal 2 by controlling each unit of the user terminal 2.

[0037] In this embodiment, the user terminal 2 starts an application in response to a user's operation, and displays the photographs stored in the storage unit 22 on the display 23 (S1).

[0038] The user looks at the image displayed on the display 23. At this time, the wearable device 3 continuously measures the user's biological information and transmits the measured biological information to the user terminal 2. The user terminal 2 receives and acquires the biological information as reaction data (S2). As described above, the biological information is the body temperature measured by the body temperature sensor 33 and the heart rate measured by the heart rate sensor 34, but is not limited to these. For example, it may be brain waves, etc.

[0039] The user terminal 2 analyzes the reaction data (S3), and if the reaction data indicates the user's emotions such as joy, surprise, calm, etc. (S4; YES), the user terminal 2 records information indicating that emotion occurred when the user viewed the photo as a reaction result, linking it to the photo (S5). If the biometric information does not indicate the user's emotions (S4; NO), the process proceeds directly to S6.

[0040] The method of analyzing and judging the reaction data is not particularly limited, but for example, in S3, the value of the reaction data or its change over time is compared with a predetermined threshold, and in S4, if the value of the reaction data or its change over time is equal to or greater than the threshold, it can be judged that the emotion corresponding to the threshold (e.g., joy, etc.) is present in the reaction data. Alternatively, if the value of the reaction data is less than the threshold, it can be judged that the emotion corresponding to the threshold (e.g., calm, etc.) is present in the reaction data. Furthermore, two or more types of bioinformation may be combined and used for analysis and judgment, or one type of bioinformation may be used for analysis and judgment.

[0041] The user terminal 2 repeats the processes from S1 to S5 until all the photos to be displayed have been displayed (S6; NO), and when all the photos have been displayed (S6; YES), ends the process.

[0042] The reaction results recorded in this way can be used for searching and extracting photos. For example, photos that have brought emotions such as joy to the viewer can be easily searched and extracted using the reaction results as a key, and compiled into a photo book or the like. In addition, by recording the viewer's emotions as added value to the photos, it is possible to learn the feelings of the viewer toward the photos left behind after their death. In addition, by having celebrities or athletes as viewers, it is possible to sell photos with added value such as the emotions of the celebrities or athletes.

[0043] As described above, in this embodiment, passive reaction data based on the emotions of the person who viewed the photo is acquired, and the reaction result when the person viewed the photo is linked to the photo and recorded. Even for the same photo, the reaction varies depending on the person who views it, and the reaction often differs from the evaluation of the photo based on an objective scale. In this embodiment, by adding the user's emotional (subjective) evaluation to the photo, such as when the photo made him / her excited, made his / her heart flutter, or evoked past memories, it becomes possible to search or extract the photo later based on these evaluations, which contributes to the effective use of photos. In addition, the user only needs to look through many photos at once, and there is no need for active actions such as inputting impressions in text or pressing the "Like" button, so the operation is simple.

[0044] In addition, in this embodiment, by using the above-mentioned reaction data as biological information such as the user's body temperature and heart rate, it is possible to record reactions that the user is not particularly aware of.

[0045] However, the present invention is not limited to the above embodiment. For example, in this embodiment, the photographs stored in the user terminal 2 are displayed, but as shown in Fig. 6, the photographs may be stored in a server 4 (computer) connected to the user terminal 2 via a network in association with a user account, etc., and the photographs may be transmitted to the user terminal 2 and displayed on the display 23.

[0046] In this case, the user terminal 2 receives the reaction data from the wearable device 3 and requests the server 4 to record the reaction results, or the server 4 receives the reaction data (measured by the wearable device 3) from the user terminal 2 and records the reaction results. In the former case, the user terminal 2 functions as the recording device of the present invention, and in the latter case, the server 4 functions as the recording device of the present invention.

[0047] In this embodiment, biometric information measured by the wearable device 3 is used as passive reaction data of the user, but currently, research is progressing on image recognition technology using AI (artificial intelligence) that analyzes emotions from images of people's facial expressions. Therefore, instead of biometric information, it is also possible to use images of the user's facial expressions captured by the camera 26 of the user terminal 2.

[0048] The flow in this case is shown in Fig. 7, which differs from the flow in Fig. 5 in that instead of receiving reaction data from the wearable device 3 (see S2 in Fig. 5), the user terminal 2 activates the camera 26 to capture the user's facial expression and acquires the captured image as reaction data (S2' in Fig. 7). In S3, the above-mentioned image recognition technology is used to analyze the user's facial expression in the captured image, and in S4, it is determined whether or not the facial expression shows any emotion. If the user's facial expression shows any emotion, the type of emotion (happiness, surprise, calm, etc.) can be determined and the result can be recorded as a reaction result.

[0049] In this embodiment, the user's reaction results to the photo displayed on the user terminal 2 such as a smartphone can be recorded, but the photo can also be sent from the user terminal 2 to a wearable device 3 such as a head mounted display for display, and the wearable device 3 can capture the user's facial expression and transmit the captured image as reaction data to the user terminal 2. Biometric information such as pupil size can also be obtained as reaction data from the captured image of the user.

[0050] In addition, in this embodiment, a photograph (still image) is displayed and the reaction results to the photograph are recorded in association with the photograph, but a video (moving image) may be displayed instead of a photograph and the reaction results to the video may be recorded in association with the video.

[0051] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0052] 1: System of Record 2: User terminal 3: Wearable devices 4: Server 23, 32: Display 210:Display means 220: Acquisition means 230: Recording means

Claims

1. An acquisition means for acquiring passive reaction data of a user based on emotions felt when viewing an image displayed on a display; a recording means for recording a reaction result of the user when viewing the image based on the reaction data in association with the image; A recording apparatus comprising:

2. The reaction data is biometric information of a user, 2. The recording apparatus according to claim 1, wherein the acquisition means receives the reaction data from a wearable device worn by a user.

3. the reaction data is a captured image of the user showing a facial expression of the user; 2. The recording apparatus according to claim 1, wherein the acquisition means captures an image of the user by photographing the user.

4. 2. The recording apparatus according to claim 1, wherein the recording apparatus is a portable terminal equipped with the display.

5. Computer, An acquisition means for acquiring passive reaction data of a user based on emotions felt when viewing an image displayed on a display; a recording means for recording a reaction result of the user when viewing the image based on the reaction data in association with the image; A program for causing the device to function as a recording device having the above configuration.

Citation Information

Patent Citations

  • Imaging method and system

    JP2004178593A

  • Mobile terminal with camera

    JP2010148052A

  • Display device

    JP2011193275A

  • Information processing device, information processing method, and program

    JP2019047234A

  • Emotion tagging system, method, and program

    JP2023025400A