Information processing device, method, program, and system

Abstract

To enhance the sense of presence of another person even when one is performing treatment on oneself.SOLUTION: An information processing device according to an embodiment of the present invention comprises: an acquisition unit configured to acquire a video image of at least a first person and a second person; and a video editing unit configured to generate a video image displaying the first person in an area including a center of the image and a part of the second person in accordance with a preset display rate.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a method, a program, and a system. [Background technology]

[0002] BACKGROUND ART Conventionally, beauty treatments have been known in which practitioners such as estheticians perform beauty treatments on customers at beauty salons, etc. Customers can feel satisfied by receiving beauty treatments from others.

[0003] On the other hand, there is also a known beauty technique called "self-beauty treatment," in which the customer performs beauty treatments themselves, rather than through a therapist. In addition, customers may perform beauty treatments themselves at home, imitating the actions of professionals (Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-37626 Summary of the Invention [Problem to be solved by the invention]

[0005] However, even when performing treatment on oneself, it is difficult to feel as if one is receiving treatment from another person.

[0006] Therefore, the present invention aims to improve the sense of reality of another person even when performing treatment on oneself. [Means for solving the problem]

[0007] An information processing device according to one embodiment of the present invention includes an acquisition unit that acquires a video of at least a first person and a second person, and a video editing unit that generates a video that displays the first person in an area including the center of an image and also displays a portion of the second person according to a preset display ratio. [Effects of the Invention]

[0008] According to the present invention, even when a person is performing a treatment on himself, the sense of reality of another person can be improved. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram for explaining an overview of the present invention. [Figure 2] 1 is a diagram showing an overall configuration according to an embodiment of the present invention; [Figure 3] 1 is a diagram showing functional blocks of an information processing apparatus according to an embodiment of the present invention; [Figure 4] 1 is a diagram illustrating an information processing system according to an embodiment of the present invention. [Figure 5] 1 is a flowchart showing the overall processing flow according to an embodiment of the present invention. [Figure 6] 1 is a flowchart showing a flow of video editing processing according to an embodiment of the present invention. [Figure 7] 1 is a diagram illustrating a hardware configuration of an information processing apparatus according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configuration are designated by the same reference numerals, and redundant description will be omitted.

[0011] <Terminology> "Beauty-related activities" are any beauty-related activities performed with hands, cosmetic tools, etc. (e.g., applying cosmetics, etc.).

[0012] <Summary of the Invention> FIG. 1 is a diagram for explaining an overview of the present invention. In one embodiment of the present invention, a video is captured of a practitioner, such as an esthetician, performing a beauty treatment on a subject. This video is edited to generate a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a pre-set display ratio. By performing a beauty treatment on themselves while watching the video generated in this way, the subject can feel as if they are receiving a beauty treatment from another person (i.e., the practitioner). Steps S1 to S4 are described below.

[0013] In step 1 (S1), a practitioner such as an esthetician (an example of a second person described below) performs a beauty treatment on a subject (an example of a first person described below). At this time, the practitioner and the subject are photographed.

[0014] In step 2 (S2), the information processing device 10 (described in detail below) acquires the video of the practitioner and the subject taken in S1.

[0015] In step 3 (S3), the information processing device 10 edits the video acquired in S2 to generate a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a pre-set display ratio.

[0016] In step 4 (S4), the subject performs beauty-related procedures on himself / herself while watching the video generated by the information processing device 10 in S3 (a video in which the subject is displayed at the center of the image and a portion of the practitioner is displayed according to a pre-set display ratio).

[0017] <Overall structure> FIG. 2 is a diagram showing the overall configuration according to one embodiment of the present invention.

[0018] The information processing device 10 generates and plays back a video based on a video of people 20 (at least a first person and a second person), in which the first person is displayed at the center of the image and a part of the second person is displayed according to a preset display ratio. For example, the information processing device 10 is a smartphone, tablet, personal computer, or the like having a camera function (which may be a 3D camera). The information processing device 10 will be described in detail later with reference to FIG. 3.

[0019] In this specification, the case where the information processing device 10 is one device (for example, a smartphone with a camera function) will be described, but the information processing device 10 may be composed of multiple devices (for example, a device without a camera function and a digital camera). Note that a device for shooting a video, a device for generating a video based on the shot video, and a device for playing the generated video may be implemented in one device, or at least one of them may be implemented in a different device.

[0020] <Functional blocks of the information processing device 10> 3 is a diagram showing functional blocks of an information processing device 10 according to one embodiment of the present invention. The information processing device 10 can include a shooting unit 101, an acquisition unit 102, an identification unit 103, a video editing unit 104, and a playback unit 105. Furthermore, by executing a program, the information processing device 10 can function as the shooting unit 101, the acquisition unit 102, the identification unit 103, the video editing unit 104, and the playback unit 105. Each of these units will be described below.

[0021] The image capturing unit 101 captures images of at least a first person and a second person. The second person is performing a treatment (action) on the first person (for example, a beauty treatment).

[0022] The acquisition unit 102 acquires a video in which at least a first person and a second person are shot. Note that the acquisition unit 102 may acquire a video shot by the information processing device 10, or may acquire a video shot by a device other than the information processing device 10.

[0023] The identification unit 103 identifies the first person in a video in which at least a first person and a second person are captured. For example, the identification unit 103 can identify a person registered in advance as the first person (for example, by using face recognition, iris recognition, or the like). For example, the identification unit 103 can identify the person who moves the least in a video in which at least the first person and the second person are captured (i.e., the person who moves the least for the longest period of time in the entire video) as the first person. For example, the identification unit 103 can identify the person who appears most centrally in a video in which at least the first person and the second person are captured (i.e., the person who appears most centrally for the longest period of time in the entire video) as the first person.

[0024] The identification unit 103 identifies the second person in a video in which at least the first person and the second person are captured. For example, the identification unit 103 can identify a person other than the first person (for example, a person closest to the first person) as the second person.

[0025] The video editing unit 104 generates a video that displays a first person (e.g., the face of the first person) in an area including the center of the image, and also displays a part of a second person (e.g., either the face, whole body, or upper body of the second person) according to a pre-set display ratio (e.g., a display ratio at which a cut-off (i.e., partially cut-off) person can be recognized as a different person).

[0026] For example, the video editing unit 104 can generate a video in which the face of a first person is displayed at the center and the hand or cosmetic tool of a second person is displayed touching the face of the first person. While watching the video generated in this way, the first person can feel as if they are being touched by the second person by touching their own hand or cosmetic tool to their own face in the same way as the second person's hand or cosmetic tool.

[0027] <Display rate> Here, the display ratio will be explained. The display ratio indicates the ratio of the area in which the second person is displayed in the video to be generated. For example, the display ratio is the ratio of the area in which the second person is displayed to the area in which the first person is displayed. For example, the display ratio is the ratio of the area in which the second person is displayed when the size of the entire body of the second person estimated from the image is 100%. For example, the display ratio is the ratio of the number of pixels in the area in which the second person is displayed to the total number of pixels in the image. Note that the display ratio may be calculated based on the ratio of the area in which the first person is displayed to the area in which the second person is displayed.

[0028] For example, the subject may be able to specify the display rate itself, or may be able to specify the ratio of the area in which the first person is displayed to the area in which the second person is displayed (the display rate is calculated based on the ratio of the area in which the first person is displayed to the area in which the second person is displayed).

[0029] <Delete unnecessary videos> For example, the video editing unit 104 can delete a part of a video in which at least the first person and the second person are shot, based on the movement of the first person. Specifically, the video editing unit 104 can generate a video after deleting a video in which the magnitude of the movement of the first person is greater than a threshold (for example, a video taken when treatment is not being performed).

[0030] <Display of first person> The video editing unit 104 can display the first person (for example, face) so that the position of the center (for example, nose) of the first person coincides with the position of the center of the image or a position within a predetermined range from the center of the image. For example, the subject or the like may be able to specify a predetermined range (i.e., an allowable range of deviation from the center).

[0031] <Partial display of second person> Below, two examples of displaying a part of the second person will be described: <trimming the second person> and <enlarging / reducing the image>. Note that the subject may be able to specify either <trimming the second person> or <enlarging / reducing the image>. The subject may also be able to specify the part of the second person to be displayed (for example, the face, the whole body, or the upper body of the second person).

[0032] <Cropping of the second person (no zoom in / out)> The video editing unit 104 displays a part of the second person in accordance with a preset display ratio by trimming the second person (that is, the part of the second person other than the part is not displayed).

[0033] <Zoom in and out of the video> The video editing unit 104 displays a part of the second person in accordance with a preset display ratio by enlarging or reducing the video in which at least the first person and the second person are shot.

[0034] For example, when the difference between a preset display ratio and the (actual) display ratio of the second person is greater than a threshold, the video editing unit 104 can enlarge or reduce the video in which at least the first person and the second person are shot. In other words, by enlarging or reducing the video (for example, 9 percent, 31 percent) when the difference exceeds an allowable range (for example, 10-30 percent) including a preset display ratio (for example, 20 percent), the display range of the first person can be secured, and the visibility can be high, making it possible to clearly visually represent that the first person is receiving treatment.

[0035] For example, the video editing unit 104 can prevent a person watching the played video from feeling sick by gradually enlarging or reducing the video instead of instantly enlarging or reducing it.

[0036] <Background removal> The video editing unit 104 may trim the background from a video in which at least the first person and the second person are shot.

[0037] The playback unit 105 plays back the video generated by the video editing unit 104. Note that the video generated by the video editing unit 104 may be played back on the information processing device 10 or on a device other than the information processing device 10.

[0038] <Information Processing System> 4 is a diagram showing an information processing system 1 according to one embodiment of the present invention. The information processing system 1 may include an information processing device 10, an image capturing device 30, and a playback device 40.

[0039] As described above, at least one of the device for shooting a moving image, the device for generating a moving image based on the shot moving image, and the device for playing back the generated moving image may be implemented as separate devices. Fig. 4 shows an embodiment in which the device for shooting a moving image (shooting device 30), the device for generating a moving image based on the shot moving image (information processing device 10 (e.g., a server)), and the device for playing back the generated moving image (playback device 40) are implemented as separate devices. Note that the shooting device 30 and the playback device 40 may be the same device.

[0040] The image capturing device 30 has the function of the image capturing unit 101. The image capturing device 30 transmits the captured original image (a moving image capturing at least the first person and the second person) to the information processing device 10.

[0041] The information processing device 10 includes an acquisition unit 102, an identification unit 103, and a video editing unit 104. The information processing device 10 acquires original images (videos in which at least a first person and a second person are captured) from the image capturing device 30, and performs video editing processing. The information processing device 10 also delivers processed images (processed video) to the playback device 40.

[0042] The playback device 40 has the function of the playback unit 105. The playback device 40 plays back the video acquired from the information processing device 10.

[0043] <Processing method> An overall processing method and a video editing processing method according to one embodiment of the present invention will be described below.

[0044] FIG. 5 is a flowchart showing the overall processing flow according to one embodiment of the present invention.

[0045] In step 11 (S11), the photographing unit 101 (or the photographing device 30) photographs an original image. Specifically, the photographing unit 101 (or the photographing device 30) photographs at least a first person and a second person. The second person performs an arbitrary treatment (action) on the first person (for example, a beauty treatment).

[0046] In step 12 (S12), the video editing unit 104 performs video editing processing. Specifically, the video editing unit 104 generates a video based on the original images captured in S11.

[0047] In step 13 (S13), the playback unit 105 (or the playback device 40) plays back the processed image. Specifically, the playback unit 105 (or the playback device 40) plays back the moving image processed in S12.

[0048] FIG. 6 is a flowchart showing the flow of the video editing process (S12 in FIG. 5) according to one embodiment of the present invention.

[0049] In step 101 (S101), the acquisition unit 102 acquires an original image (a moving image in which at least a first person and a second person are shot).

[0050] In step 102 (S102), the identification unit 103 identifies the first person in a still image (frame) constituting the original image of S101 (a moving image in which at least the first person and the second person are shot).

[0051] In step 103 (S103), the video editing unit 104 moves the still image (frame) so that the first person identified in S102 is positioned at the center of the image to be generated.

[0052] In step 104 (S104), the video editing unit 104 trims the second person or enlarges or reduces the original image (more specifically, the still images (frames) that make up the original image) according to a display ratio that is set in advance.

[0053] In step 105 (S105), the video editing unit 104 determines whether or not processing has been completed for all frames constituting the original image acquired in S101. If so, the video editing process ends. If not, the process returns to step 102 and processes the next still image (for example, the frame at the next time among the frames constituting the video).

[0054] <Beauty method> One embodiment of the present invention is a beauty treatment method for performing a beauty treatment while viewing the image described above. That is, one embodiment of the present invention is a beauty treatment method for performing a beauty treatment while viewing a video of at least a first person and a second person, in which the video of at least the first person and the second person is displayed in an area including the center of the image and a portion of the second person is displayed according to a preset display ratio.

[0055] <Image with CG image superimposed on real-time image> In one embodiment of the present invention, a video of at least a first person and a second person may be a video in which a CG (computer graphics) image of the second person is superimposed on an image of the first person being captured in real time. Specifically, the information processing device 10 acquires an image of the first person being captured in real time (referred to as a real-time image). Then, the information processing device 10 determines the position and posture of the second person based on the first person in the real-time image (for example, based on the position and posture of the first person's hands or the contact points between the first person's hands or cosmetic tools and the face), and superimposes the real-time image with the CG image of the second person in the determined position and posture. Then, the information processing device 10 generates a video in which the first person is displayed in an area including the center of the real-time image and a portion of the CG image of the second person is displayed according to a preset display ratio.

[0056] <Effects> Thus, in one embodiment of the present invention, the subject (the subject) is displayed in the center as he or she receives a beauty treatment from a practitioner, and by watching a video in which a portion of the practitioner is displayed while the beauty treatment is performed on the subject, the subject can feel as if they are receiving a beauty treatment from another person (i.e., the practitioner).

[0057] <Hardware configuration> FIG. 7 is a diagram showing the hardware configuration of an information processing device according to an embodiment of the present invention.

[0058] The information processing device 10 includes a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1002, and a RAM (Random Access Memory) 1003. The CPU 1001, the ROM 1002, and the RAM 1003 form a so-called computer.

[0059] The information processing device 10 may also include an auxiliary storage device 1004 , a display device 1005 , an operation device 1006 , an I / F (Interface) device 1007 , and a drive device 1008 .

[0060] The hardware components of the information processing device 10 are connected to one another via a bus B.

[0061] The CPU 1001 is a computing device that executes various programs installed in the auxiliary storage device 1004 .

[0062] The ROM 1002 is a non-volatile memory. The ROM 1002 functions as a main storage device that stores various programs, data, etc. required for the CPU 1001 to execute various programs installed in the auxiliary storage device 1004. Specifically, the ROM 1002 functions as a main storage device that stores boot programs such as a BIOS (Basic Input / Output System) and an EFI (Extensible Firmware Interface).

[0063] The RAM 1003 is a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The RAM 1003 functions as a main storage device that provides a working area in which various programs installed in the auxiliary storage device 1004 are expanded when the CPU 1001 executes them.

[0064] The auxiliary storage device 1004 is an auxiliary storage device that stores various programs and information used when the various programs are executed.

[0065] The display device 1005 is a display device that displays the internal state of the information processing device 10 and the like.

[0066] The operation device 1006 is an input device through which the person operating the information processing device 10 inputs various instructions to the information processing device 10 .

[0067] The I / F device 1007 is a communication device that connects to a network and communicates with other devices.

[0068] Drive device 1008 is a device for loading storage medium 1009. The storage medium 1009 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, and magneto-optical disks. Storage medium 1009 may also include semiconductor memories that record information electrically, such as EPROMs (Erasable Programmable Read Only Memory) and flash memories.

[0069] The various programs to be installed in the auxiliary storage device 1004 are installed, for example, by setting the distributed storage medium 1009 in the drive device 1008 and reading out the various programs recorded on the storage medium 1009 by the drive device 1008. Alternatively, the various programs to be installed in the auxiliary storage device 1004 may be installed by being downloaded from a network via the I / F device 1007.

[0070] The information processing device 10 includes an image capturing device 1010. The image capturing device 1010 captures an image of a person 20.

[0071] Although the examples of the present invention have been described in detail above, the present invention is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present invention as set forth in the claims. [Explanation of symbols]

[0072] 1. Information Processing Systems 10. Information processing equipment 20 people 30 Imaging equipment 40 Playback device 101 Photography Department 102 Acquisition Department 103 Specific section 104 Video Editing Department 105 Playback Department 1001 CPU 1002 ROM 1003 RAM 1004 Auxiliary storage device 1005 Display device 1006 Operating device 1007 I / F device 1008 Drive device 1009 Storage medium 1010 Imaging device

Claims

1. A video editing unit that displays a first person in an area including the center of an image and generates a video that displays a part of a second person according to a preset display ratio. Equipped with The display rate is indicating a proportion of an area in which the second person is displayed in the generated video; indicating a ratio of an area in which the second person is displayed to an area in which the first person is displayed; indicating a ratio of an area in which the first person is displayed to an area in which the second person is displayed; The size of the entire body of the second person estimated from the image is taken as 100 percent, and the size indicates the proportion of the area in which the second person is displayed; or a ratio of the number of pixels in the area where the second person is displayed to the total number of pixels in the image; Information processing device.

2. a playback unit that plays the generated video; The information processing device according to claim 1 , further comprising:

3. 2. The information processing device according to claim 1, wherein the video editing unit displays a portion of the second person according to the preset display ratio by trimming the second person, and deletes a portion of the acquired video based on the movement of the first person.

4. 2. The information processing device according to claim 1, wherein the video editing unit displays a portion of the second person according to the pre-set display ratio by enlarging or reducing the acquired video, and enlarges or reduces the acquired video when the difference between the pre-set display ratio and the display ratio of the second person is greater than a threshold value.

5. an identification unit that identifies the first person, 5. The information processing device according to claim 1, wherein the identification unit identifies a person who has been registered in advance as the first person, or identifies a person who moves the least in the acquired video as the first person, or identifies a person who is displayed most centrally in the acquired video as the first person.

6. 6. The information processing device according to claim 1, wherein the video editing unit displays the first person so that a center position of the first person coincides with a center position of the image or a position within a predetermined range from the center of the image.

7. The information processing device according to claim 1 , wherein the video editing unit displays the face of the first person in an area including a center of the image.

8. The information processing device according to claim 1 , wherein the generated video displays one of a face, a whole body, and an upper body of the second person.

9. The information processing device according to claim 1 , wherein the first person is a person who receives a beauty treatment, and the second person is a practitioner who performs the beauty treatment on the first person.

10. a video editing unit that displays a first person in an area including the center of an image and generates a video that displays a part of a second person according to a preset display ratio; The video editing unit displays a portion of the second person according to the preset display ratio by trimming the second person, and deletes a portion of the acquired video based on the movement of the first person.

11. a video editing unit that displays a first person in an area including the center of an image and generates a video that displays a part of a second person according to a preset display ratio; The video editing unit displays a portion of the second person according to the preset display ratio by enlarging or reducing the acquired video, and enlarges or reduces the acquired video when the difference between the preset display ratio and the display ratio of the second person is greater than a threshold value.

12. an imaging unit that images at least the first person and the second person; a playback unit that plays the video created by the video editing unit; Furthermore, The information processing device according to any one of claims 1 to 11.

13. The video capturing device further includes an acquisition unit that acquires a video of at least the first person and the second person. The information processing device according to any one of claims 1 to 12.

14. Computer, A program for causing the information processing device according to any one of claims 1 to 13 to function as the information processing device.

15. A beauty method in which at least a first person and a second person perform beauty-related activities while viewing a video that has been shot, a step of generating a video by a computer in which the first person is displayed in an area including the center of an image and a part of the second person is displayed according to a preset display ratio; A step in which a viewer of the video performs a beauty-related activity while viewing the video; Including, The display rate is indicating a proportion of an area in which the second person is displayed in the generated video; indicating a ratio of an area in which the second person is displayed to an area in which the first person is displayed; indicating a ratio of an area in which the first person is displayed to an area in which the second person is displayed; The size of the entire body of the second person estimated from the image is taken as 100 percent, and the size indicates the proportion of the area in which the second person is displayed; or and indicating a ratio of the number of pixels in the area in which the second person is displayed to the total number of pixels in the image.

16. A beauty method in which at least a first person and a second person perform beauty-related activities while viewing a video that has been shot, a step of generating a video by a computer in which the first person is displayed in an area including the center of an image and a part of the second person is displayed according to a preset display ratio; A step in which a viewer of the video performs a beauty-related activity while viewing the video; and The step of generating the video includes: the computer crops the second person to display a portion of the second person according to the preset display ratio, and deletes a portion of the captured video based on a movement of the first person; or a step in which the computer displays a portion of the second person according to the preset display ratio by enlarging or reducing the acquired video, and enlarging or reducing the acquired video when a difference between the preset display ratio and the display ratio of the second person is greater than a threshold value.

17. The beauty treatment method according to claim 15 or 16, wherein the first person is a viewer of the video.

Images