Information processing device, information processing method, and information processing program

Abstract

The information processing device according to the present disclosure includes a first determination unit that determines whether the size of the area occupied by the user's face facing forward in the shooting area is within a threshold value, a second determination unit that determines whether the size of the area occupied by the user's face facing side in the shooting area is within the threshold value when it is determined that the size of the area occupied by the user's face facing side is within the threshold value, and a shooting unit that photographs the user's face facing side when it is determined that the size of the area occupied by the user's face facing side is within the threshold value.

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program, and more particularly, to the use of a head-related transfer function. [Background technology]

[0002] A technology is being used to reproduce a three-dimensional sound image in headphones, etc., by using a head-related transfer function, which mathematically represents how sound reaches the ears from the sound source.

[0003] Since head-related transfer functions vary greatly from person to person, it is desirable to use head-related transfer functions generated for each individual when using them. For example, a technology is known in which a three-dimensional digital model of the head (hereinafter, referred to as a "3D model") is generated based on an image of the user's auricle, and the head-related transfer function of the user is calculated from the generated 3D model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Pat. No. 9,544,706 Summary of the Invention [Problem to be solved by the invention]

[0005] According to the conventional technology, a head-related transfer function calculated individually for each user can be used for information processing, thereby improving the sense of positioning of a sound image.

[0006] However, in the above conventional technology, it is difficult to improve the convenience of the user. For example, in the conventional technology, the head-related transfer function is calculated using an image of the user's auricle, but since it is difficult for the user to photograph the auricle himself, it is expected that the photographing may fail or the auricle of an appropriate size may not be included in the photographed image. In this case, the user may be required to retake the photograph many times, or may not be able to obtain a head-related transfer function that is calculated with high accuracy.

[0007] Therefore, the present disclosure proposes an information processing device, an information processing method, and an information processing program that can improve the convenience of a user with regard to imaging performed for calculating a head-related transfer function. [Means for solving the problem]

[0008] In order to solve the above problems, an information processing device of one embodiment of the present disclosure includes a first determination unit that determines whether or not a size of an area occupied by a user's forward-facing face in a shooting area is within a threshold, a second determination unit that determines whether or not a size of an area occupied by the user's side-facing face in the shooting area is within a threshold when it is determined that the size of the area occupied by the user's forward-facing face is within the threshold, and an imaging unit that captures the user's side-facing face when it is determined that the size of the area occupied by the user's side-facing face is within the threshold. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an information processing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a conceptual diagram illustrating information processing according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a configuration example of a user terminal according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating a first determination process according to the embodiment. [Diagram 5] 11A to 11C are diagrams illustrating a second determination process according to the embodiment. [Figure 6]FIG. 2 is a diagram illustrating an example of the configuration of a cloud server according to the embodiment. [Figure 7] 1 is a flowchart (1) showing a procedure of information processing according to the embodiment. [Figure 8] 11 is a flowchart (2) showing the procedure of information processing according to the embodiment. [Figure 9] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and duplicated descriptions will be omitted.

[0011] The present disclosure will be described in the following order. 1. Embodiment 1-1. Configuration of information processing system according to embodiment and example of information processing 1-2. Configuration of the information processing device according to the embodiment 1-3. Configuration of cloud server according to embodiment 1-4. Information Processing Procedure According to the Embodiment 2. Variations 3. Other embodiments 4. Effects of the information processing device according to the present disclosure 5. Hardware Configuration

[0012] (1. EMBODIMENT) [1-1. Configuration of information processing system according to embodiment and example of information processing] First, a configuration of an information processing system 1 according to the present disclosure and an overview of information processing executed by the information processing system 1 will be described with reference to Figs. 1 and 2. Fig. 1 is a diagram illustrating the information processing system 1 according to the embodiment of the present disclosure. Information processing according to the embodiment of the present disclosure is realized by the information processing system 1 illustrated in Fig. 1. As illustrated in Fig. 1, the information processing system 1 includes a user terminal 10 and a cloud server 100. Each device included in the information processing system 1 communicates with each other using a wired or wireless network (not illustrated). Note that the number of each device constituting the information processing system 1 is not limited to that illustrated.

[0013] The user terminal 10 is an example of an information processing device according to the present disclosure, and is an information processing terminal that executes information processing using a head-related transfer function (hereinafter, may be referred to as HRTF (Head-Related Transfer Function)). Specifically, the user terminal 10 uses HRTF when playing music, videos, etc., to perform highly realistic playback and create highly effective 3D sound. The user terminal 10 is realized, for example, by a smartphone or the like having a shooting function.

[0014] The cloud server 100 calculates an HRTF corresponding to each user and provides the calculated HRTF to the user terminal 10. The cloud server 100 acquires an image of the user's ear from the user terminal 10 via the network, and calculates the HRTF of the user based on the acquired image.

[0015] As described above, the user terminal 10 executes information processing using HRTF. HRTF represents, as a transfer function, a change in sound caused by surrounding objects including the shape of the human auricle (ear shell) and head. In general, measurement data for determining HRTF is acquired by measuring a measurement acoustic signal using a microphone attached inside the human auricle or a dummy head microphone.

[0016] For example, HRTFs used in technologies such as 3D audio are often calculated using measurement data acquired by a dummy head microphone or the average value of measurement data acquired from a large number of people. However, since HRTFs vary greatly from person to person, it is desirable to use the user's own HRTF in order to achieve a more effective audio production effect. In other words, by replacing the general HRTF with the user's own HRTF, a more realistic audio experience can be provided to the user.

[0017] One example of a method for calculating a user's personal HRTF is to calculate the user's HRTF from an image capturing the shape of the user's ear. For example, the cloud server 100 acquires an image including the shape of the user's ear captured by the user terminal 10, and calculates the user's personal HRTF based on the acquired image. The cloud server 100 then transmits the calculated HRTF to the user terminal 10. This allows the user terminal 10 to play back a sound source in which 3D sound has been recorded using the user's personal HRTF, or to play back a video with a sense of realism.

[0018] However, there are some problems in implementing the above method. For example, the cloud server 100 acquires an image in which the shape of the user's ear can be recognized, and calculates the HRTF based on the acquired image. In this case, the user is required to take an image including his / her ear and upload the captured image to the cloud server 100. At this time, it is assumed that the user takes the image using a smartphone such as the user terminal 10 having a photographing function.

[0019] However, when the user himself / herself takes a side view, it is difficult to properly photograph the ear because the user cannot check the photographing area (i.e., the screen of the user terminal 10). Therefore, even if the user takes a side view, there is a high possibility that the image obtained will not include the ear or the ear will be extremely tilted. In this case, the user may be required to retake the photograph many times, or may not be able to obtain an HRTF that is calculated with high accuracy. In addition, depending on the shape of the user terminal 10, it may be difficult for the user to press the shutter button at the appropriate timing. Furthermore, in such a selfie, it is assumed that the user himself / herself holds the user terminal 10 in his / her hand and photographs the image, so there is a high possibility that the photographed image will be blurred.

[0020] As described above, there are various problems associated with the shooting performed to calculate HRTF. The user terminal 10 according to the embodiment solves the above problems by the information processing according to the present disclosure described below. Below, an overview of the information processing according to the present disclosure will be described along with the flow using FIG. 1 and FIG. 2.

[0021] First, an overview of the flow of the HRTF generation process by the information processing system 1 will be described with reference to FIG. 1. As shown in FIG. 1, the user photographs himself / herself from the side of his / her head to obtain an image including an image of his / her ear (step S1). The detailed description of the photographing process by the user terminal 10 will be described with reference to FIG. 2 and subsequent figures. In addition, in the present disclosure, the image is not limited to a two-dimensional color image that can generally be photographed by the user terminal 10 or the like, but may be a monochrome image, a depth image including depth information, or any combination thereof. In addition, the image used in the process of the present disclosure may be not only one image, but also multiple images.

[0022] The user terminal 10 captures an image 5 of the side of the user's head by photographing in step S1, in which the shape of the user's ears can be recognized (step S2).

[0023] Next, the user terminal 10 transmits the acquired image 5 to the cloud server 100 (step S3). The cloud server 100 acquires the image transmitted from the user terminal 10. Then, the cloud server 100 calculates the user's HRTF from the image (step S4). Various methods may be adopted for the cloud server 100 to calculate the HRTF. For example, when a two-dimensional image including an image of an ear is input, the cloud server 100 may calculate the HRTF using a model that has been trained to output an HRTF corresponding to the ear. Alternatively, the cloud server 100 may generate a 3D model of the user's head from the two-dimensional image including an image of the ear, and calculate the HRTF from the generated 3D model. In the following description, the HRTF calculated according to an individual user may be referred to as a personalized HRTF (personal HRTF) and may be distinguished from a general HRTF.

[0024] Furthermore, the cloud server 100 may convolve the calculated personalized HRTF with an RIR (Room Impulse Response) to generate a BRIR (Binaural Room Impulse Response). In this specification, when HRTF is used, it is meant to include not only HRTF but also information such as BRIR generated using HRTF. For example, data used when playing music on the user terminal 10 may actually be BRIR rather than HRTF, but in the following description, the two may be interpreted interchangeably. For example, a description such as "transmitting the calculated HRTF" also includes a situation in which the calculated BRIR is transmitted. Furthermore, a description such as "playing using the HRTF" also includes a situation in which the BRIR is played.

[0025] Thereafter, the cloud server 100 transmits the generated personalized HRTF to the user terminal 10 (step S5). The user terminal 10 acquires the personalized HRTF. In this manner, the user acquires his / her own personalized HRTF by photographing his / her own profile and transmitting the image to the cloud server 100. Thereafter, the user terminal 10 can use the personalized HRTF corresponding to the user, for example, when playing back 3D sound or a multi-channel sound source.

[0026] Next, an overview of information processing (photographing processing) according to the present disclosure by the user terminal 10 will be described with reference to FIG. 2. FIG. 2 is a conceptual diagram showing information processing according to an embodiment. FIG. 2 shows an image displayed on the screen of the user terminal 10 when the user takes a selfie. In other words, the screen of the user terminal 10 shown in FIG. 2 shows the area that the user terminal 10 is about to photograph. In the explanation of FIG. 2, when illustrating the state transition of the user terminal 10, the screen display states of the user terminal 10 are expressed as a first state, a second state, . . . , and an Nth state (N is an arbitrary number). In addition, in the following explanation, a known image recognition method may be used as appropriate for the method of recognizing the front and side of the user and the user's ears.

[0027] When a user desires to generate a personalized HRTF, the user activates a photographing function and takes a photograph of his / her own ear. This photographing function is realized, for example, by an application (hereinafter, simply referred to as an "app") provided by a business entity that manages the cloud server 100.

[0028] When the user starts the application, the user terminal 10 transitions to the first state. The shooting area 21 shown in FIG. 2 is an area that the user terminal 10 is going to shoot. That is, the shooting area 21 is a space captured by the camera of the user terminal 10 displayed on the screen. As shown in FIG. 2, the user terminal 10 first outputs a voice A01 that prompts the user to shoot an image of the user facing forward (step S11). For example, the user terminal 10 outputs a voice A01 with the content of "Please face forward and fit within the frame of the screen." In this way, the user terminal 10 appropriately outputs TTS (text-to-speech) to give the user appropriate information and feedback. Note that such a response process is not limited to voice, and may be performed by a method such as vibration that vibrates the user terminal 10 or displaying a character string on the screen. In addition, the user terminal 10 may output not only TTS but also a shutter sound, a sound effect indicating that appropriate shooting has been performed, and the like as the voice.

[0029] That is, the user terminal 10 does not immediately prompt the user to take a side shot, but first outputs a response to adjust the position of the user facing forward. With a front-facing image, the user can recognize the position and size of his or her own face while holding the user terminal 10 in his or her hand. In this way, the user terminal 10 first displays the user's face facing forward on the screen, and adjusts the user so that he or she is in a position suitable for shooting. This allows the user terminal 10 to improve the probability of success in taking a side shot of the head that is taken later.

[0030] In the first state, the user terminal 10 determines whether the position and size of the user's face are appropriate for photographing in the photographing area 21. At this time, the user terminal 10 may display a guide frame 22 on the screen for the user to adjust the position and size of the front face. This allows the user to adjust the position of their face and the position of the user terminal 10 held in their hand while referring to the guide frame 22.

[0031] The user terminal 10 determines whether the size of the area occupied by the user's front-facing face in the shooting area 21 is within a threshold. At this time, the user terminal 10 may output a response appropriately according to the position and size of the user's face. For example, when the size of the user's face is displayed small compared to the shooting area 21 or the guide frame 22, the user terminal 10 may output a voice A02 such as "Please move the camera closer" to prompt the user to move the face or the user terminal 10 to an appropriate position.

[0032] When the user terminal 10 determines that the size of the area occupied by the user's forward facing face in the photographic area 21 is within the threshold, the user terminal 10 transitions to the second state (step S12).

[0033] In the second state, the user terminal 10 notifies the user that the position of the user's face is now suitable for photographing by, for example, changing the color of the guide frame 22 or outputting a voice A03 such as "You are now in the appropriate position" (step S13).

[0034] Next, the user terminal 10 outputs a voice A04 such as "Please turn to the right slowly," to prompt the user to turn to the side with respect to the camera (i.e., the user terminal 10).

[0035] After outputting the sound A04, the user terminal 10 transitions to the third state (step S14). In the third state, the user terminal 10 erases the guide frame 22 and displays a new guide frame 23 for the side surface in the shooting area 21.

[0036] The user terminal 10 determines whether or not the size of the area occupied by the user's face facing the side in the shooting area 21 is within a threshold. More specifically, the user terminal 10 determines whether or not the area including the user's ear is in an appropriate position with respect to the shooting area 21. The appropriate position means, for example, that a part of the area including the ear is not outside the edge of the shooting area 21, that the ratio of the area including the ear to the shooting area 21 is within a threshold, or that the ratio of the area including the ear to the side of the user's head is within a threshold. In other words, the appropriate position is a position where the user's ear is estimated to be included in the shooting image to such an extent that the cloud server 100 does not have any trouble generating the HRTF.

[0037] If the position of the side of the user's head or ear is not appropriate, the user terminal 10 may output an appropriate response. For example, if the area including the user's ear is outside the shooting area 21 or the guide frame 23, the user terminal 10 may output a voice A05 such as "Please turn the camera slightly to the right" to prompt the user to move the face or the user terminal 10 to an appropriate position.

[0038] Thereafter, if the user terminal 10 determines that the size of the area occupied by the user's side-facing face in the shooting area 21 is within the threshold, in other words, when the area including the user's ears is in the appropriate position, it outputs a voice A06 such as "Ears have been detected" (step S15).

[0039] Furthermore, the user terminal 10 judges whether the behavior amount of the user terminal 10 (for example, the acceleration or speed of the user terminal 10 itself) is within a threshold, and whether the position of the user's ear once detected has moved beyond the threshold in the next frame. Then, when the behavior amount of the user terminal 10 or the amount of movement of the position of the user's ear is within the threshold, the user terminal 10 automatically captures an image of the side of the user's head. Note that this process is performed to prevent blurring in the captured image.

[0040] As described above, the user terminal 10, which is an example of an information processing device according to the present disclosure, determines whether or not the size of the area occupied by the user's frontal face is within a threshold in the photographing area 21, and then determines whether or not the size of the area occupied by the user's side-facing face is within a threshold. Then, when it is determined that the size of the area occupied by the user's side-facing face is within the threshold, the user terminal 10 photographs the user's side-facing face.

[0041] That is, the user terminal 10 photographs the side of the user's head after aligning the user's face so that the user can easily view the user. In this way, the user terminal 10 can reliably photograph an image of the side of the user's head, including the ear in an appropriate position, which is difficult to obtain by taking a selfie, by performing a process for photographing the side of the user's head in stages. This allows the user terminal 10 to reduce the probability of failure in photographing and reduce the user's effort and burden. As a result, the user terminal 10 can improve the user's convenience regarding photographing performed to calculate the head-related transfer function.

[0042] The overall flow of information processing according to the present disclosure has been outlined above. The configuration of the user terminal 10 will be described with reference to Fig. 3 and subsequent figures, and various types of processing will be described in detail in sequence.

[0043] [1-2. Configuration of the information processing device according to the embodiment] A configuration of a user terminal 10, which is an example of an information processing device according to the present disclosure, will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of a configuration of the user terminal 10 according to the embodiment. As shown in Fig. 3, the user terminal 10 has a communication unit 11, an input unit 12, a display unit 13, a detection unit 14, a storage unit 15, and a control unit 16.

[0044] The communication unit 11 is realized by, for example, a network interface card (NIC) etc. The communication unit 11 is connected to a network N (such as the Internet) by wire or wirelessly, and transmits and receives information to and from the cloud server 100, the service server 200, the headphones 50, etc. via the network N.

[0045] The input unit 12 is an input device that accepts various operations from a user. For example, the input unit 12 is realized by operation keys or the like provided on the user terminal 10. The display unit 13 is a display device for displaying various information. For example, the display unit 13 is realized by a liquid crystal display or the like. Note that, when a touch panel is adopted in the user terminal 10, a part of the input unit 12 and the display unit 13 are integrated together.

[0046] The detection unit 14 is a collective term for various sensors, and detects various information related to the user terminal 10. Specifically, the detection unit 14 detects user operations on the user terminal 10, location information of the location of the user terminal 10, information related to devices connected to the user terminal 10, the environment of the user terminal 10, and the like.

[0047] The detection unit 14 also has a lens and an image sensor for capturing images, as an example of a sensor. That is, the detection unit 14 functions as a camera when, for example, a user activates an application that operates a capturing function.

[0048] Furthermore, the detection unit 14 detects information related to the environment by utilizing various sensors and functions provided in the user terminal 10. For example, the detection unit 14 utilizes a microphone that collects sounds around the user terminal 10, an illuminance sensor that detects the illuminance around the user terminal 10, an acceleration sensor (or a gyro sensor, etc.) or a speed sensor that detects the physical movement of the user terminal 10, a geomagnetic sensor that detects the magnetic field at the location of the user terminal 10, etc. The detection unit 14 detects various information by using the various sensors.

[0049] The storage unit 15 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 15 stores various data used in information processing.

[0050] For example, the storage unit 15 stores captured images acquired using a camera function, etc. The storage unit 15 also stores information related to HRTF generated by the cloud server 100. The storage unit 15 may also store an image recognition trained model for recognizing a frontal view of a user's face, a side view of a user's face, a user's ears, and the like.

[0051] The control unit 16 is realized by, for example, a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc., executing a program (for example, an information processing program according to the present disclosure) stored inside the user terminal 10 using a random access memory (RAM) etc. as a working area. The control unit 16 is a controller, and may be realized by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0052] 3, the control unit 16 has an acquisition unit 161, an image capture processing unit 162, a transmission unit 167, and a reception unit 168, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the control unit 16 is not limited to the configuration shown in FIG. 3, and may be other configurations as long as they perform the information processing described below.

[0053] The acquisition unit 161 acquires various types of information. For example, the acquisition unit 161 acquires information detected by the detection unit 14. The acquisition unit 161 stores the acquired information in the storage unit 15 as appropriate.

[0054] The photographing processing unit 162 controls the photographing processing of the image used for generating the personalized HRTF. The photographing processing unit 162 includes a first determination unit 163, a second determination unit 164, and a photographing unit 165, and these processing units cooperate with each other to execute the photographing processing of the image.

[0055] The first determination unit 163 determines whether or not the size of the area occupied by the user's front facing face in the shooting area is within a threshold value.

[0056] The first determination unit 163 sets a first guide area for identifying an area occupied by the user's front-facing face in the shooting area, and determines whether or not the size of the area occupied by the user's front-facing face with respect to the first guide area is within a threshold. The first guide area corresponds to, for example, the inner part of the guide frame 22 shown in Fig. 2. By setting the first guide area, the first determination unit 163 can allow the user to intuitively understand what size and position the face should be when photographed on the camera.

[0057] Specifically, the first determination unit 163 determines whether or not the area extracted as the user's front-facing face is included in the captured area. The area extracted as the user's front-facing face is, for example, an area that is recognized as a front-facing face by using an arbitrary image recognition model. The area extracted as the user's front-facing face is shown, for example, by a rectangle of a predetermined size.

[0058] For example, the first determination unit 163 determines whether or not the area extracted as the user's front facing face is included in the captured area, based on the proportion of the area extracted as the user's front facing face included in the captured area.

[0059] In addition, the first determination unit 163 may determine whether or not the area extracted as the user's forward-facing face is included in the captured area based on the distance between the edge of the area extracted as the user's forward-facing face and the edge of the captured area.

[0060] The process executed by the first determination unit 163 will be described with reference to Fig. 4. Fig. 4 is a diagram illustrating the first determination process according to the embodiment.

[0061] 4, when the first determination unit 163 recognizes the front face of the user through image recognition, it extracts the area 25 as the user's front facing face. The first determination unit 163 determines whether the area occupied by the area 25 is within a threshold (e.g., occupies an area of ​​more than 60% but not more than 90%) with respect to the shooting area 21 or the first guide area (the inner part of the guide frame 22), and determines whether the user's front facing face is included in the shooting area 21. Note that the threshold may be changed as appropriate, for example, in response to a request from the cloud server 100.

[0062] Furthermore, the first determination unit 163 determines the distance between an end of the region 25 and an end of the shooting region 21. For example, the first determination unit 163 detects the number of pixels included between the end of the region 25 and the end of the shooting region 21, and determines that the region 25 is included in the shooting region 21 when the number of pixels exceeds a predetermined number.

[0063] Through the above process, the first determination unit 163 can determine whether the size of the user's face is extremely small or large, or whether the position of the user's face is outside the shooting area of ​​the camera.

[0064] 4, the area 25 extracted as the front face of the user is expressed as a rectangle, but the first determination unit 163 may extract the area 25 in a shape that is not limited to a rectangle and conforms to the front face of the user. Also, the area 25 may or may not be displayed on the screen and presented to the user.

[0065] When the first determination unit 163 determines that the size of the area occupied by the user's face facing forward is within the threshold, it outputs a response to the effect that the process will proceed to capturing an image of the user's face facing sideways. This allows the first determination unit 163 to shift to capturing an image of the sideways face after adjusting the positional relationship between the user and the user terminal 10 (imaging device) to an appropriate one.

[0066] Specifically, the first determination unit 163 outputs at least one of a voice, a vibration, or a display of a character string as a response. For example, the first determination unit 163 uses a TTS function to output a voice prompting the user to turn right or left to capture a side view. Alternatively, the first determination unit 163 may notify the user that the determination of the front face has been completed by outputting a sound effect indicating that the determination of the front face has been successful or vibrating the user terminal 10.

[0067] In addition, if the first judgment unit 163 determines that the size of the area occupied by the user's forward-facing face is not within the threshold, it may output a response to move the position of the user's face or to move the position of the device used to photograph the user.

[0068] In this case, the first determination unit 163 also outputs at least one of a voice, a vibration, and a display of a character string as a response. For example, the first determination unit 163 uses the TTS function to output a voice that prompts the user to move closer to the user terminal 10, speak, or move the position of the user or the user terminal 10 to the right or left.

[0069] When the first judgment unit 163 judges that the size of the area occupied by the user's face facing forward is within the threshold, the second judgment unit 164 judges whether the size of the area occupied by the user's face facing side in the shooting area is within the threshold.

[0070] The second determination unit 164 may set a second guide area for identifying an area occupied by the user's face in profile within the photographing area, and determine whether or not the size of the area occupied by the user's face in profile relative to the second guide area is within a threshold. The second guide area corresponds to, for example, the inner part of the guide frame 23 shown in FIG. 2. By setting the second guide area, the second determination unit 164 can allow the user to intuitively understand at what size and position the face should be captured on the camera.

[0071] The second determination unit 164 determines whether or not the area including the user's ear from the area extracted as the user's side-facing face is included in the captured area, and if the area including the user's ear is included in the captured area, determines that the size of the area occupied by the user's side-facing face is within a threshold.

[0072] Specifically, the second determination unit 164 determines whether or not the area including the user's ear is included in the imaging area based on the proportion of the area including the user's ear included in the imaging area.

[0073] In addition, the second determination unit 164 may determine whether or not the area including the user's ear is included in the captured area based on the proportion of the area including the user's ear to the area occupied by the user's face turned to the side.

[0074] The second determination section 164 may also determine whether or not the area including the user's ear is included in the imaging area, based on the distance between the edge of the area including the user's ear and the edge of the imaging area.

[0075] The process executed by the second determination unit 164 will be described with reference to Fig. 5. Fig. 5 is a diagram illustrating the second determination process according to the embodiment.

[0076] 5, when the second determination unit 164 recognizes the profile of the user's face through image recognition, it extracts an area 26 as the user's face facing sideways. The second determination unit 164 determines, for example, whether or not the area occupied by the area 26 is within a threshold value for the shooting area 21 or the second guide area (the inner portion of the guide frame 23), and determines whether or not the user's face facing sideways is included in the shooting area 21. Note that the threshold value may be changed as appropriate, for example, in response to a request from the cloud server 100.

[0077] The second determination unit 164 may also determine whether or not the area including the user's ears is included in the captured area based on the ratio of the area 26 occupied by the user's face viewed from the side to the area 27 including the user's ears. This is because the ratio of the size of the user's head on the side to the size of the ears in an image used to generate an HRTF affects the accuracy of the generated HRTF. By such processing, the second determination unit 164 can capture an image in which the size of the user's head and the size of the ears have an appropriate relationship.

[0078] The second determination unit 164 may also determine the distance between an end of the region 27 representing the ear and an end of the shooting region 21. For example, the second determination unit 164 detects the number of pixels included between the end of the region 27 and the end of the shooting region 21, and determines that the region 27 is included in the shooting region 21 when the number of pixels exceeds a predetermined number.

[0079] Through the above process, the second determination unit 164 can prevent an image that does not include the ears from being captured, thereby reducing the burden on the user associated with retaking the image, etc.

[0080] 4, region 26 extracted as the side of the user's face and region 27 including the ears are expressed as rectangles, but second determination unit 164 may extract region 26 and region 27 in shapes that are not limited to rectangles and that conform to the side of the user's face and ears. Region 26 and region 27 may or may not be displayed on the screen and presented to the user.

[0081] In addition, when the second determination unit 164 determines that the area including the user's ears among the areas extracted as the user's face facing the side is not included in the photographing area, the second determination unit 164 may output a response to move the position of the user's face or a response to move the position of the device used to photograph the user.

[0082] Specifically, the second determination unit 164 outputs at least one of a sound, a vibration, and a display of a character string as a response. For example, the second determination unit 164 can notify the user in an easy-to-understand manner as to whether or not the image capture is being performed appropriately by providing a response in a manner that the user can easily understand without looking at the screen, such as a sound or a vibration.

[0083] The photographing unit 165 photographs the user's face in profile when the second determination unit 164 determines that the size of the area occupied by the user's face in profile is within the threshold. The photographing unit 165 can photograph the user's face in profile at an appropriate timing by automatically performing photographing in response to the determination result by the second determination unit 164 without determining whether the user has pressed the shutter button.

[0084] Furthermore, the photographing unit 165 may photograph the user's face in profile when it is determined that the size of the area occupied by the user's face in profile is within the threshold value by the second determination unit 164 and then that the behavior amount of the device used to photograph the user is within the threshold value. In the embodiment, the device used to photograph the user refers to the user terminal 10. That is, the photographing unit 165 performs photographing when it receives the determination result by the second determination unit 164 and determines that the behavior amount (acceleration, speed, etc.) of the user terminal 10 is within the threshold value. This allows the photographing unit 165 to prevent photographing that causes blurring.

[0085] In addition, after the second determination unit 164 determines that the size of the area occupied by the user's side-on face is within a threshold, the photographing unit 165 may photograph the user's side-on face if it determines that the amount of movement of the user's side-on face in the photographing area is within a threshold.

[0086] That is, the photographing unit 165 compares the image at the timing when the determination result by the second determination unit 164 is received with the image at the next photographing interval, for example, the image after a predetermined number of frames, and determines that the amount of movement of the user's face or ears is equal to or less than a threshold. Such processing can be achieved, for example, by applying a tracking process of an object in the image. The photographing unit 165 performs photographing when it is determined that the amount of movement of the user is within the threshold, thereby preventing photographing that causes blurring.

[0087] Furthermore, the photographing unit 165 may edit the photographed image and perform pre-processing before transmitting the image to the cloud server 100. For example, the photographing unit 165 may edit the image so that the area including the user's ear is located at the center of the image. Furthermore, the photographing unit 165 may trim the area outside a predetermined range with the area including the user's ear as the center.

[0088] The transmission unit 167 transmits various types of information. For example, the transmission unit 167 transmits an image captured by the image capturing unit 165 to the cloud server 100.

[0089] The receiving unit 168 receives various information. For example, the receiving unit 168 receives the personalized HRTF generated by the cloud server 100.

[0090] [1-3. Configuration of cloud server according to embodiment] Next, the configuration of the cloud server 100 according to the embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram showing an example of the configuration of the cloud server 100 according to the embodiment.

[0091] 6, cloud server 100 includes a communication unit 110, a storage unit 120, and a control unit 130. Note that cloud server 100 may also include an input unit (e.g., a keyboard, a mouse, etc.) that accepts various operations from an administrator who manages cloud server 100, and a display unit (e.g., a liquid crystal display, etc.) that displays various information.

[0092] The communication unit 110 is realized by, for example, a NIC etc. The communication unit 110 is connected to a network N (such as the Internet) by wire or wirelessly, and transmits and receives information to and from the user terminal 10 etc. via the network N.

[0093] The storage unit 120 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 120 stores images acquired from the user terminal 10 and a model used to generate (calculate) HRTFs (for example, a trained model trained to receive an image as input and output an HRTF).

[0094] The control unit 130 is realized, for example, by a CPU, an MPU, a GPU, or the like, executing a program stored in the cloud server 100 using a RAM or the like as a working area. The control unit 130 is also a controller, and may be realized, for example, by an integrated circuit such as an ASIC or an FPGA.

[0095] 6, the control unit 130 has a receiving unit 131, a generating unit 132, and a providing unit 133, and realizes or executes the functions and actions of information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 6, and may be other configurations as long as they perform the information processing described below.

[0096] The reception unit 131 receives various types of information. For example, the reception unit 131 receives a request for HRTF generation processing from the user terminal 10. The reception unit 131 also receives from the user terminal 10 images to be used in the HRTF generation processing.

[0097] The generating unit 132 generates a personalized HRTF corresponding to an image based on the image accepted by the accepting unit 131. For example, the generating unit 132 generates the personalized HRTF using a trained model that has been trained to input an image and output an HRTF. Note that the trained model is not specified to a specific type, and for example, the generating unit 132 may generate the personalized HRTF using various models generated using various learning algorithms such as a neural network, a support vector machine, clustering, and reinforcement learning.

[0098] The generation unit 132 may transmit parameters related to the photographing process performed by the user terminal 10 to the user terminal 10. For example, the generation unit 132 transmits information specifying the size of the ears included in the image (for example, "an image in which the area including the ears is about 200×200 pixels in size"), and parameters such as the acceleration and speed of the user terminal 10 permitted for photographing, and the amount of movement of the user.

[0099] The providing unit 133 provides the personalized HRTF generated by the generating unit 132 to the user terminal 10. If the providing unit 133 is unable to generate a personalized HRTF based on an image received from the user terminal 10, the providing unit 133 may provide error information or the like.

[0100] [1-4. Information Processing Procedure According to the Embodiment] Next, the procedure of information processing according to the embodiment will be described with reference to Fig. 7 and Fig. 8. In Fig. 7, the procedure of processing in which the user terminal 10 executes the first determination processing will be described. Fig. 7 is a flowchart (1) showing the flow of information processing according to the embodiment.

[0101] 7, the user terminal 10 determines whether or not the front face of the user is detected within the viewing angle of the camera function (step S21). If the front face of the user is not detected (step S21; No), the user terminal 10 waits until the front face of the user is detected.

[0102] On the other hand, when the front face of the user is detected (Step S21; Yes), the user terminal 10 determines whether or not the size of the front face is appropriate in the shooting area (Step S22).

[0103] If the size of the frontal face is not appropriate (step S22; No), the user terminal 10 outputs a response such as a voice guide to prompt the user to adjust the size (step S23).

[0104] If the size of the user's frontal face is appropriate (Step S22; Yes), the user terminal 10 determines whether the position of the face with respect to the screen is appropriate (Step S24).

[0105] If the position of the frontal face is not appropriate (step S24; No), the user terminal 10 outputs a response such as a voice guide to prompt the user to adjust the position (step S25).

[0106] If the position of the user's frontal face is appropriate (step S24; Yes), the user terminal 10 determines that the recognition of the user's frontal face has been successful (step S26).Then, the user terminal 10 outputs feedback (sound, vibration, etc.) indicating that the recognition has been successful (step S27).

[0107] Next, a process procedure for the user terminal 10 to capture an image of the user's ear will be described with reference to Fig. 8. Fig. 8 is a flowchart (2) showing the flow of information processing according to the embodiment.

[0108] 8, the user terminal 10 determines whether or not the user's ear (side view of the face) is detected within the viewing angle of the camera function (step S31). If the user's ear is not detected (step S31; No), the user terminal 10 waits until the user's ear is detected.

[0109] On the other hand, when the user's ear is detected (Step S31; Yes), the user terminal 10 determines whether or not the size of the ear is appropriate in the shooting area (Step S32).

[0110] If the size of the ears is not appropriate (step S32; No), the user terminal 10 outputs a response such as a voice guide to prompt the user to adjust the size (step S33).

[0111] If the size of the user's ears is appropriate (Step S32; Yes), the user terminal 10 determines whether the position of the ears with respect to the screen is appropriate (Step S34).

[0112] If the ear position is not appropriate (step S34; No), the user terminal 10 outputs a response such as a voice prompt to prompt the user to adjust the ear position (step S35).

[0113] If the position of the user's ears is appropriate (step S34; Yes), the user terminal 10 determines whether or not the user terminal 10 has moved beyond a threshold value (step S36).

[0114] If the user terminal 10 is moving beyond the threshold (step S36; No), the user terminal 10 outputs a response such as a voice prompt to urge the user not to move the user terminal 10 (step S37).

[0115] If the user terminal 10 has not moved beyond the threshold (step S36; Yes), the user terminal 10 determines whether the positional deviation of the user's ears or head between frames to be captured is within a threshold (step S38).

[0116] If the positional deviation is not within the threshold value (step S38; No), the user terminal 10 outputs a response such as a voice prompt to urge the user not to move the face (step S39).

[0117] If the positional deviation is within the threshold (step S38; Yes), the user terminal 10 determines that the ear has been successfully recognized (step S40). Then, the user terminal 10 photographs the user's ear and acquires the photographed image (step S41).

[0118] Thereafter, the user terminal 10 outputs feedback to the user indicating that the ear has been photographed (step S42). The user terminal 10 then determines whether or not photographing of both ears has been completed (step S43). If both ears have not been photographed, that is, if photographing of only one ear has been completed (step S43; No), the user terminal 10 repeats the process of detecting the other ear (step S31). On the other hand, if photographing of both ears has been completed (step S43; Yes), the user terminal 10 ends the photographing process.

[0119] (2. Modifications) The above-described information processing system 1 may be implemented in various different forms other than the above-described embodiment. Therefore, modifications of the embodiment will be described below.

[0120] In the embodiment, an example has been described in which the cloud server 100 generates the personalized HRTF. However, the personalized HRTF may be generated by the user terminal 10 instead of the cloud server 100. In this case, the user terminal 10 includes a processing unit similar to the generation unit 132 shown in FIG. 6.

[0121] In the embodiment, it is assumed that the cloud server 100 is installed on a cloud network, but this is not limited to the example, and the cloud server 100 may be installed on a network such as a LAN (Local Area Network) as long as it is capable of communicating with the user terminal 10.

[0122] When generating a personalized HRTF, the cloud server 100 may acquire identification information of the user, associate the identification information with the personalized HRTF, and store the personalized HRTF.

[0123] In the embodiment, the cloud server 100 has been described as one server, but the cloud server 100 may be composed of multiple server devices. For example, the cloud server 100 may be divided into a generation server that generates personalized HRTFs and a provision server that provides personalized HRTFs.

[0124] In the embodiment, an example in which the user terminal 10 photographs the user has been shown, but the photographing may be performed by another photographing device. In this case, the photographing device sequentially transmits information of the image to be photographed to the user terminal 10. Then, the user terminal 10 executes the information processing according to the embodiment by recognizing the position of the user's face and ears in the received image. In addition, in the embodiment, it has been shown that the user terminal 10 performs the process of determining the size of the user's face in profile, but the user terminal 10 may skip the process of determining the size of the face and determine only the size of the ears. In other words, the user terminal 10 does not necessarily determine the size of the user's face in profile and then determine the size and position of the ears, and may directly determine the appropriateness of the size and position of the user's ears when possible.

[0125] The user terminal 10 may set the amount of light for the captured image, the tolerance for ear tilt, and the like, according to instructions from the cloud server 100. This allows the user terminal 10 to reduce the probability that the cloud server 100 will return an error message indicating that a personalized HRTF cannot be generated from the captured image.

[0126] (3. Other embodiments) The processing according to each of the above-described embodiments may be implemented in various different forms other than the above-described embodiments.

[0127] Furthermore, among the processes described in the above embodiments, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically by a known method. In addition, the information including the processing procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

[0128] In addition, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit according to various loads, usage conditions, etc.

[0129] Furthermore, the above-described embodiments and modifications can be appropriately combined as long as the processing contents are not contradictory.

[0130] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0131] (4. Effects of the information processing device according to the present disclosure) As described above, the information processing device according to the present disclosure (user terminal 10 in the embodiment) includes a first determination unit (first determination unit 163 in the embodiment), a second determination unit (second determination unit 164 in the embodiment), and an image capture unit (image capture unit 165 in the embodiment). The first determination unit determines whether or not the size of the area occupied by the user's front-facing face in the image capture area is within a threshold. When it is determined that the size of the area occupied by the user's front-facing face is within the threshold, the second determination unit determines whether or not the size of the area occupied by the user's side-facing face in the image capture area is within a threshold. When it is determined that the size of the area occupied by the user's side-facing face is within the threshold, the image capture unit captures the user's side-facing face.

[0132] In this way, the information processing device according to the present disclosure aligns the user's face from the front so that the user can easily see the user, and then captures the side of the user's head. This reduces the probability of image capture failure and reduces the user's effort and burden, thereby improving the user's convenience regarding image capture performed to calculate the head-related transfer function.

[0133] The first determination unit sets a first guide area for identifying an area occupied by the user's front-facing face in the shooting area, and determines whether or not the size of the area occupied by the user's front-facing face in relation to the first guide area is within a threshold value. This allows the information processing device to allow the user to intuitively understand at what size and position the face should be captured on the camera.

[0134] In addition, the first determination unit determines whether the area extracted as the user's face facing forward is included in the photographing area. This allows the information processing device to perform photographing processing after confirming that the user is in a position suitable for photographing, thereby reducing photographing failures.

[0135] The first determination unit determines whether the area extracted as the user's front-facing face is included in the captured area based on the proportion of the area extracted as the user's front-facing face included in the captured area. This allows the information processing device to accurately grasp information such as whether the user's location is far or close, allowing smooth capture of the image.

[0136] The first determination unit determines whether the area extracted as the user's front-facing face is included in the photographing area based on the distance between the edge of the area extracted as the user's front-facing face and the edge of the photographing area. This allows the information processing device to accurately grasp information such as whether the user's location is within or outside the photographing area, thereby enabling smooth photographing.

[0137] Furthermore, when the first determination unit determines that the size of the area occupied by the user's face facing forward is within the threshold, it outputs a response to the effect that the image will be captured of the user's face facing side. This allows the information processing device to capture the side image after confirming that the user has performed accurate alignment from the front, thereby improving the success rate of the image capture.

[0138] Furthermore, when the first determination unit determines that the size of the area occupied by the user's front-facing face is not within the threshold, it outputs a response to move the position of the user's face or a response to move the position of the device used to photograph the user. This allows the information processing device to give an appropriate response to the user, thereby enabling accurate and rapid photographing.

[0139] The first determination unit outputs at least one of a voice, a vibration, and a display of a character string as a response. Note that the voice is not limited to a message voice by TTS, and may include sound effects such as a shutter sound. This allows the information processing device to give a response to the user in various ways, and therefore, can accurately notify the user of necessary information.

[0140] The second determination unit sets a second guide area for identifying an area occupied by the user's face in profile within the photographing area, and determines whether or not the size of the area occupied by the user's face in profile relative to the second guide area is within a threshold value. This allows the information processing device to allow the user to intuitively understand at what size and position the face should be photographed on the camera.

[0141] The second determination unit determines whether the area including the user's ear is included in the captured area among the areas extracted as the user's face in profile, and if the area including the user's ear is included in the captured area, determines that the size of the area occupied by the user's face in profile is within the threshold. This allows the information processing device to reliably include the ear in the captured image, thereby obtaining an image suitable for generating a personalized HRTF.

[0142] The second determination unit determines whether the area including the user's ear is included in the captured area based on the ratio of the area including the user's ear to the captured area, thereby enabling the information processing device to obtain an image including the ear of a size suitable for the generation process of the personalized HRTF.

[0143] The second determination unit determines whether the area including the user's ear is included in the captured area based on the ratio of the area including the user's ear to the area occupied by the user's face in profile. This allows the information processing device to obtain an image in which the size of the user's head and the size of the ear have a relationship suitable for HRTF generation.

[0144] The second determination unit determines whether the area including the user's ear is included in the imaging area based on the distance between the edge of the area including the user's ear and the edge of the imaging area, thereby enabling the information processing device to prevent capturing an image in which HRTF generation is not possible because the ear is located at the edge of the image.

[0145] Furthermore, when the second determination unit determines that the area including the user's ears is not included in the photographing area among the areas extracted as the user's face facing the side, it outputs a response to move the position of the user's face or a response to move the position of the device used to photograph the user. This allows the information processing device to give an appropriate response to the user, thereby enabling accurate and rapid photographing.

[0146] In addition, the second determination unit outputs at least one of a voice, a vibration, and a display of a character string as a response, so that the information processing device can notify the user of an appropriate response even when the user cannot see the screen.

[0147] In addition, when the photographing unit determines that the size of the area occupied by the user's face in profile is within a threshold and then determines that the behavior amount of the device used to photograph the user is within a threshold, the photographing unit photographs the user's face in profile. This allows the information processing device to prevent a blurred image from being photographed.

[0148] Furthermore, after it is determined that the size of the area occupied by the user's side-facing face is within a threshold, the photographing unit photographs the user's side-facing face when it is determined that the amount of movement of the user's side-facing face in the photographing area is within a threshold. This allows the information processing device to prevent a blurred image from being photographed.

[0149] (5. Hardware Configuration) Information devices such as the user terminal 10 and cloud server 100 according to each embodiment described above are realized by a computer 1000 having a configuration as shown in FIG. 9, for example. The user terminal 10 according to the embodiment will be described below as an example. FIG. 9 is a hardware configuration diagram showing an example of a computer 1000 that realizes the functions of the user terminal 10. The computer 1000 has a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. Each unit of the computer 1000 is connected by a bus 1050.

[0150] The CPU 1100 operates and controls each unit based on programs stored in the ROM 1300 or the HDD 1400. For example, the CPU 1100 loads the programs stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processes corresponding to the various programs.

[0151] The ROM 1300 stores boot programs such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is started, and programs that depend on the hardware of the computer 1000.

[0152] HDD 1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of program data 1450.

[0153] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

[0154] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input / output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, or a printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading a program or the like recorded on a predetermined recording medium. The medium may be, for example, an optical recording medium such as a DVD (Digital Versatile Disc) or a PD (Phase change rewritable Disk), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

[0155] For example, when the computer 1000 functions as the user terminal 10 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize functions of the control unit 16, etc. Also, the information processing program according to the present disclosure and data in the storage unit 15 are stored in the HDD 1400. Note that the CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, these programs may be obtained from other devices via an external network 1550.

[0156] The present technology can also be configured as follows. (1) a first determination unit that determines whether or not a size of an area occupied by the user's front facing face in the photographing area is within a threshold; a second determination unit that, when it is determined that the size of the area occupied by the user's face facing forward is within a threshold, determines whether or not the size of the area occupied by the user's face facing side in the shooting area is within a threshold; an imaging unit that captures an image of the user's face in profile when it is determined that the size of the area occupied by the user's face in profile is within a threshold; An information processing device comprising: (2) The first determination unit is A first guide area for identifying an area occupied by the user's front-facing face is set within the photographing area, and it is determined whether or not the size of the area occupied by the user's front-facing face with respect to the first guide area is within a threshold value. The information processing device according to (1). (3) The first determination unit is It is determined whether the area extracted as the user's face facing forward is included in the photographed area. The information processing device according to (1) or (2). (4) The first determination unit is Based on a ratio of an area extracted as the user's face facing forward to being included in the captured area, it is determined whether or not the area extracted as the user's face facing forward is included in the captured area. The information processing device according to (3). (5) The first determination unit is Based on the distance between an edge of the area extracted as the user's face facing forward and an edge of the photographed area, it is determined whether or not the area extracted as the user's face facing forward is included in the photographed area. The information processing device according to (3) or (4). (6) The first determination unit is When it is determined that the size of the area occupied by the user's face facing forward is within a threshold, a response is output to the effect that the image will be captured of the user's face facing sideways. The information processing device according to any one of (1) to (5). (7) The first determination unit is When it is determined that the size of the area occupied by the front-facing face of the user is not within a threshold, a response to move the position of the face of the user or a response to move the position of a device used to photograph the user is output. The information processing device according to any one of (1) to (6). (8) The first determination unit is As the response, at least one of a sound, a vibration, and a display of a character string is output. The information processing device according to (6) or (7). (9) The second determination unit is A second guide area for identifying an area occupied by the user's face in profile is set within the photographing area, and it is determined whether or not the size of the area occupied by the user's face in profile relative to the second guide area is within a threshold value. The information processing device according to any one of (1) to (8). (10) The second determination unit is Among the areas extracted as the user's face in profile, it is determined whether or not an area including the user's ear is included in the photographed area, and if the area including the user's ear is included in the photographed area, it is determined that the size of the area occupied by the user's face in profile is within a threshold value. The information processing device according to any one of (1) to (9). (11) The second determination unit is Based on a ratio of the area including the user's ear included in the imaging area, it is determined whether or not the area including the user's ear is included in the imaging area. The information processing device according to (10). (12) The second determination unit is Based on a ratio of an area including the user's ear to an area occupied by the user's face in profile, it is determined whether or not an area including the user's ear is included in the photographed area. The information processing device according to (10) or (11). (13) The second determination unit is Based on a distance between an edge of the area including the user's ear and an edge of the imaging area, it is determined whether or not the area including the user's ear is included in the imaging area. The information processing device according to any one of (10) to (12). (14) The second determination unit is When it is determined that an area including the user's ears among the areas extracted as the user's face facing sideways is not included in the photographing area, a response is output to move the position of the user's face or to move the position of a device used to photograph the user. The information processing device according to any one of (10) to (13). (15) The second determination unit is As the response, at least one of a sound, a vibration, and a display of a character string is output. The information processing device according to (14). (16) The imaging unit includes: After it is determined that the size of the area occupied by the side-facing face of the user is within a threshold, if it is determined that the amount of behavior of the device used to photograph the user is within a threshold, the side-facing face of the user is photographed. The information processing device according to any one of (1) to (15). (17) The imaging unit includes: After it is determined that the size of the area occupied by the side-facing face of the user is within a threshold, if it is determined that the amount of movement of the side-facing face of the user in the photographing area is within a threshold, the side-facing face of the user is photographed. The information processing device according to any one of (1) to (16). (18) The computer determining whether the size of the area occupied by the user's front-facing face in the photographing area is within a threshold; When it is determined that the size of the area occupied by the user's face facing forward is within a threshold, it is determined whether or not the size of the area occupied by the user's face facing side in the photographing area is within a threshold; When it is determined that the size of the area occupied by the side-facing face of the user is within a threshold, the side-facing face of the user is photographed. Information processing methods. (19) Computer, a first determination unit that determines whether or not a size of an area occupied by the user's front facing face in the photographing area is within a threshold; a second determination unit that, when it is determined that the size of the area occupied by the user's face facing forward is within a threshold, determines whether or not the size of the area occupied by the user's face facing side in the shooting area is within a threshold; an imaging unit that captures an image of the user's face when it is determined that the size of the area occupied by the user's face is within a threshold; An information processing program that functions as a [Explanation of symbols]

[0157] 1. Information Processing Systems 10 User terminal 11 Communications Department 12 Input section 13 Display section 14 Detection unit 15 Storage section 16 Control section 161 Acquisition Department 162 Shooting Processing Unit 163 1st Judgment Department 164 Second Judgment Department 165 Filming Department 167 Transmitter 168 Receiver 100 Cloud Servers

Claims

1. a first determination unit that determines whether or not a size of an area occupied by a front-facing face of the user in a first imaging area, which is a screen that displays an imaging range of an imaging device that images the user, is within a threshold; a second determination unit that, when it is determined that the size of the area occupied by the user's face facing forward is within a threshold, determines whether the size of the area occupied by the user's face facing side in a second imaging area, which is a screen switched from the first imaging area, is within a threshold and whether the image capture device is moving beyond a threshold; an imaging unit that captures an image of the user's face in profile when it is determined that the size of the area occupied by the user's face in profile is within a threshold and, after that, it is determined that the movement of the imaging device does not exceed the threshold; a communication unit that transmits the captured image of the user's face in a side view to a cloud server and receives a head related transfer function corresponding to the captured image of the user's face in a side view from the cloud server; An information processing device comprising:

2. The first determination unit is A first guide area for identifying an area occupied by the user's forward-facing face is set in the first shooting area, and it is determined whether or not the size of the area occupied by the user's forward-facing face with respect to the first guide area is within a threshold value. The information processing device according to claim 1 .

3. The first determination unit is It is determined whether the area extracted as the user's face facing forward is included in the first imaging area. The information processing device according to claim 1 .

4. The first determination unit is Based on a ratio of an area extracted as the user's face facing forward to being included in the first imaging area, it is determined whether or not the area extracted as the user's face facing forward is included in the first imaging area. The information processing device according to claim 3 .

5. The first determination unit is Based on a distance between an edge of the area extracted as the user's face facing forward and an edge of the first imaging area, it is determined whether or not the area extracted as the user's face facing forward is included in the first imaging area. The information processing device according to claim 3 .

6. The first determination unit is When it is determined that the size of the area occupied by the user's face facing forward is within a threshold, a response is output to the effect that the image will be captured of the user's face facing sideways. The information processing device according to claim 1 .

7. The first determination unit is When it is determined that the size of the area occupied by the front-facing face of the user is not within a threshold, a response to move the position of the face of the user or a response to move the position of the imaging device used to photograph the user is output. The information processing device according to claim 1 .

8. The first determination unit is As the response, at least one of a sound, a vibration, and a display of a character string is output. The information processing device according to claim 6.

9. The second determination unit is A second guide area for identifying an area occupied by the user's face in profile is set in the second shooting area, and it is determined whether or not the size of the area occupied by the user's face in profile relative to the second guide area is within a threshold value. The information processing device according to claim 1 .

10. The second determination unit is Among the areas extracted as the user's face facing sideways, it is determined whether or not an area including the user's ear is included in the second imaging area, and if the area including the user's ear is included in the second imaging area, it is determined that the size of the area occupied by the user's face facing sideways is within a threshold value. The information processing device according to claim 1 .

11. The second determination unit is Based on a ratio of an area including the user's ear included in the second imaging area, it is determined whether or not the area including the user's ear is included in the second imaging area. The information processing device according to claim 10.

12. The second determination unit is and determining whether or not an area including the user's ear is included in the second imaging area based on a ratio of an area including the user's ear to an area occupied by the user's face facing sideways. The information processing device according to claim 10.

13. The second determination unit is Based on a distance between an end of the area including the user's ear and an end of the second imaging area, it is determined whether or not the area including the user's ear is included in the second imaging area. The information processing device according to claim 10.

14. The second determination unit is When it is determined that an area including the user's ears among the areas extracted as the user's face facing the side is not included in the second shooting area, a response to move the position of the user's face or a response to move the position of the imaging device used to shoot the user is output. The information processing device according to claim 10.

15. The second determination unit is As the response, at least one of a sound, a vibration, and a display of a character string is output. The information processing device according to claim 14.

16. The imaging unit includes: After it is determined that the size of the area occupied by the side-facing face of the user is within a threshold, if it is determined that the amount of movement of the imaging device used to photograph the user is within a threshold, the side-facing face of the user is photographed. The information processing device according to claim 1 .

17. The imaging unit includes: After it is determined that the size of the area occupied by the side-facing face of the user is within a threshold, if it is determined that the amount of movement of the side-facing face of the user in the second imaging area is within a threshold, the side-facing face of the user is photographed. The information processing device according to claim 1 .

18. The imaging unit includes: When it is determined that the amount of movement of the imaging device exceeds a threshold in the second imaging area, a response is output to prompt the imaging device not to move. The information processing device according to claim 1 .

19. The computer determining whether or not a size of an area occupied by a front-facing face of the user is within a threshold in a first imaging area, which is a screen displaying an imaging range of an imaging device that images the user; when it is determined that the size of the area occupied by the user's face facing forward is within a threshold, determine whether the size of the area occupied by the user's face facing side in a second imaging area, which is a screen switched from the first imaging area, is within a threshold and whether the movement of the imaging device exceeds a threshold; After it is determined that the size of the area occupied by the side view of the user is within a threshold, if it is determined that the movement of the imaging device does not exceed the threshold, photograph the side view of the user's face; Transmitting the captured image of the user's face in profile to a cloud server, and receiving a head-related transfer function corresponding to the captured image of the user's face in profile from the cloud server. Information processing methods.

20. Computer, a first determination unit that determines whether or not a size of an area occupied by a front-facing face of the user in a first imaging area, which is a screen that displays an imaging range of an imaging device that images the user, is within a threshold; a second determination unit that, when it is determined that the size of the area occupied by the user's face facing forward is within a threshold, determines whether the size of the area occupied by the user's face facing side in a second imaging area, which is a screen switched from the first imaging area, is within a threshold and whether the image capture device is moving beyond a threshold; an imaging unit that captures an image of the user's face in profile when it is determined that the size of the area occupied by the user's face in profile is within a threshold and, after that, it is determined that the movement of the imaging device does not exceed the threshold; a communication unit that transmits the captured image of the user's face in a side view to a cloud server and receives a head related transfer function corresponding to the captured image of the user's face in a side view from the cloud server; An information processing program that functions as a

Images