Information processing device, information processing method and information processing program

Abstract

A user information processing device, an information processing method, and an information processing program are proposed that can improve the convenience of shooting for calculating a head-related transfer function. An information processing device (10) according to the present disclosure includes: a determination unit (164) that determines whether the size of the area occupied by the side face of the user in the photographic area is within a threshold; a shooting unit (165) that shoots the side face of the user when it is determined that the size of the area occupied by the side face of the user is within the threshold; and a shooting control unit (166) that presents a manual shooting function to the user when the side face of the user has not been shot within a predetermined period of time.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, an information processing method, and an information processing program. Specifically, the present disclosure relates to using a head-related transfer function. Background Art

[0002] The technique of reproducing sound images in stereo, such as through headphones, is used by using a head-related transfer function that mathematically represents how sound reaches the ears from the sound source.

[0003] Since individual differences in head-related transfer functions are large, it is desirable to use a head-related transfer function generated for each individual when used. For example, a technology is known that generates a three-dimensional digital model of a head (hereinafter, may be referred to as a 3D model) based on an image obtained by photographing the user's auricle and calculating the user's head-related transfer function from the 3D model.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: U.S. Patent No. 9544706 Summary of the invention

[0007] Technical issues

[0008] According to the related art, since the head-related transfer function calculated independently for each user can be used for information processing, the sense of localization of the sound image can be enhanced.

[0009] However, in the above related art, it is difficult to improve the convenience of the user. For example, in the related art, the head-related transfer function is calculated using an image capturing the auricle of the user; however, since it is difficult for the user to photograph the auricle by himself or herself, it is assumed that the photographing may fail or the photographed image may not include the auricle of an appropriate size. In this case, the user may be required to retake the photographing multiple times, or the head-related transfer function may not be calculated with high accuracy.

[0010] Therefore, the present disclosure proposes an information processing apparatus, an information processing method, and an information processing program capable of improving the convenience of a user for photographing performed for calculating a head-related transfer function.

[0011] Solution to the problem

[0012] In order to solve the above problems, an information processing device according to an embodiment of the present disclosure includes: a determination unit, which determines whether the size of the area occupied by the side face of the user in the shooting area is within a threshold; a shooting unit, which shoots the side face of the user when it is determined that the size of the area occupied by the side face of the user is within the threshold; and a shooting control unit, which presents a manual shooting function to the user when the side face of the user has not been photographed within a predetermined period of time. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a diagram illustrating an information processing system according to an embodiment of the present disclosure.

[0014] Figure 2 is a conceptual diagram illustrating information processing according to an embodiment.

[0015] Figure 3 is a diagram illustrating a configuration example of a user terminal according to an embodiment.

[0016] Figure 4 : is a diagram for explaining the first determination process according to the embodiment.

[0017] Figure 5 is a diagram illustrating a second determination process according to the embodiment.

[0018] Figure 6 is a diagram illustrating a configuration example of a cloud server according to an embodiment.

[0019] Figure 7 is a flowchart (1) illustrating the procedure of information processing according to the embodiment.

[0020] Figure 8 is a flowchart (2) illustrating the procedure of information processing according to the embodiment.

[0021] Fig. 9 is a flowchart (1) illustrating a modification example of the procedure of information processing according to the embodiment.

[0022] Fig.10 is a flowchart (2) illustrating a modification example of the procedure of information processing according to the embodiment.

[0023] Fig.11 is a flowchart (3) illustrating a modification example of the procedure of information processing according to the embodiment.

[0024] Fig.12 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the information processing device. DETAILED DESCRIPTION

[0025] Hereinafter, embodiments of the present disclosure will be described in detail based on the accompanying drawings. Note that in each of the following embodiments, the same parts are denoted by the same symbols, and redundant descriptions will be omitted.

[0026] The present disclosure will be described in the order of the following items:

[0027] 1. Example

[0028] 1-1. Configuration of information processing system according to the embodiment and example of information processing

[0029] 1-2. Configuration of information processing device according to embodiment

[0030] 1-3. Configuration of the cloud server according to the embodiment

[0031] 1-4. Process of information processing according to the embodiment

[0032] 2. Modifications

[0033] 3. Other embodiments

[0034] 4. Effects of the information processing device according to the present disclosure

[0035] 5. Hardware Configuration

[0036] (1. Example)

[0037] [1-1. Configuration of information processing system according to the embodiment and example of information processing]

[0038] First, refer to Figure 1 and Figure 2 A configuration of the information processing system 1 according to the present disclosure and an overview of information processing performed by the information processing system 1 are described. Figure 1 1 is a diagram illustrating an information processing system 1 according to an embodiment of the present disclosure. Figure 1 The information processing system 1 shown in FIG. Figure 1 As shown in FIG, the information processing system 1 includes a user terminal 10 and a cloud server 100. The devices included in the information processing system 1 communicate with each other using a wired or wireless network (not shown). Note that the number of devices included in the information processing system 1 is not limited to the number of devices shown.

[0039] The user terminal 10 is an example of an information processing device according to the present disclosure, and is an information processing terminal that performs information processing using a head-related transfer function (hereinafter, may be referred to as HRTF). Specifically, the user terminal 10 can reproduce or implement highly effective 3D sound with a high sense of reality by using HRTF when reproducing music, moving images, etc. The user terminal 10 is implemented by, for example, a smartphone with a camera function, etc.

[0040] The cloud server 100 calculates HRTF corresponding to each user, and provides the calculated HRTF to the user terminal 10. The cloud server 100 acquires an image obtained by photographing the user's ear from the user terminal 10 via the network, and calculates the user's HRTF based on the acquired image.

[0041] As described above, the user terminal 10 performs information processing using HRTF. HRTF represents the sound changes caused by surrounding objects including the shape of a person's head or auricle (ear shell), etc. as a transfer function. Generally, measurement data for deriving HRTF is obtained by measuring an acoustic signal for measurement using a microphone worn in the auricle by a person, a virtual head-mounted microphone, etc.

[0042] For example, HRTF used in technologies such as 3D sound is often calculated using measurement data obtained through a virtual head-mounted microphone, etc., an average value of measurement data obtained from a large number of people, etc. However, since individual differences in HRTF are large, in order to achieve a more effective sound performance effect, it is desirable to use the user's own HRTF. That is, by replacing the general HRTF with the user's own HRTF, it becomes possible to provide the user with a sound experience with a higher sense of reality.

[0043] Commonly, as an example of a method of calculating the HRTF of an individual user, there is a method of calculating the HRTF of an individual user based on an image obtained by photographing the shape of the user's ear. For example, the cloud server 100 acquires an image including the shape of the user's ear photographed by the user terminal 10, and calculates the HRTF of the individual user based on the acquired image. Then, the cloud server 100 transmits the calculated HRTF to the user terminal 10. As a result, the user terminal 10 can reproduce a sound source in which 3D audio is recorded using the HRTF of the individual user, or can reproduce a moving image with realism.

[0044] However, there are several disadvantages in implementing the above method. For example, the cloud server 100 acquires an image from which the ear shape of the user can be recognized, and calculates the HRTF based on the acquired image. In this case, the user needs to capture an image including the user's own ear and upload the captured image to the cloud server 100. At this time, it is assumed that the user performs shooting using a smartphone like the user terminal 10 having a shooting function.

[0045] However, in the case where the user shoots the side face by himself or herself, it is difficult for the user to properly shoot the ears because the user cannot confirm the shooting area (i.e., the screen of the user terminal 10). Therefore, even if the user shoots the side face, there is a high possibility that the ears are not included in the image or an image in which the ears are excessively tilted is obtained. In these cases, the user may be required to reshoot multiple times, or the HRTF calculated with high accuracy may not be obtained. In addition, it may be difficult for the user to press the shutter button at an appropriate timing depending on the shape of the user terminal 10. In addition, in such a selfie, it is assumed that the user himself or herself shoots with the user terminal 10 in his or her hand, so there is a high possibility that blurring will appear in the captured image.

[0046] As described above, there are various disadvantages regarding the shooting performed for calculating the HRTF. The user terminal 10 according to the embodiment solves the above problems through the information processing of the present disclosure described below. Figure 1 and Figure 2 An overview of information processing according to the present disclosure is described along the flow.

[0047] First, refer to Figure 1 The following describes an overview of the flow of HRTF generation processing by the information processing system 1. Figure 1 As illustrated in FIG. 1 , in order to acquire an image including an ear image of the user, the user photographs himself from one side of the head (step S1 ).

[0048] Note that the reference Figure 2 A detailed description of the photographing process performed by the user terminal 10 is described. In addition, in the present disclosure, the image is not limited to a two-dimensional color image that can be generally photographed by the user terminal 10, etc., and may be a monochrome image, a depth image including depth information, or any combination thereof. In addition, the image used in the process of the present disclosure may be a plurality of images, not one image.

[0049] The user terminal 10 acquires the image 5 obtained by photographing the side face of the user's head in step S1 , and the shape of the user's ear can be recognized from the image (step S2 ).

[0050] Subsequently, the user terminal 10 transmits the acquired image 5 to the cloud server 100 (step S3). The cloud server 100 acquires the image transmitted from the user terminal 10. Then, the cloud server 100 calculates the user's HRTF from the image (step S4).

[0051] Note that various methods may be employed as a method for the cloud server 100 to calculate HRTF. For example, the cloud server 100 may calculate HRTF by using a learned model so as to output an HRTF corresponding to an ear when a two-dimensional image including an ear image is input. Alternatively, the cloud server 100 may generate a 3D model of the user's head from a two-dimensional image including an ear image, and calculate HRTF using the generated 3D model. In addition, hereinafter, the HRTF calculated for an individual user is referred to as a personal HRTF to be distinguished from a general HRTF.

[0052] In addition, the cloud server 100 can convolve the room impulse response (RIR) into the calculated personal HRTF to generate a binaural room impulse response (BRIR). In the present specification, HRTF includes not only HRTF but also information such as BRIR generated using HRTF. For example, there is a case where the data used when reproducing music in the user terminal 10 is not actually HRTF but BRIR; however, the two can be used interchangeably below. For example, the description "sending the calculated HRTF" also includes the case of sending the calculated BRIR. In addition, the description "reproducing using HRTF" also includes the case of reproducing using BRIR.

[0053] Thereafter, the cloud server 100 transmits the generated personal HRTF to the user terminal 10 (step S5). Then, the user terminal 10 acquires the personal HRTF. In this way, the user acquires his or her personal HRTF by photographing his or her side profile and transmitting the image to the cloud server 100. Thereafter, for example, when reproducing 3D audio or a multi-channel sound source, the user terminal 10 can use the personal HRTF corresponding to the user.

[0054] Next, we will refer to Figure 2 An overview of information processing (photographing processing) according to the present disclosure by the user terminal 10 will be described. Figure 2 is a conceptual diagram illustrating information processing according to an embodiment. Figure 2 is a diagram illustrating an image displayed on the screen of the user terminal 10 when the user performs self-photography. In other words, Figure 2 The screen of the user terminal 10 illustrated in FIG. 1 indicates the area to be photographed by the user terminal 10. Figure 2 In the description, in the case of illustrating the state transition of the user terminal 10, the screen display state of the user terminal 10 is described as a first state, a second state, ..., and an Nth state (N is any number). In addition, in the following description, a known image recognition method can be appropriately used as a method of recognizing the front face or side face of the user or the user's ear.

[0055] In the case where the user prefers to generate a personal HRTF, the user enables the shooting function and shoots the user's own ear. For example, the shooting function is implemented by an application (hereinafter, may be simply referred to as an application) provided by a business operator that manages the cloud server 100.

[0056] When the user starts the application, the user terminal 10 transitions to the first state. Figure 2 The photographing area 21 illustrated in FIG. 1 is an area to be imaged by the user terminal 10. In other words, the photographing area 21 displays the space captured by the camera included in the user terminal 10 on the screen. Figure 2 As shown in the figure, the user terminal 10 first outputs a voice A01 prompting the user to take an image of the user facing the front (step S11). For example, the user terminal 10 outputs a voice A01 with a content such as "show your front face and put it in the frame". In this way, the user terminal 10 provides appropriate information or feedback to the user by appropriately performing output through text-to-speech (TTS). It is noted that such response processing is not limited to voice and can be performed by methods such as vibration of the user terminal 10 or displaying a character string on the screen. In addition, the user terminal 10 can output not only TTS but also shutter sounds, sound effects indicating that appropriate shooting has been performed, etc. as audio.

[0057] That is, the user terminal 10 first outputs a response for adjusting the position of the user's front face, rather than prompting the user to immediately shoot a side face. In the case of a front face image, the user can recognize the position or size of his or her face while holding the user terminal 10 in his or her hand. In this way, the user terminal 10 first displays the user's front face on the screen, and performs adjustments so that the user is located at a position suitable for shooting. As a result, the user terminal 10 can increase the success probability of subsequent shooting of the side of the head.

[0058] In the first state, the user terminal 10 determines whether the position or size of the user's face is located at a position suitable for shooting in the shooting area 21. At this time, the user terminal 10 can display the guide frame 22 on the screen for the user to adjust the position or size of the front face. As a result, the user can adjust the position of the user's own face or the position of the user terminal 10 held in the hand while referring to the guide frame 22.

[0059] The user terminal 10 determines whether the size of the area occupied by the front face of the user is within the threshold value in the shooting area 21. At this time, the user terminal 10 can output a response appropriately according to the position or size of the user's face. For example, in the case where the size of the user's face displayed relative to the shooting area 21 or the guide frame 22 is too small, the user terminal 10 can output a voice A02 such as "Move your camera closer to you" to prompt the user to move the face or the user terminal 10 to an appropriate position.

[0060] If it is determined that the size of the area occupied by the front face of the user in the photographing area 21 is within the threshold, the user terminal 10 transitions to the second state (step S12 ).

[0061] In the second state, the user terminal 10 notifies the user that the position of the user's face has been located at a proper shooting point by, for example, changing the color of the guide frame 22 or outputting a voice A03 such as “properly located” (step S13 ).

[0062] Then, the user terminal 10 outputs a voice A04 such as “Turn right slowly” to prompt the user to turn to one side with respect to the camera (ie, the user terminal 10 ).

[0063] After outputting the voice A04 , the user terminal 10 transitions to the third state (step S14 ) In the third state, the user terminal 10 erases the guide frame 22 and newly displays the guide frame 23 for the side face in the photographing area 21 .

[0064] The user terminal 10 determines whether the size of the area occupied by the user's side face is within a threshold value in the shooting area 21. More specifically, the user terminal 10 determines whether the area including the user's ears is in an appropriate position relative to the shooting area 21. The appropriate position means, for example, that there is no portion of the area including the ears that deviates from the end of the shooting area 21, the ratio of the area including the ears to the shooting area 21 is within a threshold value, or the ratio of the area including the ears to the side of the user's head is within a threshold value. In other words, the appropriate position is a position in which the user's ears are estimated to be included in the captured image so as not to be detrimental to the cloud server 100 generating the HRTF.

[0065] In the case where the position of the side of the user's head or ear is inappropriate, the user terminal 10 can output a response appropriately. For example, in the case where the area including the user's ear deviates from the shooting area 21 or the guide frame 23, the user terminal 10 can output a voice A05 such as "Turn your camera slightly to the right" to prompt the user to move the face or the user terminal 10 to an appropriate position.

[0066] Thereafter, when it is determined that the size of the area occupied by the user's side face is within the threshold value in the photographing area 21, in other words, when the area including the user's ear is in an appropriate position, the user terminal 10 outputs a voice A06 such as "Ear detected". (Step S15)

[0067] The user terminal 10 also determines whether the amount of movement of the user terminal 10 (e.g., the acceleration or speed of the user terminal 10 itself) is within a threshold, whether the position of the user's ear once detected moves more than a threshold in the next frame, etc. Then, when the amount of movement of the user terminal 10 or the amount of movement of the user's ear position is within the threshold, the user terminal 10 automatically captures one side of the user's head. Note that such processing is performed to prevent blur in the captured image.

[0068] The user terminal 10 may present a manual shooting function to the user in the case where the user's side face has not been properly photographed. For example, the user terminal 10 may present a manual shooting function to the user in the case where the user's side face has not been properly photographed within a predetermined period starting from the determination process of photographing an image of the user facing forward or the photographing process of photographing an image of the user's side face. Then, the user terminal 10 photographs the image of the user's side face by accepting the user's operation. At this time, the user terminal 10 may display a screen including the guide frame 23 and accept manual shooting by the user. Note that the predetermined period mentioned here may be any time period such as 60 seconds or 120 seconds.

[0069] In addition, the user terminal 10 can accept user operations such as adjusting the position of the side face image of the user manually captured by the user. For example, the user terminal 10 receives operations such as dragging, pinching, or stretching as user operations, thereby adjusting the position, size, etc. of the captured side face image of the user. In this way, the user terminal 10 can accept the user's operation of adjusting the position, size, etc. of the image of the side face of the user so as to fit into the predetermined area including the ear in the guide frame 23.

[0070] In addition, the user terminal 10 can accept any operation of the user. For example, the user terminal 10 accepts a rotation operation as the user's operation, thereby adjusting the inclination of the captured user's side face image. In this way, the user terminal 10 can accept the user's operation of adjusting the inclination of the user's side face image so as to be suitable for a predetermined area including the ear.

[0071] Furthermore, the user terminal 10 may present sample images of faces turned to one side to the user according to the user's operation, the number of manual shooting failures, etc. By presenting the sample images, the user can easily understand what type of image of the face turned to one side should be captured.

[0072] The user terminal 10 can adjust the predetermined period according to the user's predetermined operation. For example, the user terminal 10 can modify the predetermined period when the shooting process is interrupted or when the user inputs an operation to return to the previous process in the shooting process. As a specific example, when the predetermined period is 60 seconds, the user terminal 10 accepts the operation of returning within 60 seconds. Then, in the case where the period until the return operation is received is already 30 seconds, the user terminal 10 can set the predetermined period to 30 seconds. It is noted that in the case where the user terminal 10 receives the predetermined operation a predetermined number of times or more, the predetermined period can be set to short according to the number of predetermined operations.

[0073] As described above, the user terminal 10 as an example of an information processing device according to the present disclosure determines whether the size of the area occupied by the side face of the user is within the threshold value in the shooting area 21, and shoots the side face of the user when it is determined that the size of the area occupied by the side face of the user is within the threshold value. Then, if the side face of the user has not been shot within a predetermined period of time, the user terminal 10 presents the user with a manual shooting function.

[0074] As described above, in the case where the user terminal 10 cannot determine that the size of the area occupied by the user's side face is within the threshold value in the shooting area 21, the user terminal 10 can prompt the user to manually shoot one side of the head. With this configuration, the user terminal 10 can be prevented from returning to the initial stage of the shooting process that may occur due to the inability to determine the user's side face, and can be prevented from falling into a processing loop of determining the user's front face and side face again, so the user's trouble or burden can be reduced. Therefore, the user terminal 10 can improve the user convenience regarding the shooting performed for calculating the head-related transfer function.

[0075] The above has described the overview of the overall process of information processing according to the present disclosure. Figure 3 The configuration of the user terminal 10 is described, and details of various processes will be described in sequence.

[0076] [1-2. Configuration of information processing device according to embodiment]

[0077] Reference Figure 3 A configuration of the user terminal 10 as an example of the information processing device according to the present disclosure is described. Figure 3 1 is a diagram illustrating a configuration example of a user terminal 10 according to an embodiment. Figure 3 As illustrated in FIG. 1 , the user terminal 10 includes a communication unit 11 , an input unit 12 , a display unit 13 , a detection unit 14 , a storage unit 15 , and a control unit 16 .

[0078] The communication unit 11 is implemented by, for example, a network interface card (NIC), etc. The communication unit 11 is connected to a network N (such as the Internet) in a wired or wireless manner, and sends information to the cloud server 100, the service server 200, the headset 50, etc. via the network N, and receives information from the cloud server 100, the service server 200, the headset 50, etc.

[0079] The input unit 12 is an input device for receiving various operations from the user. For example, the input unit 12 is implemented by an operation key or the like included in the user terminal 10. The display unit 13 is a display device for displaying various types of information. For example, the display unit 13 is implemented by a liquid crystal display or the like. Note that, in the case where the user terminal 10 adopts a touch panel, parts of the input unit 12 and the display unit 13 are integrated.

[0080] The detection unit 14 is a general term for various sensors, and detects various types of information about the user terminal 10. Specifically, the detection unit 14 detects the user's operation of the user terminal 10, the location information of the user terminal 10, the information about the device connected to the user terminal 10, the environment of the user terminal 10, and the like.

[0081] In addition, the detection unit 14 includes a lens and an image sensor for photographing as an example of a sensor. That is, for example, when the user activates an application operating a photographing function, the detection unit 14 functions as a camera.

[0082] In addition, the detection unit 14 detects information about the environment using various sensors and functions included in the user terminal 10. For example, the detection unit 14 uses a microphone that collects sounds around the user terminal 10, an illuminance sensor that detects illuminance around the user terminal 10, an acceleration sensor (or a gyro sensor, etc.) or a velocity sensor that detects physical movement of the user terminal 10, a geomagnetic sensor that detects a magnetic field at the location of the user terminal 10, etc. Then, the detection unit 14 detects various types of information using various sensors.

[0083] The storage unit 15 is realized by, for example, a semiconductor storage element such as a random access memory (RAM) and a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 15 stores various types of data used for information processing.

[0084] For example, the storage unit 15 stores captured images acquired using a camera function, etc. The storage unit 15 also stores information about HRTF generated by the cloud server 100. In addition, the storage unit 15 may store an image recognition learning model for recognizing a user's front face, an image recognition learning model for recognizing a user's side face or a user's ear, etc.

[0085] The control unit 16 is implemented by, for example, a central processing unit (CPU), a microprocessing unit (MPU), a graphics processing unit (GPU), etc. executing a program (for example, an information processing program according to the present disclosure) stored inside the user terminal 10 using a random access memory (RAM) or the like as a work area. The control unit 16 is also a controller and can be implemented by, for example, an integrated circuit such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0086] like Figure 3 As shown in the figure, the control unit 16 includes an acquisition unit 161, a shooting processing unit 162, a sending unit 167 and a receiving unit 168, and implements or executes the following information processing functions or actions. Note that the internal configuration of the control unit 16 is not limited to Figure 3 The configuration shown in the figure can be another configuration as long as the following information processing is performed.

[0087] The acquisition unit 161 acquires various types of information. For example, the acquisition unit 161 acquires information detected by the detection unit 14. The acquisition unit 161 stores the acquired information in the storage unit 15 as appropriate.

[0088] The photographing processing unit 162 controls the photographing process of the image for generating the personal HRTF. The photographing processing unit 162 includes a first determination unit 163, a second determination unit 164 (corresponding to the determination unit), a photographing unit 165, and a photographing control unit 166, and the processing units perform the photographing process of the image in cooperation with each other.

[0089] The first determination unit 163 determines whether the size of the area occupied by the front face of the user is within a threshold value in the photographing area.

[0090] Note that the first determination unit 163 sets a first guide area for specifying the area occupied by the front face of the user in the photographing area, and determines whether the size of the area occupied by the front face of the user relative to the first guide area is within a threshold. The first guide area corresponds to, for example, Figure 2 The inside of the guide frame 22 is shown in FIG. By setting the first guide area, the first determination unit 163 can make the user intuitively understand what size or where the face should be captured by the camera.

[0091] Specifically, the first determination unit 163 determines whether the area extracted as the front face of the user is included in the photographing area. The area extracted as the front face of the user means, for example, an area recognized as a front face by using a desired image recognition model. The area extracted as the front face of the user is indicated by, for example, a rectangle having a predetermined size.

[0092] For example, the first determination unit 163 determines whether the area extracted as the front face of the user is included in the photographing area based on the ratio at which the area extracted as the front face of the user is included in the photographing area.

[0093] Furthermore, the first determination unit 163 may determine whether the region extracted as the front face of the user is included in the photographing region based on the distance between the end of the region extracted as the front face of the user and the end of the photographing region.

[0094] Reference Figure 4 The processing performed by the first determination unit 163 will be described. Figure 4 : is a diagram for explaining the first determination process according to the embodiment.

[0095] like Figure 4 , in the case of recognizing an image of the front face of the user, the first determination unit 163 extracts the region 25 as the front face of the user. For example, the first determination unit 163 determines whether the area occupied by the region 25 is within a threshold value relative to the photographing region 21 or the first guide region (the inside of the guide frame 22) (such as, the occupied area exceeds 60%, but does not exceed 90%), and determines whether the front face of the user is included in the photographing region 21. Note that, for example, in response to a request from the cloud server 100, the threshold value may be appropriately modified.

[0096] In addition, the first determination unit 163 determines the distance between the end of the region 25 and the end of the photographing region 21. For example, the first determination unit 163 detects the number of pixels included between the end of the region 25 and the end of the photographing region 21, and determines that the region 25 is included in the photographing region 21 when there are pixels whose number exceeds a predetermined number.

[0097] With the above processing, the first determination unit 163 can determine that the size of the user's face is extremely small or extremely large, or that the position of the user's face is outside the photographing area of ​​the camera.

[0098] Note that in Figure 4 , the region 25 extracted as the front face of the user is represented by a rectangle; however, the first determination unit 163 can extract the region 25 in a shape along the front face of the user, not limited to a rectangle. In addition, the region 25 may or may not be displayed on the screen to be presented to the user.

[0099] When it is determined that the size of the area occupied by the front face of the user is within the threshold, the first determination unit 163 outputs a response indicating a transition to photographing the side face of the user. As a result, the first determination unit 163 can switch to photographing the side face after adjusting the positional relationship between the user and the user terminal 10 (photographing device) to an appropriate positional relationship.

[0100] Specifically, the first determination unit 163 outputs at least one of voice, vibration, or display of a character string as a response. For example, the first determination unit 163 uses the TTS function to output a voice prompting the user to turn right or left to shoot a side face. Alternatively, the first determination unit 163 can notify the user that the determination of the front face has been completed by outputting a sound effect indicating that the determination of the front face has been completed or vibrating the user terminal 10.

[0101] In addition, when determining that the size of the area occupied by the user's front face is not within the threshold, the first determination unit 163 may output a response instructing to move the position of the user's face or a response instructing to move the position of a device for photographing the user.

[0102] In addition, in this case, the first determination unit 163 outputs at least one of voice, vibration, or display of a character string as a response. For example, the first determination unit 163 uses the TTS function to output a voice prompt that makes the distance between the user and the user terminal 10 closer or farther, or moves the position of the user or the user terminal 10 to the right or left.

[0103] When the first determination unit 163 determines that the size of the area occupied by the front face of the user is within the threshold, the second determination unit 164 determines whether the size of the area occupied by the side face of the user in the photographing area is within the threshold.

[0104] Note that the second determination unit 164 may set a second guide area for specifying the area occupied by the front face of the user in the photographing area, and determine whether the size of the area occupied by the front face of the user relative to the second guide area is within a threshold. The second guide area corresponds to, for example, Figure 2 The inside of the guide frame 23 shown in FIG. By setting the second guide area, the second determination unit 164 can make the user intuitively understand how big the face should be captured by the camera or at which position.

[0105] The second determination unit 164 determines whether an area including the user's ears in the area extracted as the user's side face is included in the photographing area, and if the area including the user's ears is included in the photographing area, determines that the size of the area occupied by the user's side face is within a threshold.

[0106] Specifically, the second determination unit 164 determines whether the area including the user's ears is included in the photographing area based on a ratio at which the area including the user's ears is included in the photographing area.

[0107] In addition, the second determination unit 164 may determine whether the area including the user's ears is included in the photographing area based on a ratio occupied by the area including the user's ears in the area occupied by the user's side face.

[0108] Furthermore, the second determination unit 164 may determine whether the area including the user's ears is included in the photographing area based on a distance between an end of the area including the user's ears and an end of the photographing area.

[0109] Reference Figure 5 The processing performed by the second determination unit 164 will be described. Figure 5 2 is a diagram for explaining the second determination process according to the embodiment.

[0110] like Figure 5 , in the case of recognizing an image of the side face of the user, the second determination unit 164 extracts the area 26 as the side face of the user. For example, the second determination unit 164 determines whether the area occupied by the area 26 is within the threshold with respect to the photographing area 21 or the second guide area (the inside of the guide frame 23), and determines whether the side face of the user is included in the photographing area 21. Note that, for example, in response to a request from the cloud server 100, the threshold may be appropriately modified.

[0111] In addition, the second determination unit 164 can determine whether the area including the user's ears is included in the photographing area based on the ratio of the area 26 occupied by the user's side face to the area 27 including the user's ears. This is because, in the image used to generate the HRTF, the ratio between the head size and the ear size of the user's side face affects the accuracy of the HRTF to be generated. With such processing, the second determination unit 164 can capture an image in which the size of the user's head has an appropriate relationship with the size of the ear.

[0112] In addition, the second determination unit 164 may determine the distance between the end of the region 27 indicating the ear and the end of the photographing region 21. For example, the second determination unit 164 detects the number of pixels included between the end of the region 27 and the end of the photographing region 21, and determines that the region 27 is included in the photographing region 21 if there are pixels whose number exceeds a predetermined number.

[0113] With the above processing, the second determination unit 164 can prevent an image that does not include ears from being captured, and thus can reduce the burden on the user due to re-shooting or the like.

[0114] Note that in Figure 4 , the region 26 extracted as the side face of the user and the region 27 including the ear are represented by a rectangle; however, the second determination unit 164 may extract the region 26 or the region 27 in a shape along the side face or the ear of the user, without being limited to a rectangle. In addition, the region 26 or the region 27 may or may not be displayed on the screen and presented to the user.

[0115] In addition, in the case where it is determined that the area including the user's ears in the area extracted as the user's side face is not included in the photographing area, the second determination unit 164 may output a response indicating to move the position of the user's face or a response indicating to move the position of the device used to photograph the user.

[0116] Specifically, the second determination unit 164 outputs at least one of voice, vibration, or display of a character string as a response. For example, the second determination unit 164 can inform the user whether shooting is performed appropriately in an easily understandable manner by responding in a pattern that is easily understandable to the user without looking at the screen, such as voice or vibration.

[0117] When the second determination unit 164 determines that the size of the area occupied by the side face of the user is within the threshold, the photographing unit 165 photographs the side face of the user. The photographing unit 165 can perform photographing at an appropriate timing by automatically performing photographing in response to the determination result of the second determination unit 164 without determining whether the shutter button is pressed by the user.

[0118] In addition, the photographing unit 165 can photograph the side face of the user when the second determination unit 164 determines that the size of the area occupied by the side face of the user is within the threshold and then determines that the amount of movement of the device used to photograph the user is within the threshold. In an embodiment, the device used to photograph the user refers to the user terminal 10. That is, the photographing unit 165 performs photographing when receiving the determination result of the second determination unit 164 and determining that the amount of movement (acceleration, speed, etc.) of the user terminal 10 is within the threshold. As a result, the photographing unit 165 can prevent such photographing that causes blur.

[0119] Furthermore, the photographing unit 165 may photograph the side face of the user when the second determination unit 164 determines that the size of the area occupied by the side face of the user is within a threshold and then determines that the movement amount of the side face of the user in the photographing area is within a threshold.

[0120] That is, the photographing unit 165 compares the image at the time of receiving the determination result of the second determination unit 164 with the image after the subsequent photographing interval (for example, the image after a predetermined number of frames), and determines that the amount of movement of the face or ear of the user is equal to or less than the threshold value. For example, such processing can be performed by applying tracking processing of the object in the image, etc. The photographing unit 165 can prevent such photographing that causes blur by performing photographing when it is determined that the amount of movement of the user is within the threshold value.

[0121] In addition, the shooting unit 165 may edit the captured image and perform preprocessing before sending the image to the cloud server 100. For example, the shooting unit 165 may perform editing so that the area including the user's ears is at the center of the image. In addition, the shooting unit 165 may trim the area outside the predetermined range surrounding the area including the user's ears.

[0122] When the photographing unit 165 does not photograph the user's profile within a predetermined period of time, the photographing control unit 166 presents the user with an automatic photographing function. Specifically, when the user's profile has not been properly photographed, the photographing control unit 166 presents the user with a manual photographing function.

[0123] For example, the shooting control unit 166 presents the user with a manual shooting function when the user's side face is not properly shot within a predetermined period of time from the determination process of shooting an image of the user facing forward or the shooting process of shooting a side face image of the user. Then, the shooting control unit 166 receives the user's operation of shooting the side face image of the user. At this time, the shooting control unit 166 can display a screen including the guide frame 23 and accept the user's manual shooting.

[0124] The transmission unit 167 transmits various types of information. For example, the transmission unit 167 transmits an image captured by the capturing unit 165 and the capturing control unit 166 to the cloud server 100.

[0125] The receiving unit 168 receives various types of information. For example, the receiving unit 168 receives a personal HRTF generated by the cloud server 100.

[0126] [1-3. Configuration of cloud server according to embodiment]

[0127] Next, we will refer to Figure 6 The configuration of the cloud server 100 according to the embodiment is described. Figure 6 is a diagram illustrating a configuration example of the cloud server 100 according to the embodiment.

[0128] like Figure 6 As illustrated in FIG, the cloud server 100 includes a communication unit 110, a storage unit 120, and a control unit 130. Note that the cloud server 100 may include an input unit (such as a keyboard or a mouse) for receiving various operations from an administrator or the like who manages the cloud server 100, and a display unit (such as a liquid crystal display) for displaying various types of information.

[0129] The communication unit 110 is implemented by, for example, a NIC, etc. The communication unit 110 is connected to a network N (such as the Internet) in a wired or wireless manner, and transmits and receives information to and from the user terminal 10 and the like via the network N.

[0130] The storage unit 120 is implemented by, for example, a semiconductor storage element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. For example, the storage unit 120 stores an image acquired from the user terminal 10 or a model for generating (calculating) HRTF (for example, a learning model that has learned to output HRTF using an image as input).

[0131] The control unit 130 is implemented by, for example, a CPU, MPU, GPU, etc. executing a program stored in the cloud server 100 using a RAM, etc. as a work area. In addition, the control unit 130 is a controller and may be implemented by, for example, an integrated circuit such as an ASIC or an FPGA.

[0132] like Figure 6 As shown in FIG. 1 , the control unit 130 includes a receiving unit 131, a generating unit 132, and a providing unit 133, and implements or executes the following information processing functions or actions. Note that the internal configuration of the control unit 130 is not limited to Figure 6 The configuration illustrated in the figure may be another configuration as long as the information processing described below is performed.

[0133] The receiving unit 131 receives various types of information. For example, the receiving unit 131 receives a request for HRTF generation processing from the user terminal 10. The receiving unit 131 also receives an image for HRTF generation processing from the user terminal 10.

[0134] The generation unit 132 generates a personal HRTF corresponding to the image based on the image received by the receiving unit 131. For example, the generation unit 132 generates the personal HRTF using a learning model that has learned to output the HRTF using the image as an input. Note that the learned model is not specified for any type, and for example, the generation unit 132 can generate the personal HRTF using various models generated using various learning algorithms such as neural networks, support vector machines, clustering, and reinforcement learning.

[0135] Note that the generation unit 132 may transmit parameters related to the photographing process performed by the user terminal 10 to the user terminal 10. For example, the generation unit 132 transmits information specifying the size of the ear to be included in the image (such as, "an image in which the size of the area including the ear is approximately 200×200 pixels") or parameters such as the amount of movement of the user that can be accepted in photographing and the acceleration or speed of the user terminal 10.

[0136] In addition, the generation unit 132 can generate a learning model by learning an image manually captured as a normal example using a learning algorithm when the user terminal 10 does not capture the user's side face within a predetermined period of time. In addition, the generation unit 132 can generate a personal HRTF using such a learning model. As described above, since the generation unit 132 can learn an image in which it is difficult to automatically capture the user's side face as a normal example, the accuracy of the learned model can be improved.

[0137] The providing unit 133 provides the personal HRTF generated by the generating unit 132 to the user terminal 10. Note that, in the case where the personal HRTF cannot be generated based on the image received from the user terminal 10, the providing unit 133 provides error information or the like.

[0138] [1-4. Procedure of information processing according to the embodiment]

[0139] By reference Figure 7 and Figure 8 The process of information processing according to the embodiment is described below. Figure 7 In the following, a processing procedure in which the user terminal 10 performs the first determination processing will be described. Figure 7 is a flowchart (1) illustrating the flow of information processing according to the embodiment.

[0140] like Figure 7 As shown in FIG. 1 , the user terminal 10 determines whether the front face of the user is detected within the viewing angle of the camera function (step S21 ). If the front face of the user is not detected (step S21 ; No), the user terminal 10 waits until the front face of the user is detected.

[0141] On the other hand, if the front face of the user is detected (step S21 ; Yes), the user terminal 10 determines whether the size of the front face is appropriate in the photographing area (step S22 ).

[0142] If the size of the front face is not appropriate (step S22; No), the user terminal 10 outputs a response such as voice guidance prompting the user to adjust the size (step S23).

[0143] If the size of the front face of the user is appropriate (step S22 ; YES), the user terminal 10 determines whether the position of the front face relative to the screen is appropriate (step S24 ).

[0144] If the position of the front face is not appropriate (step S24; No), the user terminal 10 outputs a response such as voice guidance prompting the user to adjust the position (step S25).

[0145] If the position of the user's front face is appropriate (step S24; yes), the user terminal 10 determines that the user's front face has been successfully recognized (step S26). Then, the user terminal 10 outputs feedback (such as voice or vibration) indicating successful recognition (step S27).

[0146] Next, we will refer to Figure 8 The procedure of a process in which the user terminal 10 photographs the user's ear is described. Figure 8 is a flowchart (2) illustrating the flow of information processing according to the embodiment.

[0147] like Figure 8 As shown in FIG, the user terminal 10 determines whether the user's ear (side face) is detected within the viewing angle of the camera function (step S31). If the user's ear is not detected (step S31; No), the user terminal 10 waits until the user's ear is detected.

[0148] On the other hand, if the user's ear is detected (step S31 ; Yes), the user terminal 10 determines whether the size of the ear is appropriate in the photographing area (step S32 ).

[0149] If the size of the ear is not appropriate (step S32; No), the user terminal 10 outputs a response such as voice guidance prompting the user to adjust the size (step S33).

[0150] If the size of the user's ear is appropriate (step S32; Yes), the user terminal 10 determines whether the position of the ear is appropriate relative to the screen (step S34).

[0151] If the position of the ear is not appropriate (step S34; No), the user terminal 10 outputs a response such as voice guidance prompting the user to adjust the ear position (step S35).

[0152] If the position of the user's ear is appropriate (step S34; Yes), the user terminal 10 determines whether the user terminal 10 has moved more than a threshold value (step S36).

[0153] If the user terminal 10 moves more than the threshold value (step S36; No), the user terminal 10 outputs a response such as a voice guidance instructing the user not to move the user terminal 10 (step S37).

[0154] If the user terminal 10 has not moved more than the threshold value (step S36; Yes), the user terminal 10 determines whether the position deviation of the ear or head of the user is within the threshold value for the frame to be photographed (step S38).

[0155] If the positional deviation is not within the threshold value (step S38; No), the user terminal 10 outputs a response such as voice guidance prompting the user not to move the face (step S39).

[0156] If the position deviation is within the threshold value (step S38; Yes), the user terminal 10 determines that the ear has been successfully recognized (step S40). Then, the user terminal 10 photographs the user's ear and acquires the photographed image (step S41).

[0157] Then, the user terminal 10 outputs feedback indicating that the ear is photographed to the user (step S42). Then, the user terminal 10 determines whether both ears are photographed (step S43). If both ears are not photographed, that is, if only one ear is photographed (step S43; No), the user terminal 10 repeats the process of detecting the other ear (step S31). On the other hand, if both ears are photographed (step S43; Yes), the user terminal 10 ends the photographing process.

[0158] [1-4-1. Modification of information processing procedure]

[0159] Next, we will refer to Figures 9 to 11 To describe a variation of the information processing process according to the embodiment. Note that in this description, by citing the above information processing process, references to the above referenced information will be omitted. Figure 7 and Figure 8 The description of an information processing process is a redundant description of the same process.

[0160] exist Fig. 9 In the following, a modification example of the processing procedure in which the user terminal 10 performs the first determination processing will be described. Fig. 9 is a flowchart (1) illustrating a modification of the flow of information processing according to the embodiment.

[0161] like Fig. 9 As shown in the figure, in this variant, the user terminal 10 executes Figure 7 Before performing an operation similar to the operation illustrated in FIG. 1 , start measuring the elapsed time from the start of this operation (step S51). Then, if the user terminal 10 fails to detect the front face of the user within a predetermined period of time from the start of the measurement of the elapsed time (step S21; No and step S52; Yes), the processing switches to a manual shooting operation (step S53), and the operation ends. In other words, in this variant, if the automatic shooting of the ear cannot be completed within a predetermined period of time (step S52; Yes), the processing switches to a manual shooting operation. Other operations may be similar to Figure 7 The operations are illustrated in the figure, so their description is omitted here.

[0162] Next, we will refer to Fig.10A modification example of the procedure of a process in which the user terminal 10 photographs the user's ear is described. Fig.10 is a flowchart (2) illustrating a modification of the flow of information processing according to the embodiment.

[0163] like Fig.10 In the figure, in this variant, Figure 8 In an operation similar to the operation shown in FIG. Fig. 9 In the case where the user's ear cannot be detected within a predetermined period of time after the measurement of the elapsed time in step S51 is started (step S31; No and step S61; Yes), the processing is switched to a manual shooting operation (step S62), and the operation ends. In other words, in this variant, if the automatic shooting of the ear cannot be completed within a predetermined period of time (step S61; Yes), the processing is switched to a manual shooting operation.

[0164] Note that the step of determining the transition to the manual shooting operation is not limited to the case where the ear cannot be detected in step S31 (step S31; No), and can be provided for cases such as the case where the ear size is inappropriate in step S32 (step S32; No), the ear position is inappropriate relative to the screen in step S34 (step S34; No), the terminal is moving in step S36 (step S36; No), the position deviation is greater than the threshold in step S38 (step S38; No), or the ear has not been photographed in step S43 (step S43; No). Other operations can be similar to Figure 8 The operations are illustrated in the figure, so their description is omitted here.

[0165] Next, we will refer to Fig.11 A procedure of a process in which the user terminal 10 photographs an ear by a manual operation of the user is described. Fig.11 is a flowchart (3) illustrating a modification of the flow of information processing according to the embodiment.

[0166] like Fig.11 As shown in the figure, when Fig. 9 Step S53, Fig.10 When the manual ear photographing operation is activated in step S62 or the like, the user terminal 10 first outputs a response such as a voice guidance prompting the user to manually photograph the ear (step S71).

[0167] Next, the user terminal waits until the user presses the shutter button of the user terminal 10 (step S72; no), and when the user presses the shutter button (step S72; yes), the user terminal 10 displays the captured image on the screen (step S73) and outputs a response such as a voice guidance prompting the user to confirm the captured image (step S74).

[0168] Next, the user terminal 10 determines whether the user has input an operation to confirm the captured image (step S75). In addition, before the user's confirmation operation is input (step S75; No), the user terminal 10 accepts an operation to edit the position, size or tilt of the captured image taken by the user (step S76), and edits the captured image according to the operation content (step S77).

[0169] Then, when the user's confirmation operation is input (step S75 ; Yes), the user terminal 10 ends the photographing process.

[0170] For example, in Fig.11 In the process illustrated in FIG. 1 , the user terminal 10 accepts user operations such as adjusting the size or position of an image of a user's ear manually taken by the user (step S76) and adjusting the size or position of the taken image of the user's ear (step S77).

[0171] In this case, the user terminal 10 may provide information about user assistance. For example, the user terminal 10 provides the user with an appropriate image of an ear in advance as content such as a web page or application content displayed by a browser. As a result, the user can adjust the size or position of the captured image of the user's ear by referring to the appropriate image of the ear. Note that user assistance is not limited to providing content, and may be user assistance by voice, for example.

[0172] Then, the user terminal 10 ends the shooting process by accepting the operation corresponding to the end of shooting as the user's operation. Note that, here, the operation corresponding to the end of shooting can be any operation. For example, when the user presses the shooting end button included by default in the manual shooting function, the user terminal 10 ends the shooting process.

[0173] (2. Modifications)

[0174] The above-described information processing system 1 can be implemented in various different forms other than the above embodiment. Therefore, a modified example of the embodiment will be described below.

[0175] In the embodiment, an example has been described in which the cloud server 100 generates the personal HRTF. However, the personal HRTF may be generated by the user terminal 10 instead of the cloud server 100. In this case, the user terminal 10 includes Figure 6 A processing unit similar to the generation unit 132 shown in FIG.

[0176] In the embodiment, it is assumed that the cloud server 100 is installed on a cloud network; however, without being limited to this example, the cloud server 100 may be installed on a network such as a local area network (LAN) as long as communication with the user terminal 10 is possible.

[0177] When generating a personal HRTF, the cloud server 100 may acquire identification information of the user, associate the identification information with the personal HRTF, and maintain the personal HRTF.

[0178] In the embodiment, the cloud server 100 is described as a single server; however, the cloud server 100 may include a plurality of server devices. For example, the cloud server 100 may be divided into a generating server that generates a personal HRTF and a providing server that provides a personal HRTF.

[0179] In the embodiment, an example is described in which the user terminal 10 photographs the user; however, the photographing may be performed by another photographing device. In this case, the photographing device sequentially sends information of the image to be photographed to the user terminal 10. Then, the user terminal 10 performs information processing according to the embodiment by identifying the position of the face or ear of the user in the received image. In addition, in the embodiment, it is described that the user terminal 10 performs the process of determining the size of the side face of the user; however, the user terminal 10 may skip the process of determining the face size and only determine the ear size. That is, the user terminal 10 does not have to determine the size or position of the ear after determining the size of the side face of the user, but may directly determine the appropriateness of the size or position of the user's ear if possible.

[0180] The user terminal 10 can set the amount of light of the image to be captured, the allowable value of the tilt of the ear, etc. according to the designation from the cloud server 100. As a result, the user terminal 10 can reduce the probability of returning an error from the cloud server 100 that a personal HRTF cannot be generated from the captured image.

[0181] (3. Other embodiments)

[0182] The processing according to the above embodiment may be performed in various different embodiments other than the above embodiment.

[0183] Among the processes described in the above embodiments, all or part of the processes described as being automatically performed may be performed manually, or all or part of the processes described as being manually performed may be performed automatically by known methods. In addition, unless otherwise specified, the processing procedures, specific names, and information including various types of data or parameters illustrated above or in the accompanying drawings may be modified as desired. For example, the various types of information illustrated in the drawings are not limited to the illustrated information.

[0184] In addition, the functions of the components of the devices illustrated in the figures are conceptual and do not necessarily need to be physically configured as illustrated in the figures. That is, the specific forms of distribution and integration of the devices are not limited to the forms illustrated in the figures, and all or some of them can be functionally or physically distributed or integrated in any unit according to various loads, usage status, etc.

[0185] In addition, the above embodiments and modifications can be appropriately combined within the range where there is no conflict in the processing contents.

[0186] Furthermore, the effects described herein are merely examples and not limitations, and other effects may be achieved.

[0187] (4. Effects of the information processing device according to the present disclosure)

[0188] As described above, the information processing device (user terminal 10 in the embodiment) according to the present disclosure includes a determination unit (a second determination unit 164 in the embodiment), a shooting unit (a shooting unit 165 in the embodiment), and a shooting control unit (a shooting control unit 166 in the embodiment). The determination unit determines whether the size of the area occupied by the side face of the user is within the threshold in the shooting area. In the case where the size of the area occupied by the side face of the user is determined to be within the threshold, the shooting unit shoots the side face of the user. In the case where the side face of the user has not been shot within a predetermined period of time, the shooting control unit presents a manual shooting function to the user.

[0189] As described above, the information processing device according to the present disclosure can prompt the user to manually shoot the side of the head when the side face of the user has not been photographed in the shooting area within a predetermined period of time. With this configuration, it is possible to prevent the information processing device from returning to the initial stage of the shooting process that may occur due to the inability to determine the side face of the user, and it is possible to prevent the information processing device from falling into the processing loop of determining the front face and side face of the user again, so that the trouble or burden of the user can be reduced. Therefore, the information processing device can improve the convenience of the user in shooting performed for calculating the head-related transfer function.

[0190] The determination unit is further provided with a second guide area for specifying an area occupied by the side face of the user in the shooting area, and determines whether the size of the area occupied by the side face of the user relative to the second guide area is within a threshold value. As a result, the information processing device can allow the user to intuitively understand how large the face should be captured by the camera or at which position it should be captured.

[0191] In addition, the determination unit determines whether the area including the user's ears in the area extracted as the user's side face is included in the photographing area, and, if the area including the user's ears is included in the photographing area, determines that the size of the area occupied by the user's side face is within the threshold value. As a result, the information processing device can reliably include the ears in the image to be captured, and thus can acquire an image suitable for the generation process of the personal HRTF.

[0192] The determination unit further determines whether the area including the user's ears is included in the photographing area based on the ratio of the area including the user's ears being included in the photographing area. As a result, the information processing device can acquire an image including ears of a size suitable for the generation process of the personal HRTF.

[0193] The determination unit also determines whether the area including the user's ears is included in the photographing area based on the ratio of the area including the user's ears to the area occupied by the user's side face. As a result, the information processing device can obtain an image in which the user's head size and ear size have a relationship suitable for generating an HRTF.

[0194] In addition, the determination unit determines whether the area including the user's ears is included in the photographing area based on the distance between the end of the area including the user's ears and the end of the photographing area. As a result, the information processing device can prevent the image from being captured, from which the HRTF cannot be generated because the ears are located at the end of the image.

[0195] In addition, in the case where it is determined that the area including the user's ears in the area extracted as the user's side face is not included in the photographing area, the determination unit outputs a response indicating to move the position of the user's face or a response indicating to move the position of the device for photographing the user. As a result, the information processing device can provide an appropriate response to the user, so that photographing can be performed accurately and quickly.

[0196] In addition, the determination unit outputs at least one of voice, vibration, or display of a character string as a response. As a result, even in a state where the user cannot visually recognize the screen, the information processing device can notify the user of an appropriate response.

[0197] In addition, the photographing unit photographs the user's side face when determining that the size of the area occupied by the user's side face is within a threshold and then determining that the amount of movement of the device for photographing the user is within a threshold. As a result, the information processing device can prevent the photographing of an image in which blur occurs.

[0198] In addition, the photographing unit photographs the user's side face when determining that the size of the area occupied by the user's side face is within a threshold and then determining that the amount of movement of the user's side face in the photographing area is within a threshold. As a result, the information processing device can prevent the photographing of an image in which blur has occurred.

[0199] (5. Hardware configuration)

[0200] For example, the information devices such as the user terminal 10 and the cloud server 100 according to the above-described embodiment are composed of Fig.12 Hereinafter, the user terminal 10 according to the embodiment will be described as an example. Fig.1210 is a hardware configuration diagram illustrating an example of a computer 1000 that implements the functions of the user terminal 10. The computer 1000 includes a CPU 1100, a RAM 1200, a read-only memory (ROM) 1300, a hard disk drive (HDD) 1400, a communication interface 1500, and an input and output interface 1600. The respective units of the computer 1000 are connected by a bus 1050.

[0201] The CPU 1100 operates in accordance with the program stored in the ROM 1300 or the HDD 1400 and controls each unit. For example, the CPU 1100 loads the program stored in the ROM 1300 or the HDD 1400 onto the RAM 1200 and executes processing corresponding to the various programs.

[0202] The ROM 1300 stores a boot program such as a basic input output system (BIOS) executed by the CPU 1100 when the computer 1000 is activated, a program depending on the hardware of the computer 1000 , and the like.

[0203] The HDD 1400 is a computer-readable recording medium that non-transitorily records a program to be executed by the CPU 1100 , data used by the program, and the like. Specifically, the HDD 1400 is a recording medium that records an information processing program according to the present disclosure, which is an example of the program data 1450 .

[0204] The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (eg, the Internet). For example, the CPU 1100 receives data from another device, or transmits data generated by the CPU 1100 to another device via the communication interface 1500.

[0205] The input and output interface 1600 is an interface for connecting the input and output device 1650 to the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard and a mouse via the input and output interface 1600. The CPU 1100 also sends data to output devices such as a display, a speaker, or a printer via the input and output interface 1600. In addition, the input and output interface 1600 can be used as a medium interface for reading a program recorded in a predetermined recording medium (medium), etc. For example, the medium refers to an optical recording medium such as a digital versatile disk (DVD) or a phase change rewritable disk (PD), a magneto-optical recording medium such as a magneto-optical disk (MO), a magnetic tape medium, a magnetic recording medium, an optical recording medium such as a semiconductor memory, etc.

[0206] For example, in the case where the computer 1000 is used as the user terminal 10 according to the embodiment, the CPU 1100 of the computer 1000 implements the functions of the control unit 16 or other units by executing the information processing program loaded on the RAM 1200. The HDD 1400 also stores the information processing program according to the present disclosure or the data in the storage unit 15. Note that although the CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program data 1450, as another example, the CPU 1100 may obtain these programs from another device via the external network 1550.

[0207] Note that the present technology may also have the following configurations.

[0208] (1) An information processing device comprising:

[0209] a determination unit, for determining whether the size of the area occupied by the side face of the user in the shooting area is within a threshold;

[0210] a photographing unit that photographs the side face of the user when it is determined that the size of the area occupied by the side face of the user is within a threshold; and

[0211] The shooting control unit presents a manual shooting function to the user when the user's side face has not been shot within a predetermined period of time.

[0212] (2) The information processing device according to (1),

[0213] Among them, the judgment unit

[0214] A guide area for specifying an area occupied by the side face of the user in the photographing area is set, and it is determined whether the size of the area occupied by the side face of the user relative to the guide area is within a threshold.

[0215] (3) The information processing device according to (1) or (2),

[0216] Among them, the judgment unit

[0217] It is determined whether an area including the user's ears in the area extracted as the user's side face is included in the photographing area, and if the area including the user's ears is included in the photographing area, it is determined that the size of the area occupied by the user's side face is within a threshold.

[0218] (4) The information processing device according to (3),

[0219] Among them, the judgment unit

[0220] Whether the area including the ears of the user is included in the photographing area is determined based on a ratio at which the area including the ears of the user is included in the photographing area.

[0221] (5) The information processing device according to (3) or (4),

[0222] Among them, the judgment unit

[0223] Whether the area including the user's ears is included in the photographing area is determined based on an occupation ratio of the area including the user's ears in the area occupied by the user's side face.

[0224] (6) The information processing device according to any one of (3) to (5),

[0225] Whether the area including the user's ears is included in the photographing area is determined based on a distance between an end of the area including the user's ears and an end of the photographing area.

[0226] (7) The information processing device according to any one of (3) to (6),

[0227] When it is determined that the region including the ears of the user in the region extracted as the side face of the user is not included in the photographing region, the determination unit

[0228] A response indicating moving the position of the user's face or a response indicating moving the position of a device for photographing the user is output.

[0229] (8) The information processing device according to (7),

[0230] Among them, the judgment unit

[0231] At least one of voice, vibration, or display of a character string is output as a response.

[0232] (9) The information processing device according to any one of (1) to (8),

[0233] Among them, the shooting unit

[0234] The side face of the user is photographed when it is determined that the size of the area occupied by the side face of the user is within a threshold and then it is determined that the amount of movement of the device for photographing the user is within a threshold.

[0235] (10) The information processing device according to any one of (1) to (9),

[0236] Among them, the shooting unit

[0237] The side face of the user is photographed when it is determined that the size of the area occupied by the side face of the user is within a threshold and then it is determined that the amount of movement of the side face of the user in the photographing area is within a threshold.

[0238] (11) An information processing method comprising:

[0239] By computer

[0240] Determine whether the size of the area occupied by the side face of the user in the shooting area is within a threshold;

[0241] When it is determined that the size of the area occupied by the side face of the user is within a threshold, photographing the side face of the user; and

[0242] In the case where the user's profile has not been captured within a predetermined period of time, a manual capture function is presented to the user.

[0243] (12) An information processing program for causing a computer to:

[0244] a determination unit, for determining whether the size of the area occupied by the side face of the user in the shooting area is within a threshold;

[0245] a photographing unit that photographs the side face of the user when it is determined that the size of the area occupied by the side face of the user is within a threshold; and

[0246] The shooting control unit presents a manual shooting function to the user when the user's side face has not been shot within a predetermined period of time.

[0247] Reference Symbols List

[0248] 1 Information Processing System

[0249] 10 User Terminal

[0250] 11 Communication unit

[0251] 12 Input Units

[0252] 13 Display unit

[0253] 14 Detection Unit

[0254] 15 Storage Unit

[0255] 16 Control Unit

[0256] 161 Get Unit

[0257] 162 Shooting Processing Unit

[0258] 163 First Judgment Unit

[0259] 164 Second judgment unit

[0260] 165 Shooting Unit

[0261] 166 Shooting Control Unit

[0262] 167 Sending Unit

[0263] 168 Receiving Unit

[0264] 100 Cloud Servers

Claims

1. An information processing device, comprising: a determination unit, wherein after determining that the size of the area occupied by the front face of the user in the shooting area is within a first threshold value suitable for shooting, the determination unit determines whether the size of the area occupied by the side face of the user including the ears of the user in the shooting area is within a second threshold value suitable for shooting, the second threshold value being suitable for shooting; a capturing unit that images the side face of the user when it is determined that the size of the area occupied by the side face of the user including the ear of the user is within a second threshold value suitable for photographing including an upper limit and a lower limit; a capture control unit, wherein the capture control unit presents a manual shooting function to the user if the side face of the user has not been captured within a predetermined period of time; as well as A receiving unit receives a head-related transfer function HRTF acquired based on a captured image of a side face portion of the user.

2. The information processing device according to claim 1, in, The determination unit A guide area for specifying an area occupied by the side face of the user in the photographing area is provided, and it is determined whether a size of the area occupied by the side face of the user relative to the guide area is within a second threshold.

3. The information processing device according to claim 1, in, The determination unit Determine whether an area including the user's ears in the area extracted as the user's side face is included in the photographing area extracted as the user's side face, and if the area including the user's ears is included in the photographing area, determine whether the size of the area occupied by the user's side face is within a second threshold.

4. The information processing device according to claim 3, in, The determination unit Whether the area including the ear of the user is included in the photographing area is determined based on a ratio at which the area including the ear of the user is included in the photographing area.

5. The information processing device according to claim 3, in, The determination unit Whether the area including the ears of the user is included in the photographing area is determined based on an occupation ratio of the area including the ears of the user in the area occupied by the side face of the user.

6. The information processing device according to claim 3, in, The determination unit Whether the area including the ear of the user is included in the photographing area is determined based on a distance between an end of the area including the ear of the user and an end of the photographing area.

7. The information processing device according to claim 3, in, In a case where it is determined that a region including the ears of the user in the region extracted as the side face of the user is not included in the photographing region, the determination unit A response indicating movement of the position of the user's face or a response indicating movement of the position of a device for photographing the user is output.

8. The information processing device according to claim 7, in, The determination unit At least one of voice, vibration, or display of a character string is output as the response.

9. The information processing device according to claim 1, in, The capture unit The side face of the user is imaged when it is determined that the size of the area occupied by the side face of the user is within a second threshold and then when it is determined that the amount of movement of the device used to photograph the user is within the threshold.

10. The information processing device according to claim 1, in, The capture unit The side face of the user is imaged when it is determined that the size of the area occupied by the side face of the user is within a second threshold and then it is determined that the movement amount of the side face of the user in the shooting area is within a threshold.

11. An information processing method, comprising: By computer After determining that the size of the area occupied by the front face of the user in the shooting area is within a first threshold value suitable for shooting, including an upper limit and a lower limit, determining whether the size of the area occupied by the side face of the user including the user's ears in the shooting area is within a second threshold value suitable for shooting, including an upper limit and a lower limit; When it is determined that the size of the area occupied by the side face of the user including the ear of the user is within a second threshold value suitable for photographing including an upper limit and a lower limit, imaging the side face of the user; If the side face of the user has not been captured within a predetermined period of time, presenting a manual shooting function to the user; as well as A head related transfer function HRTF acquired based on a captured image of a side face portion of the user is received.

12. A computer-readable recording medium that non-transitorily records an information processing program for causing a computer to function as: a determination unit, wherein after determining that the size of the area occupied by the front face of the user in the shooting area is within a first threshold value suitable for shooting, the determination unit determines whether the size of the area occupied by the side face of the user including the ears of the user in the shooting area is within a second threshold value suitable for shooting, the second threshold value being suitable for shooting; a capturing unit that images the side face of the user when it is determined that the size of the area occupied by the side face of the user including the ear of the user is within a second threshold value suitable for photographing including an upper limit and a lower limit; as well as a capture control unit, wherein the capture control unit presents a manual shooting function to the user if the side face of the user has not been captured within a predetermined period of time; as well as A receiving unit receives a head-related transfer function HRTF acquired based on a captured image of a side face portion of the user.

Images