Information processing device, operation method for information processing device, and operation program for information processing device

Abstract

This information processing device comprises a processor. The processor: acquires a personality index value derived from a target image related to a target user whose personality is to be determined; compares the personality index value with statistical index values of a plurality of reference users; and determines the personality of the target user on the basis of the comparison results.

Description

Information processing device, operating method for information processing device, and operating program for information processing device

[0001] The technology of the present disclosure relates to an information processing device, an operating method for an information processing device, and an operating program for an information processing device.

[0002] The facial pattern analysis system disclosed in Japanese Patent Application Laid-Open No. 2010-140100 includes a facial image data recording means, an average facial image data recording means, a facial main body image extraction means, a facial part image identification / extraction means, a facial part image measurement means, a difference data calculation means, a rank determination means, and a facial pattern determination means. The facial image data recording means receives and temporarily stores digitally captured facial images. The average facial image data recording means stores average facial image data including an average facial image pattern and measurement values ​​of the eyes, nose, and face of the average facial image. The facial main body image extraction means identifies and extracts the position of the facial main body from the temporarily stored facial image data using pattern matching based on the average facial image pattern, and stores the extracted facial part image. The facial part image identification / extraction means identifies and extracts each facial part image, including at least the eyes, nose, and face, from the extracted facial main body image. The facial part image measurement means measures at least the area, shape, position (coordinates), and tilt of the extracted facial part image and temporarily stores these measurements. The difference data calculation means calculates numerical difference data for the distance between the eyes, the length of the nose, and the contour (horizontal to vertical) using a predetermined formula from the measurement values ​​of the eyes, nose, and contour obtained by the facial part image measurement means and the measurement values ​​of the eyes, nose, and contour of an average face image.The rank determination means ranks the distance between the eyes, the length of the nose, and the contour (horizontal to vertical) on a scale of 3 to 5 based on the difference data obtained by the difference data calculation means.The face pattern determination means determines which of 4 to 8 face patterns previously set based on the impression given by facial features the face pattern to which the received face image data belongs, based on the combination of the ranks of the distance between the eyes, the length of the nose, and the contour (horizontal to vertical) determined by the rank determination means, and outputs determination data.

[0003] The feature impression axis determination system disclosed in JP 2018-156136 A includes an image input means, a reference impression value determination means, and a feature impression axis determination means. The image input means inputs a plurality of images. The reference impression value determination means determines a reference impression value for each of the plurality of images input to the image input means for at least two reference impression axes. The feature impression axis determination means determines, as the feature impression axis representing a feature of the plurality of images input to the image input means, an axis along which the variance of the reference impression values ​​determined by the reference impression value determination means is greatest in a reference region defined by the at least two reference impression axes.

[0004] The image evaluation device disclosed in Japanese Patent Laid-Open No. 2001-195579 comprises a feature extraction means, a storage means, a calculation means, a sentence creation means, and an output control means. The feature extraction means extracts features of an input image. The storage means stores adjectives and features corresponding to the adjectives. The calculation means calculates the difference between the image features extracted by the feature extraction means and the features corresponding to the adjectives stored in the storage means. The sentence creation means creates a sentence using the calculation result by the calculation means. The output control means controls the output of the sentence created by the sentence creation means.

[0005] One embodiment of the technique of the present disclosure provides an information processing device, an operating method for the information processing device, and an operating program for the information processing device that are capable of determining a user's personality from an image with high accuracy.

[0006] The information processing device of the present disclosure includes a processor, which acquires a personality index value derived from a target image related to a target user whose personality is to be determined, compares the personality index value with statistical index values ​​of multiple reference users, and determines the personality of the target user based on the comparison result.

[0007] It is preferable that the personality index value is a representative value of index values ​​for each of a plurality of target images derived from a plurality of target images, and the statistical index value is a representative value of index values ​​for each of a plurality of reference images derived from a plurality of reference images related to the reference user.

[0008] There are multiple types of personality index values ​​and statistical index values, and it is preferable that the personality index value is the average value of the index values ​​for each target image excluding values ​​outside the set range, and the statistical index value is the average value of the index values ​​for each reference image excluding values ​​outside the set range.

[0009] The index value is preferably derived using a trained model.

[0010] Preferably, the processor compares the personality index value with statistical index values ​​of reference users who match or are similar to the target user in certain attributes.

[0011] Preferably, the processor obtains personality index values ​​derived from attribute-related target images that relate to attributes of the target images.

[0012] Preferably, the processor obtains personality index values ​​derived from all target images.

[0013] It is preferable that there are a plurality of types of personality index values ​​and statistical index values.

[0014] The operating method of the information processing device of the present disclosure includes obtaining a personality index value derived from a target image associated with a target user whose personality is to be determined, comparing the personality index value with statistical index values ​​of a plurality of reference users, and determining the personality of the target user based on the comparison result.

[0015] The operating program of the information processing device disclosed herein causes a computer to perform processing including obtaining a personality index value derived from a target image related to a target user whose personality is to be determined, comparing the personality index value with statistical index values ​​of multiple reference users, and determining the personality of the target user based on the comparison result.

[0016] 1 is a diagram illustrating a user terminal and an information processing server. FIG. 2 is a block diagram illustrating computers constituting the user terminal and the information processing server. FIG. 3 is a block diagram illustrating a processing unit of a CPU of the user terminal. FIG. 4 is a block diagram illustrating a processing unit of a CPU of the information processing server. FIG. 5 is a diagram illustrating data stored in an image information DB. FIG. 6 is a diagram illustrating image information. FIG. 7 is a diagram illustrating attribute information. FIG. 8 is a diagram illustrating personality determination information. FIG. 9 is a diagram illustrating the processing of an index value derivation unit. FIG. 10 is a diagram illustrating the processing in the learning phase of an index value derivation model. FIG. 11 is a block diagram illustrating the detailed configuration of the personality determination unit. FIG. 12 is a block diagram illustrating the detailed configuration of the personality determination unit. FIG. 13 is a diagram illustrating a group of target image information of a target user and a group of reference image information of a reference user whose specific attribute matches or is similar to that of the target user. FIG. 14 is a diagram illustrating the processing of a first derivation unit. FIG. 15 is a diagram illustrating the processing of a second derivation unit. FIG. 16 is a diagram illustrating the processing of a comparison unit and a generation unit. FIG. 17 is a diagram illustrating difference / intensity correspondence information. FIG. 18 is a flowchart illustrating the processing procedure of the information processing server. FIG. 19 is a flowchart illustrating the processing procedure of the information processing server. FIG. 19 is a block diagram illustrating the detailed configuration of the personality determination unit of the second embodiment.

[0017] [First Embodiment] As an example, as shown in FIG. 1 , a user U owns a user terminal 10. The user terminal 10 is a device having a camera function, an image playback / display function, an image editing function, an image transmission / reception function, and the like. The camera function has an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and obtains an image 56 of a subject (see FIG. 5 ) by focusing subject light captured through a lens on the imaging element. Specifically, the user terminal 10 is a digital camera, a smartphone, a tablet terminal, a notebook personal computer, or the like. The user U uses the camera function to capture an image 56 and uses the image editing function to edit the image 56 to his or her liking.

[0018] The user terminal 10 is connected to the information processing server 12 via a network 11 so as to be able to communicate with each other. The network 11 is, for example, a wide area network (WAN) such as the Internet or a public communication network. The user terminal 10 transmits (uploads) images 56 to the information processing server 12. The user terminal 10 also receives (downloads) the images 56 from the information processing server 12.

[0019] The information processing server 12 is, for example, a server computer, a workstation, or the like, and is an example of an “information processing device” according to the technology of the present disclosure. A plurality of user terminals 10 of a plurality of users U are connected to the information processing server 12 via a network 11.

[0020] 2, the computers that make up the user terminal 10 and the information processing server 12 basically have the same configuration, and include a storage 20, a memory 21, a CPU (Central Processing Unit) 22, a communication unit 23, a display 24, and an input device 25. These are interconnected via a bus line 26.

[0021] The storage 20 is a hard disk drive built into the computer that constitutes the user terminal 10 and the information processing server 12, or connected via a cable or network. Alternatively, the storage 20 is a disk array consisting of multiple hard disk drives. The storage 20 stores control programs such as an operating system, various application programs (hereinafter abbreviated as APs (Application Programs)), and various data associated with these programs. Note that a solid state drive may be used instead of a hard disk drive.

[0022] The memory 21 is a work memory for the CPU 22 to execute processing. The CPU 22 loads a program stored in the storage 20 into the memory 21 and executes processing in accordance with the program. In this way, the CPU 22 comprehensively controls each part of the computer. The CPU 22 is an example of a "processor" according to the technology of the present disclosure. The memory 21 may be built into the CPU 22.

[0023] The communication unit 23 is a network interface that controls the transmission of various types of information via the network 11, etc. The display 24 displays various screens. The various screens are equipped with operation functions using a GUI (Graphical User Interface). The computers that make up the user terminal 10 and the information processing server 12 accept input of operation instructions from an input device 25 via the various screens. The input device 25 is a keyboard, a mouse, a touch panel, a microphone for voice input, etc.

[0024] In the following explanation, the parts of the computer that make up the user terminal 10 (storage 20, CPU 22, display 24, and input device 25) are distinguished by adding the suffix "A" to their symbols, and the parts of the computer that make up the information processing server 12 (storage 20 and CPU 22) are distinguished by adding the suffix "B" to their symbols.

[0025] As an example, as shown in Figure 3, an image AP 30 is stored in the storage 20A of the user terminal 10. The image AP 30 is installed in the user terminal 10 by the user U. The image AP 30 is an AP for playing back, displaying, and editing an image 56 on the user terminal 10. When the image AP 30 is started, the CPU 22A of the user terminal 10 functions as a browser control unit 32 in cooperation with the memory 21 and the like. The browser control unit 32 controls the operation of a web browser dedicated to the image AP 30.

[0026] The browser control unit 32 receives screen data for various screens from the information processing server 12. The browser control unit 32 reproduces various screens to be displayed on the web browser based on the screen data and displays them on the display 24A. The browser control unit 32 also accepts various operation instructions input by the user U from the input device 25A via the various screens. The browser control unit 32 transmits various requests corresponding to the operation instructions to the information processing server 12.

[0027] 4, an operating program 35 is stored in the storage 20B of the information processing server 12. The operating program 35 is an AP that causes the computer constituting the information processing server 12 to function as an "information processing device" according to the technology of the present disclosure. In other words, the operating program 35 is an example of an "operating program for an information processing device" according to the technology of the present disclosure.

[0028] The storage 20B also stores an image information database (hereinafter referred to as DB (Data Base)) 36, an index value derivation model 37, and judgment auxiliary information 38. Although not shown in the figure, the storage 20B also stores, as account information of the user U, a user ID (Identification Data) for uniquely identifying the user U, a password set by the user U, and a terminal ID for uniquely identifying the user terminal 10.

[0029] When the operating program 35 is started, the CPU 22B of the information processing server 12 cooperates with the memory 21 and the like to function as a request receiving unit 45, an index value derivation unit 46, a read / write (hereinafter referred to as RW (Read Write)) control unit 47, a personality determination unit 48, and a distribution control unit 49.

[0030] The request receiving unit 45 receives various requests from the user terminal 10. The request receiving unit 45 outputs various requests to the index value derivation unit 46 and / or the RW control unit 47, and the distribution control unit 49. The various requests include a storage request for storing the image 56 in the image information DB 36, a distribution request for distributing the image 56 stored in the image information DB 36, etc. The various requests include the terminal ID of the user terminal 10 that is the sender of the various requests.

[0031] The index value derivation unit 46 derives an index value 58 (see FIG. 5 ) from the image 56 for which a storage request has been made. The index value 58 is a numerical value that serves as a clue for determining the personality of the user U who captured the image 56. The index value derivation unit 46 outputs the derived index value 58 together with the image 56 to the RW control unit 47. Here, "personality" refers to the tendency of personal preferences, such as what an individual, such as the user U, likes and what activities they like. The tendency of preferences analyzed for an individual that may be useful for estimating what they like, and a value indicating the tendency (which may be a scalar value or a vector value composed of five factors, such as openness, conscientiousness, extroversion, agreeableness, and neuroticism, as described below), are encompassed in the term "personality."

[0032] The RW control unit 47 controls the storage of various data in the storage 20B and the reading of various data from the storage 20B. In particular, the RW control unit 47 controls the storage of images 56 in the image information DB 36 and the reading of images 56 from the image information DB 36. The RW control unit 47 also controls the storage of index values ​​58 in the image information DB 36 and the reading of index values ​​58 from the image information DB 36.

[0033] The RW control unit 47 reads the index value derivation model 37 from the storage 20B and outputs the read index value derivation model 37 to the index value derivation unit 46. The RW control unit 47 also reads the judgment auxiliary information 38 from the storage 20B and outputs the read judgment auxiliary information 38 to the personality determination unit 48.

[0034] The personality determination unit 48 determines the personality of the user U based on the index value 58 while referring to the determination auxiliary information 38. The personality determination unit 48 outputs personality determination information 61 (see FIG. 5 ), which is the personality determination result, to the RW control unit 47. The RW control unit 47 stores the personality determination information 61 in the image information DB 36. Hereinafter, the user U whose personality is to be determined will be referred to as the target user UT (see FIG. 13 ). An image 56 related to the target user UT will be referred to as a target image 56T (see FIG. 11 ). The incidental information 57 (see FIG. 5 ) and index value 58 of the target image 56T will be referred to as target incidental information 57T and a target image index value 58T (both see FIG. 11 ). Furthermore, image information 59 (see FIG. 5 ) composed of a set of the target image 56T, the target incidental information 57T, and the target image index value 58T will be referred to as target image information 59T (see FIG. 11 ). Moreover, the attribute information 60 of the target user UT (see FIG. 5) is expressed as target attribute information 60T (see FIG. 11).

[0035] The distribution control unit 49 controls the distribution of various data including screen data to the user terminal 10. The distribution control unit 49 distributes the screen data to the user terminal 10 in the form of screen data for web distribution created using a markup language such as XML (Extensible Markup Language). Note that instead of XML, other data description languages ​​such as JSON (Javascript (registered trademark) Object Notation) may be used.

[0036] As an example, as shown in FIG. 5 , the image information DB 36 has a storage area 55 for each user U, such as user U1 and user U2. User IDs are registered in the storage area 55. The storage area 55 stores a plurality of pairs of images 56, their associated information 57, and index values ​​58. Hereinafter, the pairs of images 56, associated information 57, and index values ​​58 will be referred to as image information 59. The storage area 55 also stores attribute information 60 and personality assessment information 61.

[0037] The images 56 stored in the memory area 55 are related to the user U whose user ID is registered in the memory area 55. The images 56 include images taken by the user U using the camera function of the user terminal 10, images scanned by the user U, images downloaded by the user U from an internet site, or images given to the user U by an acquaintance. The images 56 "related" to the user U are typically images 56 stored by the user U himself on the user terminal 10 or a cloud server such as the information processing server 12. Images 56 stored by someone other than the user U include images 56 "related" to the user U if the user U is given access rights to the images 56 and the user U can freely edit them.

[0038] As shown in FIG. 6 as an example, an image 56, additional information 57, and index value 58 that make up image information 59 are associated with each other by an image ID. The additional information 57 includes multiple items such as the date and time of shooting, the location of shooting, and a tag. The date and time when the image 56 was captured using the camera function of the user terminal 10 is registered as the date and time of shooting. The location of shooting is registered with an address and / or a landmark name determined from longitude and latitude information obtained using the GPS (Global Positioning System) function of the user terminal 10. A tag is a word that succinctly describes the subject appearing in the image 56. Tags include those manually entered by the user U and those derived using a trained model for subject discrimination. Although not shown, the additional information 57 also includes items such as exposure value, ISO (International Organization for Standardization) sensitivity, shutter speed, focal length, and whether or not a flash is used.

[0039] There are five index values ​​58: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. These are factors that represent human personality based on the so-called Big Five theory, and take values ​​ranging from a minimum of 0% to a maximum of 100%. These have already been generally explained in detail, so we will avoid going into detail here. Simply put, Openness is a measure of whether one is creative or conservative, and Conscientiousness is a measure of whether one is hardworking or lazy. Extraversion is a measure of whether one is extroverted or introverted, and Agreeableness is a measure of whether one is cooperative or exclusive. Finally, Neuroticism is a measure of whether one is rational or emotional.

[0040] As shown in FIG. 7 as an example, the attribute information 60 literally indicates the attributes of the user U, and includes items such as the user U's gender, age, address, marital history, family structure, pet animals (pets), and hobbies. The address field includes the prefecture and city / ward / town / village. The marital history field includes single, married, divorced, widowed, etc. The family structure field includes the relationship and age of cohabiting family members. The pet animals field includes the type of pet animal, such as dog, cat, ferret, turtle, or killifish. Multiple pet animals and hobbies can be registered. The attribute information is obtained, for example, by having the user U answer a questionnaire when the user U installs the image AP 30 on the user terminal 10. The attribute information may also include the user U's hometown, favorite athletes, entertainers, sports teams, etc. The attributes may also be determined from the image 56.

[0041] As an example, as shown in FIG. 8 , the personality assessment information 61 is a set of personality intensities corresponding to five factors with index values ​​58. Each factor has seven personality intensities: 3, 2, 1, 0, -1, -2, and -3. The personality intensities corresponding to the openness factor are assigned as 3 (creative) and -3 (conservative). Expressed in writing, 3 represents very creative, 2 represents moderately creative, 1 represents slightly creative, -1 represents slightly conservative, -2 represents moderately conservative, -3 represents very conservative, and 0 represents neither. Similarly, the personality intensities corresponding to the conscientiousness factor are assigned as 3 (hardworking) and -3 (lazy). The personality intensities corresponding to the extraversion factor are assigned as 3 (extroverted) and -3 (introverted). The personality intensities corresponding to the agreeableness factor are assigned as 3 (cooperative) and -3 (exclusive). The personality strengths corresponding to the neuroticism factor are 3 for intellectual and -3 for emotional.

[0042] 9 , the index value derivation unit 46 inputs an image 56 for which a storage request has been made to the index value derivation model 37. Then, the index value derivation model 37 outputs an index value 58. The index value derivation model 37 is constructed using, for example, a convolutional neural network. The index value derivation model 37 is an example of a "trained model" according to the technology of the present disclosure.

[0043] As an example, as shown in FIG. 10 , in the learning phase of the index value derivation model 37, learning data (also referred to as teacher data or training data) 70 is prepared. The learning data 70 is composed of a pair of learning images 56L and correct answer index values ​​58CA. A plurality of learning data 70 is prepared. The correct answer index values ​​58CA are index values ​​assigned to the learning images 56L by an annotator according to a preset annotation definition, and are, so to speak, data for checking answers. The annotation definition may, for example, assign a high value to openness to a creative, artistic, and intellectually intriguing learning image 56L, or assign a high value to conscientiousness to a learning image 56L that appears to be particular about composition. Furthermore, a high value to extroversion may be assigned to an outdoor-related learning image 56L or a learning image 56L that depicts a large number of people other than family members, and a high value to harmony may be assigned to a learning image 56L depicting a smiling person or a learning image 56L depicting a family. Furthermore, a low value may be assigned to the neurotic tendency for the learning image 56L that contains a lot of achromatic colors.

[0044] In the learning phase, a learning image 56L is input to the index value derivation model 37, which then outputs a learning index value 58L. This learning index value 58L is compared with the correct index value 58CA. Then, based on the comparison result, a loss calculation is performed for the index value derivation model 37 using a loss function. Then, depending on the result of the loss calculation, various coefficients of the index value derivation model 37 (such as the coefficients of the filters in the convolution layer) are updated, and the index value derivation model 37 is updated in accordance with the update setting.

[0045] In the learning phase of the index value derivation model 37, the above-mentioned series of processes, including input of the learning images 56L to the index value derivation model 37, output of the learning index values ​​58L from the index value derivation model 37, loss calculation, update setting, and update of the index value derivation model 37, are repeatedly performed while the learning data 70 is exchanged. The repetition of the above-mentioned series of processes is terminated when the derivation accuracy of the learning index values ​​58L reaches a predetermined set level. The index value derivation model 37 whose derivation accuracy has thus reached the set level is stored in the storage 20B of the information processing server 12. Note that learning may be terminated when the above-mentioned series of processes has been repeated a set number of times, regardless of the derivation accuracy of the learning index values ​​58L.

[0046] 11 and 12 , the personality determination unit 48 includes a first acquisition unit 751, a second acquisition unit 752, a first selection unit 761, a second selection unit 762, a first derivation unit 771, a second derivation unit 772, a comparison unit 78, and a generation unit 79. The determination auxiliary information 38 includes a setting range 85 and difference-intensity correspondence information 86.

[0047] The first acquisition unit 751 causes the RW control unit 47 to read out the target image information group 90T from the image information DB 36. The first acquisition unit 751 acquires the target image information group 90T from the RW control unit 47. The first acquisition unit 751 outputs the acquired target image information group 90T to the first selection unit 761. The target image information group 90T is a collection of target image information 59T composed of a set of target images 56T, target incidental information 57T, and target image index values ​​58T related to the target user UT whose personality is to be determined. The target image information group 90T includes the target image information 59T for all target images 56T.

[0048] Furthermore, the first acquisition unit 751 causes the RW control unit 47 to read out the target attribute information 60T of the target user UT from the image information DB 36. The first acquisition unit 751 acquires the target attribute information 60T from the RW control unit 47. The first acquisition unit 751 outputs the acquired target attribute information 60T to the second acquisition unit 752.

[0049] The target user UT is a user U for whom, by storing image information 59 in response to a storage request, a predetermined number or more of image information 59 (index values ​​58) sufficient for determining the personality is stored in the storage area 55. The target user UT is also a user U for whom a predetermined number or more of image information 59 has already been stored in the storage area 55, and for whom a predetermined period has elapsed since the last time the personality determination information 61 was generated. The predetermined number is, for example, 100. The predetermined period is, for example, one year. Therefore, personality determination information 61 is not generated for a user U for whom the predetermined number or more of image information 59 has not yet been stored in the storage area 55. The personality determination information 61 is also updated every predetermined period.

[0050] The second acquisition unit 752 uses the RW control unit 47 to read the reference image information group 90R from the image information DB 36. More specifically, as shown in FIG. 13 as an example, the second acquisition unit 752 references the target attribute information 60T and instructs the RW control unit 47 to read the reference image information group 90R of reference users UR other than the target user UT who have specific attributes that match or are similar to those of the target user UT. The second acquisition unit 752 acquires the reference image information group 90R from the RW control unit 47. The second acquisition unit 752 outputs the acquired reference image information group 90R to the second selection unit 762. The reference image information group 90R is a collection of reference image information 59R composed of a set of reference images 56R, reference incidental information 57R, and reference image index values ​​58R associated with the reference user UR. The reference image information group 90R includes the reference image information 59R of all reference images 56R.

[0051] A reference user UR whose specific attributes match or are similar to those of the target user UT is a user U whose attributes match or are similar in, for example, three or more of the items in the attribute information 60, such as gender, age, address, marital history, family structure, pet animals, and hobbies. In the case of age, a match is determined if the ages are the same. Alternatively, similarity may be determined if the ages are the same, such as in their 30s or 50s. In the case of family structure, a match is determined if the number of family members, the gender of children (e.g., three boys, one boy and two girls), and the ages of the family are all the same. Alternatively, similarity may be determined if one of the number of family members, the gender of children, and the ages of the family is the same. If multiple pet animals are registered in the target attribute information 60T, a match is determined if at least one of the multiple pet animals is registered in the attribute information 60. The same applies if multiple hobbies are registered in the target attribute information 60T.

[0052] While there is one target user UT, there are multiple reference users UR. For this reason, the reference image 56R, and therefore the reference image information 59R contained in the reference image information group 90R, is far more numerous than the target image 56T, and therefore the target image information 59T contained in the target image information group 90T. Note that the reference user UR may also include the target user UT. If the number of reference users UR other than the target user UT is large, for example, several hundred to several thousand or more, even if the target user UT is included in the reference users UR, it is considered that the impact of the target user UT's index value on the statistical index value 92 is small. For this reason, it is acceptable to include the target user UT in the reference users UR and incorporate the target user UT's index value as an index value when deriving the statistical index value 92.

[0053] 11 , the first selection unit 761 selects attribute-related target image information 59TA including attribute-related target images 56TA related to attributes from the target image information 59T in the target image information group 90T. The first selection unit 761 outputs an attribute-related target image information group 90TA, which is a collection of the selected attribute-related target image information 59TA, to the first derivation unit 771. The attribute-related target images 56TA include, for example, target images 56T depicting family members, target images 56T depicting pet animals, and target images 56T depicting scenes of enjoying hobbies. The first selection unit 761 finds the attribute-related target images 56TA by, for example, referring to tags in the target incidental information 57T.

[0054] The second selection unit 762, like the first selection unit 761, selects attribute-related reference image information 59RA including attribute-related reference images 56RA related to attributes from the reference image information 59R of the reference image information group 90R. The second selection unit 762 outputs an attribute-related reference image information group 90RA, which is a collection of the selected attribute-related reference image information 59RA, to the second derivation unit 772.

[0055] The first derivation unit 771 receives a setting range 85 of the judgment auxiliary information 38. The first derivation unit 771 derives a personality index value 91 from the target image index values ​​58T of the multiple attribute-related target image information 59TA constituting the attribute-related target image information group 90TA. More specifically, as shown in FIG. 14 as an example, the first derivation unit 771 derives the average value of the target image index values ​​58T excluding values ​​outside the setting range 85 as the personality index value 91. The average value is an example of a "representative value" according to the technology disclosed herein. In other words, the personality index value 91 is a representative value of the multiple target image index values ​​58T derived from the multiple target images 56T. The first derivation unit 771 performs the process of calculating the average value excluding values ​​outside the setting range 85 for each of the five factors (openness, conscientiousness, extraversion, agreeableness, and neuroticism) of the target image index values ​​58T. The first derivation unit 771 outputs the derived personality index value 91 to the comparison unit 78 .

[0056] Similar to the first derivation unit 771, the second derivation unit 772 also receives a setting range 85 of the judgment auxiliary information 38. The second derivation unit 772 derives a statistical index value 92 from the reference image index values ​​58R of the multiple attribute-related reference image information 59RA constituting the attribute-related reference image information group 90RA. More specifically, as shown in FIG. 15 as an example, the second derivation unit 772 derives the average value of the reference image index values ​​58R excluding values ​​outside the setting range 85 as the statistical index value 92. The average value is an example of a "representative value" according to the technology disclosed herein. In other words, the statistical index value 92 is a representative value of the multiple reference image index values ​​58R derived from the multiple reference images 56R. The second derivation unit 772 performs a process of calculating the average value excluding values ​​outside the setting range 85 for each of the five factors of the reference image index values ​​58R. The second derivation unit 772 outputs the derived statistical index value 92 to the comparison unit 78. 14 and 15 show an example in which the setting range 85 is 30% or more, so values ​​less than 30%, such as 18%, 26%, 3%, 11%, 23%, and 25%, are excluded as indicated by an x.

[0057] In this way, the personality index value 91 and the statistical index value 92 are derived basically by the same method. By "same method," we mean that although the values ​​to be excluded or used to calculate the average value differ depending on the five factors, the procedures for excluding values ​​outside the set range 85 and calculating the average value using the excluded values ​​are the same.

[0058] 12 , the comparison unit 78 compares a personality index value 91 with a statistical index value 92. Specifically, as shown in FIG. 16 as an example, the comparison unit 78 calculates a difference Δ between the personality index value 91 and the statistical index value 92 by subtracting the statistical index value 92 from the personality index value 91. The comparison unit 78 calculates the difference Δ for each of the five factors. The comparison unit 78 outputs the calculated difference Δ to the generation unit 79 as a comparison result 93.

[0059] The generation unit 79 receives input of difference-intensity correspondence information 86 of the judgment auxiliary information 38. As shown in FIG. 17 as an example, the difference-intensity correspondence information 86 registers intensities corresponding to the difference Δ. That is, when the difference Δ is greater than 10 (10<Δ), the intensity is 3; when the difference Δ is greater than 5 but less than 10 (5<10≦Δ), the intensity is 2; and when the difference Δ is greater than 0 but less than 5 (0<Δ≦5), the intensity is 1. When the difference Δ is 0, the intensity is 0. When the difference Δ is less than −10 (Δ<−10), the intensity is −3; when the difference Δ is greater than −10 but less than −5 (−10≦Δ−5), the intensity is −2; and when the difference Δ is greater than −5 but less than 0 (−5≦Δ<0), the intensity is −1. As shown in FIG. 16 as an example, the generation unit 79 generates personality determination information 61 from the comparison result 93 based on the difference-intensity correspondence information 86.

[0060] The generating unit 79 outputs the generated personality determination information 61 to the RW control unit 47. The RW control unit 47 stores the personality determination information 61 in the image information DB .

[0061] Next, the operation of the above configuration will be described with reference to the flowcharts shown in Figures 18 and 19 as an example. As shown in Figure 3, the CPU 22A of the user terminal 10 functions as the browser control unit 32 when the image AP 30 is activated. As shown in Figure 4, the CPU 22B of the information processing server 12 functions as the request receiving unit 45, the index value derivation unit 46, the RW control unit 47, the personality determination unit 48, and the distribution control unit 49 when the operating program 35 is activated. As shown in Figures 11 and 12, the personality determination unit 48 includes a first acquisition unit 751, a second acquisition unit 752, a first selection unit 761, a second selection unit 762, a first derivation unit 771, a second derivation unit 772, a comparison unit 78, and a generation unit 79.

[0062] The user U takes an image 56 using the camera function of the user terminal 10. After taking the image 56, a request to store the image 56 is sent to the information processing server 12 under the control of the browser control unit 32.

[0063] In the information processing server 12, the request receiving unit 45 receives a request to store the image 56 (YES in step ST100 in FIG. 18). The request receiving unit 45 outputs the request to the index value derivation unit 46 and the like.

[0064] 9 , in the index value derivation unit 46, the image 56 is input to the index value derivation model 37, which then outputs an index value 58 (step ST110). The index value 58, together with the image 56, is output from the index value derivation unit 46 to the RW control unit 47. Then, under the control of the RW control unit 47, the image 56, the incidental information 57, and the index value 58 are associated with each other by the image ID and stored as image information 59 in the image information DB 36 of the storage 20B (step ST120).

[0065] When there is a target user UT for which personality assessment information 61 should be generated (YES in step ST200 of FIG. 19 ), as shown in FIG. 11 , the RW control unit 47 reads out a target image information group 90T and target attribute information 60T of the target user UT from the image information DB 36. The target image information group 90T and the target attribute information 60T are output from the RW control unit 47 to the first acquisition unit 751 and acquired by the first acquisition unit 751 (step ST210). The target image information group 90T is output from the first acquisition unit 751 to the first selection unit 761. Furthermore, the target attribute information 60T is output from the first acquisition unit 751 to the second acquisition unit 752.

[0066] 11 and 13, the RW control unit 47 reads out, from the image information DB 36, a reference image information group 90R of a reference user UR whose attributes match or are similar to those of the target user UT. The reference image information group 90R is output from the RW control unit 47 to the second acquisition unit 752 and acquired by the second acquisition unit 752 (step ST220). The reference image information group 90R is output from the second acquisition unit 752 to the second selection unit 762.

[0067] The first selection unit 761 selects attribute-related target image information 59TA including the attribute-related target image 56TA from the target image information 59T in the target image information group 90T (step ST230). The attribute-related target image information group 90TA, which is a collection of the attribute-related target image information 59TA, is output from the first selection unit 761 to the first derivation unit 771.

[0068] The second selection unit 762 selects attribute-related reference image information 59RA including the attribute-related reference image 56RA from the reference image information 59R of the reference image information group 90R (step ST240). The attribute-related reference image information group 90RA, which is a collection of the attribute-related reference image information 59RA, is output from the second selection unit 762 to the second derivation unit 772.

[0069] 14 , the first derivation unit 771 derives, as the personality index value 91, the average value of the target image index values ​​58T of the attribute-related target image information 59TA, excluding values ​​outside the set range 85 (step ST250). The personality index value 91 is output from the first derivation unit 771 to the comparison unit 78.

[0070] 15 , the second derivation unit 772 derives the average value of the reference image index values ​​58R of the attribute-related reference image information 59RA, excluding values ​​outside the setting range 85, as the statistical index value 92 (step ST260). The statistical index value 92 is output from the second derivation unit 772 to the comparison unit 78.

[0071] 16 , the comparison unit 78 compares the personality index value 91 with the statistical index value 92 and calculates the difference Δ between the personality index value 91 and the statistical index value 92 (step ST270). The comparison result 93 including the difference Δ is output from the comparison unit 78 to the generation unit 79.

[0072] 16 , the generation unit 79 generates personality determination information 61 of the target user UT based on the comparison result 93 while referring to the difference-intensity correspondence information 86 (step ST280). The personality determination information 61 is output from the generation unit 79 to the RW control unit 47. Under the control of the RW control unit 47, the personality determination information 61 is stored in the image information DB 36 of the storage 20B (step ST290).

[0073] The personality determination information 61 is delivered to the user terminal 10 in response to a delivery request from the user U and is made available for viewing by the user U. The personality determination information 61 is also referenced when selecting images 56 that will form the basis of a photo album to be recommended to the user U. For example, for a user U who is determined to have an introverted and intellectual personality, indoor images 56, images 56 that do not include people, images 56 that are not colorful, etc. are preferentially selected as the basis of images 56 for a photo album to be recommended to the user U. The personality determination information 61 may also be referenced when selecting the content of commercials to be provided to the user U through the web browser of the image AP 30.

[0074] As described above, the comparison unit 78 of the personality determination unit 48 of the CPU 22B of the information processing server 12 acquires the personality index value 91 derived from the target image 56T related to the target user UT whose personality is to be determined, and compares the personality index value 91 with the statistical index values ​​92 of the multiple reference users UR. The generation unit 79 generates personality determination information 61 based on the comparison result 93, thereby determining the personality of the target user UT.

[0075] If the personality of the target user UT is determined based only on the personality index value 91 without comparing it with the statistical index value 92, the personality determination information 61 will be significantly influenced by the subject appearing in the target image 56T. For example, a user U with young children in the family tends to take many images 56 of young children. Such a user U will have a high level of harmony simply because they have many images 56 of young children.

[0076] On the other hand, if the personality index value 91 is compared with the statistical index value 92 and personality determination information 61 is generated based on the comparison result 93, the personality of the target user UT can be determined by comparison with the statistical personalities of multiple reference users UR that serve as a standard. Therefore, it becomes possible to determine the personality of the user U with high accuracy from the image 56.

[0077] 14, the personality index value 91 is a representative value of a plurality of target image index values ​​58T derived from a plurality of target images 56T associated with the target user UT. Also, as shown in FIG. 15, the statistical index value 92 is a representative value of a plurality of reference image index values ​​58R derived from a plurality of reference images 56R associated with the reference user UR. This allows for easy comparison of the personality index value 91 and the statistical index value 92.

[0078] As shown in Figure 14, the personality index value 91 is the average value of the target image index values ​​58T excluding values ​​outside the set range 85. Also, as shown in Figure 15, the statistical index value 92 is the average value of the reference image index values ​​58R excluding values ​​outside the set range 85. If the average value is calculated without excluding values ​​outside the set range 85, the characteristics necessary for personality assessment will be averaged. Therefore, by calculating the average value excluding values ​​outside the set range 85, it is possible to derive a more valid personality index value 91 and statistical index value 92 that represent the characteristics necessary for personality assessment.

[0079] 9, the index values ​​58, i.e., the target image index value 58T and the reference image index value 58R, are derived using the index value derivation model 37. Therefore, the target image index value 58T and the reference image index value 58R can be derived simply and with high accuracy.

[0080] The comparison unit 78 compares the personality index value 91 with the statistical index value 92 of a reference user UR whose specific attribute matches or is similar to that of the target user UT. This makes it possible to improve the accuracy of personality determination compared to when a reference user UR who is not related to the target user UT or whose attribute is different from that of the target user UT is involved.

[0081] The comparison unit 78 acquires the personality index value 91 derived from the attribute-related target image 56TA related to the attribute of the target image 56T. Therefore, it is possible to determine the personality of the target user UT according to the attribute.

[0082] There are a plurality of types of personality index values ​​91 and statistical index values ​​92. Therefore, the personality of the user U can be determined from a variety of perspectives.

[0083] Second Embodiment While the personality index value 91 derived from the attribute-related target image 56TA is used in this embodiment, this is not limiting. As an example, the first selection unit 761 and the second selection unit 762 may be omitted, as in the personality determination unit 100 shown in FIG. 20 . In this case, the first acquisition unit 751 outputs a target image information group 90T to the first derivation unit 771, and the second acquisition unit 752 outputs a reference image information group 90R to the second derivation unit 772. The first derivation unit 771 then derives personality index values ​​91 from the target image index values ​​58T of all target images 56T and outputs the derived personality index values ​​91 to the comparison unit 78. The second derivation unit 772 also derives statistical index values ​​92 from the reference image index values ​​58R of all reference images 56R and outputs the derived statistical index values ​​92 to the comparison unit 78.

[0084] In this way, in the second embodiment, the comparison unit 78 acquires the personality index values ​​91 derived from all of the target images 56T. Therefore, it is possible to determine the personality of the target user UT that is not so restricted by attributes.

[0085] The first embodiment and the second embodiment may be combined and implemented. In this case, the generation unit 79 generates personality determination information 61 based on the personality index values ​​91 derived from the attribute-related target images 56TA and personality determination information 61 based on the personality index values ​​91 derived from all the target images 56T. When selecting images 56 to be used as the basis for a photo album to be recommended to the user U, the images 56 are selected in a balanced manner by referring to both the personality determination information 61 based on the personality index values ​​91 derived from the attribute-related target images 56TA and the personality determination information 61 based on the personality index values ​​91 derived from all the target images 56T. For example, the images 56 are selected overall by referring to the personality determination information 61 based on the personality index values ​​91 derived from all the target images 56T. Then, the images 56 to be arranged on a specific page are selected by referring to the personality determination information 61 based on the personality index values ​​91 derived from the attribute-related target images 56TA.

[0086] The strength of personality is not limited to the seven levels shown in the example. It may be fewer or more than seven levels. A different setting range 85 may be prepared for each of the five factors. The representative value is not limited to the average value shown in the example, but may be the mode, median, etc.

[0087] Although the difference Δ is exemplified as the comparison result 93 between the personality index value 91 and the statistical index value 92, the comparison result 93 is not limited to this. The comparison result 93 may be a ratio obtained by dividing the personality index value 91 by the statistical index value 92.

[0088] The personality determination unit 48 may be operated in response to a request from the user U, and the personality of the user U who issued the request may be determined as the target user UT. In addition, the statistical index value 92 may be derived in advance for each attribute and stored in the storage 20B.

[0089] The index values ​​include, but are not limited to, the five exemplary factors of openness, conscientiousness, extraversion, agreeableness, and neuroticism.

[0090] The hardware configuration of the computer that constitutes the information processing server 12 can be modified in various ways. For example, the information processing server 12 can be configured with multiple computers separated as hardware in order to improve processing power and reliability. For example, the functions of the request reception unit 45 and the index value derivation unit 46 and the functions of the RW control unit 47, the personality determination unit 48 or 100, and the distribution control unit 49 can be distributed and performed by two computers. In this case, the information processing server 12 is configured with two computers. In addition, all or part of the functions of the information processing server 12 may be performed by the user terminal 10.

[0091] In this way, the hardware configuration of the computer of the user terminal 10 and the information processing server 12 can be changed as appropriate depending on the required performance such as processing power, safety, reliability, etc. Furthermore, not only the hardware, but also APs such as the image AP 30 and the operating program 35 can be duplicated or stored in a distributed manner in multiple storages in order to ensure safety and reliability.

[0092] In each of the above embodiments, for example, the hardware structure of the processing unit that executes various processes, such as the browser control unit 32, the request receiving unit 45, the index value derivation unit 46, the RW control unit 47, the personality determination units 48 and 100, the distribution control unit 49, the first acquisition unit 751, the second acquisition unit 752, the first selection unit 761, the second selection unit 762, the first derivation unit 771, the second derivation unit 772, the comparison unit 78, and the generation unit 79, can be any of the various processors shown below. The various processors include CPUs 22A and 22B, which are general-purpose processors that execute software (image AP 30 and operating program 35) and function as various processing units, as well as programmable logic devices (PLDs), which are processors whose circuit configuration can be changed after manufacture, such as FPGAs (Field Programmable Gate Arrays), and / or dedicated electrical circuits, such as ASICs (Application Specific Integrated Circuits), which are processors having a circuit configuration designed specifically for executing specific processing.

[0093] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0094] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0095] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0096] From the above description, the technology described in the following supplementary paragraphs can be understood.

[0097] [Supplementary Item 1] An information processing device including a processor, the processor acquiring a personality index value derived from a target image related to a target user whose personality is to be determined, comparing the personality index value with statistical index values ​​of a plurality of reference users, and determining the personality of the target user based on the comparison result. [Supplementary Item 2] The information processing device according to Supplementary Item 1, wherein the personality index value is a representative value of index values ​​for each of a plurality of target images derived from the plurality of target images, and the statistical index value is a representative value of index values ​​for each of a plurality of reference images derived from a plurality of reference images related to the reference user. [Supplementary Item 3] The information processing device according to Supplementary Item 2, wherein there are a plurality of types of personality index value and statistical index value, the personality index value is an average value of index values ​​for each of the target images excluding values ​​outside a set range, and the statistical index value is an average value of index values ​​for each of the reference images excluding values ​​outside the set range. [Supplementary Item 4] The information processing device according to Supplementary Item 2 or Supplementary Item 3, wherein the index value is derived using a trained model. [Supplementary Item 5] The information processing device according to any one of Supplementary Items 1 to 4, wherein the processor compares the personality index value with the statistical index value of the reference user whose specific attribute matches or is similar to that of the target user. [Supplementary Item 6] The information processing device according to Supplementary Item 5, wherein the processor acquires the personality index value derived from attribute-related target images related to the attribute among the target images. [Supplementary Item 7] The information processing device according to Supplementary Item 5 or Supplementary Item 6, wherein the processor acquires the personality index value derived from all the target images. [Supplementary Item 8] The information processing device according to any one of Supplementary Items 1 to 7, wherein there are a plurality of types of personality index value and statistical index value.

[0098] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs, but also to storage media that non-temporarily store programs, and computer program products that include programs.

[0099] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0100] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0101] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. An information processing device comprising a processor, the processor acquiring a personality index value derived from a target image related to a target user whose personality is to be determined, comparing the personality index value with statistical index values ​​of a plurality of reference users, and determining the personality of the target user based on the comparison result.

2. The information processing device described in claim 1, wherein the personality index value is a representative value of index values ​​for each of a plurality of target images derived from a plurality of the target images, and the statistical index value is a representative value of index values ​​for each of a plurality of reference images derived from a plurality of reference images related to the reference user.

3. An information processing device as described in claim 2, wherein there are multiple types of personality index values ​​and statistical index values, the personality index value is the average value of index values ​​for each of the target images excluding values ​​outside a set range, and the statistical index value is the average value of index values ​​for each of the reference images excluding values ​​outside a set range.

4. The information processing device according to claim 2, wherein the index value is derived using a trained model.

5. The information processing device according to claim 1, wherein the processor compares the personality index value with the statistical index value of the reference user whose specific attribute matches or is similar to that of the target user.

6. The information processing device according to claim 5, wherein the processor acquires the personality index value derived from an attribute-related target image related to the attribute among the target images.

7. The information processing device according to claim 5, wherein the processor acquires the personality index values ​​derived from all of the target images.

8. The information processing device according to claim 1, wherein there are a plurality of types of said personality index value and said statistical index value.

9. A method for operating an information processing device, comprising: obtaining a personality index value derived from a target image associated with a target user whose personality is to be determined; comparing the personality index value with statistical index values ​​of a plurality of reference users; and determining the personality of the target user based on the comparison result.

10. An operating program for an information processing device that causes a computer to execute a process including: obtaining a personality index value derived from a target image related to a target user whose personality is to be judged; comparing the personality index value with statistical index values ​​of a plurality of reference users; and judging the personality of the target user based on the comparison result.

Images