Information processing apparatus, method of operation of an information processing apparatus, and program for an information processing apparatus

Abstract

An information processing apparatus includes a processor that performs the following processing: acquiring personality index values ​​derived from an object image related to a target user whose personality is being determined; comparing the personality index values ​​with statistical index values ​​of multiple reference users; and determining the personality of the target user based on the comparison results.

Description

Technical Field

[0001] The present invention relates to an information processing device, a method for operating the information processing device, and a procedure for operating the information processing device. Background Technology

[0002] The facial pattern analysis system described in Japanese Patent Application Publication No. 2010-140100 includes a facial image data recording unit, an average facial image data recording unit, a facial subject image extraction unit, a facial part image determination and extraction unit, a facial part image measurement unit, a difference data calculation unit, a grade determination unit, and a facial pattern determination unit. The facial image data recording unit receives and temporarily stores digitally captured facial image photographs. The average facial image data recording unit stores data of an average facial image, including an average facial image pattern and measurement values ​​of the eyes / nose / contour of the average facial image. The facial subject image extraction unit, based on the temporarily stored facial image data, determines and extracts the position of the facial subject from the facial image data using pattern matching based on the average facial image pattern and stores it. The facial part image determination and extraction unit, based on the extracted facial subject image, determines and extracts images of at least the eyes / nose / contour of the face. The facial part image measurement unit measures at least the area / shape / position (coordinates) / tilt of the extracted facial part image and temporarily stores these measurement values. The differential data calculation unit uses a prescribed formula to calculate numerical data on the differences between interocular distance, nose length, and contour (horizontal to vertical length) based on the measured values ​​of the eyes, nose, and contour obtained by the facial image measurement unit and the measured values ​​of the eyes, nose, and contour in the average facial image. The grading unit classifies the interocular distance, nose length, and contour (horizontal to vertical length) into 3 to 5 grades based on the differential data obtained by the differential data calculation unit. The facial pattern determination unit determines, based on the combination of the grades for each of the interocular distance, nose length, and contour (horizontal to vertical length) determined by the grading unit, which of the 4 to 8 pre-defined facial patterns, based on the impression given to facial features, belongs to the received facial image data, and outputs the determination data.

[0003] The feature impression axis determination system described in Japanese Patent Application Publication No. 2018-156136 includes an image input unit, a reference impression value determination unit, and a feature impression axis determination unit. The image input unit inputs multiple images. The reference impression value determination unit determines the reference impression value of each of the multiple images input to the image input unit based on at least two reference impression axes. The feature impression axis determination unit determines the axis with the largest variance of the reference impression values ​​determined by the reference impression value determination unit within the reference region defined by the at least two reference impression axes as the feature impression axis representing the feature of the multiple images input to the image input unit.

[0004] The image evaluation apparatus described in Japanese Patent Application Publication No. 2001-195579 includes a feature extraction unit, a holding unit, a calculation unit, a text creation unit, and an output control unit. The feature extraction unit extracts features from the input image. The holding unit holds adjectives and their corresponding features. The calculation unit calculates the difference between the features extracted by the feature extraction unit and the features corresponding to the adjectives held in the holding unit. The text creation unit uses the calculation results from the calculation unit to create a text. The output control unit controls the output of the text created by the text creation unit. Summary of the Invention

[0005] One embodiment of the present invention provides an information processing device capable of accurately determining a user's personality from an image, a method for operating the information processing device, and a procedure for operating the information processing device.

[0006] means for solving technical problems

[0007] The information processing apparatus of the present invention includes a processor that performs the following processing: acquiring personality index values ​​derived from an object image related to a target user whose personality is being determined; comparing the personality index values ​​with statistical index values ​​of multiple reference users; and determining the personality of the target user based on the comparison results.

[0008] The preferred personality index value is a representative value of the index value of each of the multiple object images derived from multiple object images, and the statistical index value is a representative value of the index value of each of the multiple reference images derived from multiple reference images related to the reference user.

[0009] There are multiple preferred personality index values ​​and statistical index values. The personality index value is the average value of the index values ​​of each object image after removing values ​​outside the set range, and the statistical index value is the average value of the index values ​​of each reference image after removing values ​​outside the set range.

[0010] The optimal index values ​​are derived using the learned model.

[0011] The preferred processor performs the following processing: comparing the personality index values ​​with the statistical index values ​​of the reference user, where the specific attributes of the reference user and the target user are consistent or similar.

[0012] The preferred processor performs the following processing: obtains personality index values ​​derived from the attribute-related object image in the object image.

[0013] The preferred processor performs the following processing: obtains personality index values ​​derived from all object images.

[0014] There are multiple optimal personality index values ​​and statistical index values.

[0015] The operating method of the information processing device of the present invention includes the following steps: acquiring personality index values ​​derived from object images related to the object user whose personality is being determined; comparing the personality index values ​​with statistical index values ​​of multiple reference users; and determining the personality of the object user based on the comparison results.

[0016] The operating procedure of the information processing apparatus of the present invention enables a computer to perform a process including the following steps: acquiring personality index values ​​derived from an object image related to the object user whose personality is being determined; comparing the personality index values ​​with statistical index values ​​of multiple reference users; and determining the personality of the object user based on the comparison results. Attached Figure Description

[0017] Figure 1 This is a diagram representing the user terminal and the information processing server.

[0018] Figure 2 This is a block diagram representing the computer that constitutes the user terminal and the information processing server.

[0019] Figure 3 This is a block diagram representing the CPU processing unit of a user terminal.

[0020] Figure 4 This is a block diagram representing the processing unit of the CPU in an information processing server.

[0021] Figure 5 It is a graph representing data stored in the image information database.

[0022] Figure 6 It is a graph that represents image information.

[0023] Figure 7 It is a graph that represents attribute information.

[0024] Figure 8 It is a diagram that represents information about personality assessment.

[0025] Figure 9 This is a diagram representing the processing of the indicator value derivation section.

[0026] Figure 10 This is a graph representing the processing during the learning phase of the model from which index values ​​are derived.

[0027] Figure 11 This is a block diagram showing the detailed structure of the personality assessment department.

[0028] Figure 12 This is a block diagram showing the detailed structure of the personality assessment department.

[0029] Figure 13It is a diagram that represents the object image information group of the object user and the reference image information group of the reference user that is consistent with or similar to the specific attributes of the object user.

[0030] Figure 14 This is a diagram showing the processing of the first derived section.

[0031] Figure 15 This is a diagram showing the processing of the second derived section.

[0032] Figure 16 This is a diagram illustrating the processing of the comparison and generation parts.

[0033] Figure 17 It is a graph representing the difference-intensity correspondence information.

[0034] Figure 18 This is a flowchart illustrating the processing sequence of the information processing server.

[0035] Figure 19 This is a flowchart illustrating the processing sequence of the information processing server.

[0036] Figure 20 This is a block diagram showing the detailed structure of the personality determination unit in the second embodiment. Detailed Implementation

[0037] [First Implementation]

[0038] As an example, such as Figure 1 As shown, user U has user terminal 10. User terminal 10 is a device with camera function, image playback and display function, image editing function, and image transmission and reception function. The camera function has imaging elements such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and obtains an image 56 (reference) of the subject by imaging the subject light taken from the lens in the imaging element. Figure 5 The user terminal 10 is a digital camera, smartphone, tablet computer, or laptop computer, etc. The user U uses the camera function to capture images 56, or uses the image editing function to edit images 56 according to personal preferences.

[0039] User terminal 10 is communicatively connected to information processing server 12 via network 11. Network 11 is, for example, the Internet or a public communication network (WAN). User terminal 10 sends (uploads) image 56 to information processing server 12. Furthermore, user terminal 10 receives (downloads) image 56 from information processing server 12.

[0040] The information processing server 12 is, for example, a server computer, a workstation, etc., and is an example of an "information processing apparatus" according to the technology of this invention. Multiple user terminals 10 of multiple users U are connected to the information processing server 12 via a network 11.

[0041] As an example, such as Figure 2 As shown, the computers constituting the user terminal 10 and the information processing server 12 have essentially the same structure, including a storage device 20, a memory 21, a CPU (Central Processing Unit) 22, a communication unit 23, a display 24, and an input device 25. They are connected to each other via a bus 26.

[0042] Storage device 20 is a hard disk drive built into the computer constituting user terminal 10 and information processing server 12 or connected via cable or network. Alternatively, storage device 20 is a hard disk array consisting of multiple hard disk drives connected together. Storage device 20 stores control programs such as operating systems, various application programs (hereinafter referred to as APs), and various data associated with these programs. Alternatively, solid-state drives can be used instead of hard disk drives.

[0043] Memory 21 is a working memory used by CPU 22 for performing processing. CPU 22 loads the program stored in storage device 20 into memory 21 and executes the processing according to the program. Thus, CPU 22 centrally controls the various parts of the computer. CPU 22 is an example of a "processor" according to the technology of this invention. In addition, memory 21 may be built into CPU 22.

[0044] The communication unit 23 is a network interface for transmitting and controlling various information via the network 11, etc. The display 24 shows various screens. These screens have GUI (Graphical User Interface) based operation functions. The computer constituting the user terminal 10 and the information processing server 12 receives input from the input device 25 through these screens. The input device 25 includes a keyboard, mouse, touch panel, and microphone for voice input, etc.

[0045] In addition, in the following description, the components of the computer constituting the user terminal 10 (storage device 20, CPU 22, display 24 and input device 25) are marked with the subscript "A" in the reference numerals, and the components of the computer constituting the information processing server 12 (storage device 20 and CPU 22) are marked with the subscript "B" in the reference numerals to distinguish them.

[0046] As an example, such as Figure 3As shown, image AP 30 is stored in storage device 20A of user terminal 10. Image AP 30 is installed on user terminal 10 by user U. Image AP 30 is an AP used for playing, displaying, or editing image 56 on user terminal 10. When image AP 30 is activated, CPU 22A of user terminal 10, together with memory 21, etc., functions as browser control unit 32. Browser control unit 32 controls the operation of a web browser dedicated to image AP 30.

[0047] The browser control unit 32 receives screen data from various screens on the information processing server 12. Based on the screen data, the browser control unit 32 reproduces the various screens displayed on the web browser and displays them on the monitor 24A. Furthermore, the browser control unit 32 receives various operation instructions input by the user U from the input device 25A through these screens. The browser control unit 32 sends various requests corresponding to the operation instructions to the information processing server 12.

[0048] As an example, such as Figure 4 As shown, an operating program 35 is stored in the storage device 20B of the information processing server 12. The operating program 35 is an AP that enables the computer constituting the information processing server 12 to function as an "information processing apparatus" according to the technology of this invention. That is, the operating program 35 is an example of an "operating program of an information processing apparatus" according to the technology of this invention.

[0049] The storage device 20B also stores an image information database (hereinafter referred to as DB) 36, an indicator value derivation model 37, and judgment assistance information 38. Furthermore, although the illustration is omitted, the storage device 20B stores a user ID (Identification Data) used to uniquely identify user U, a password set by user U, and a terminal ID used to uniquely identify user terminal 10, serving as user U's account information.

[0050] If the working program 35 is started, the CPU 22B of the information processing server 12, together with the memory 21, etc., will function as the request receiving unit 45, the index value exporting unit 46, the read / write (hereinafter referred to as RW (Read Write)) control unit 47, the personality determination unit 48, and the distribution control unit 49.

[0051] The request receiving unit 45 receives various requests from the user terminal 10. The request receiving unit 45 outputs these requests to the index value exporting unit 46 and / or the RW control unit 47 and the distribution control unit 49. Among the various requests are storage requests for storing image 56 in image information DB36, distribution requests for distributing image 56 stored in image information DB36, etc. Each request includes the terminal ID of the user terminal 10, the source of the request.

[0052] The indicator value exporting unit 46 exports indicator value 58 from the image 56 that has been requested to be stored (see reference). Figure 5 The index value 58 serves as a clue for determining the personality of user U, who took image 56. The index value derivation unit 46 outputs the derived index value 58 along with image 56 to the RW control unit 47. Here, "personality" refers to an individual's preferences, such as what they like or what behaviors they enjoy. The tendency to analyze an individual's preferences that may help in estimating what they like, and the value representing that tendency (which can be a scalar value or a vector value composed of the five factors described later in this example: openness, conscientiousness, extraversion, agreeableness, and neuroticism) are included in the term "personality".

[0053] The RW control unit 47 controls the storage of various data to and from the storage device 20B. Specifically, the RW control unit 47 controls the storage of image 56 to and from image information DB36. Furthermore, the RW control unit 47 also controls the storage of index value 58 to and from image information DB36.

[0054] The RW control unit 47 reads the index value export model 37 from the storage device 20B and outputs the read index value export model 37 to the index value export unit 46. Furthermore, the RW control unit 47 reads the judgment assistance information 38 from the storage device 20B and outputs the read judgment assistance information 38 to the personality judgment unit 48.

[0055] The personality determination unit 48 refers to the determination auxiliary information 38 and determines the personality of user U based on the index value 58. The personality determination unit 48 uses the personality determination information 61 (referencing) as the personality determination result. Figure 5 The information is output to the RW control unit 47. The RW control unit 47 stores the personality determination information 61 in the image information DB36. Hereinafter, the user U who determines the personality will be referred to as the target user UT (see reference). Figure 13 Furthermore, the image 56 related to the object user UT is represented as the object image 56T (see reference). Figure 11 Furthermore, the accompanying information 57 of the object image 56T (see reference) Figure 5 The index value 58 is described as the object's attached information 57T and the object's image index value 58T (both referenced). Figure 11 Furthermore, image information 59, consisting of object image 56T, object-related information 57T, and object image index values ​​58T (see reference), will be used. Figure 5 ) is expressed as object image information 59T (reference) Figure 11 Furthermore, the attribute information of the object user UT (reference 60) will be included. Figure 5 ) is expressed as object attribute information 60T (reference) Figure 11 ).

[0056] The distribution control unit 49 controls the distribution of various data, including screen data, to the user terminal 10. The distribution control unit 49 distributes the screen data to the user terminal 10 in the form of web page distribution screen data, for example, using a markup language such as XML (Extensible Markup Language). Alternatively, other data description languages ​​such as JSON (Javascript Object Notation) can be used instead of XML.

[0057] As an example, such as Figure 5 As shown, in the image information DB36, a storage area 55 is set up for each user U, such as user U1 and user U2. The user ID is registered in the storage area 55. The storage area 55 stores multiple images 56, their accompanying information 57, and indicator values ​​58. Hereinafter, the group of images 56, accompanying information 57, and indicator values ​​58 will be referred to as image information 59. Furthermore, attribute information 60 and personality judgment information 61 are also stored in the storage area 55.

[0058] Images 56 stored in storage area 55 are associated with user U, whose user ID is registered in storage area 55. Images 56 include images taken by user U through the camera function of user terminal 10, images of photos read by user U through a scanner, images downloaded by user U from an Internet website, or images obtained by user U from an acquaintance, etc. In addition, images 56 "associated" with user U are typically images 56 that user U himself keeps on user terminal 10 or cloud servers such as information processing server 12. Furthermore, even images 56 kept by others other than user U, as long as user U has access rights and user U can freely edit them, are also included in the images 56 "associated" with user U.

[0059] As an example, such as Figure 6As shown, image 56, accompanying information 57, and index value 58, which constitute image information 59, are associated through image ID. Accompanying information 57 includes several items such as shooting date and time, shooting location, and tags. The shooting date and time are recorded as the date and time when image 56 was taken using the camera function of user terminal 10. The shooting location includes the address and / or landmark name derived from the latitude and longitude information obtained through the GPS (Global Positioning System) function of user terminal 10. Tags are concise words representing the subject reflected in image 56. Tags include tags manually entered by user U or tags derived from the learned model using subject discrimination. In addition, although the illustration is omitted, accompanying information 57 also includes items such as exposure value, ISO (International Organization for Standardization) sensitivity, shutter speed, focal length, and whether a flash is present.

[0060] The index value 58 includes five factors: Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism. These are factors representing human personality based on the so-called Big Five personality theory, ranging from a minimum of 0% to a maximum of 100%. Since these are widely known, detailed explanations are omitted, but in short, Openness measures whether someone is more creative or conservative; Conscientiousness measures whether someone is more diligent or lazy; Extraversion measures whether someone is more extroverted or introverted; Agreeableness measures whether someone is more cooperative or exclusive; and Neuroticism measures whether someone is more rational or emotional.

[0061] As an example, such as Figure 7 As shown, attribute information 60 is information that literally represents the attributes of user U, including items such as user U's gender, age, address, marital status, family structure, pets, and hobbies. The address includes prefectures and municipalities. Marital status includes single, married, divorced, and widowed. Family structure includes kinship relationships within the same household and ages. Pets include the types of pets such as dogs, cats, ferrets, turtles, and medaka. Multiple pets and hobbies can be registered. Attribute information can be obtained, for example, by having user U answer a questionnaire when user U installs image AP30 onto user terminal 10. Additionally, user U's birthplace, favorite athletes, celebrities, sports teams, etc., can also be included in the attribute information. Furthermore, attributes can also be determined from image 56.

[0062] As an example, such as Figure 8As shown, personality assessment information 61 is a set of personality intensities corresponding to the five factors of index value 58. Regarding personality intensity, each factor has seven levels: 3, 2, 1, 0, -1, -2, and -3. For the personality intensity corresponding to the openness factor, creativity is assigned at 3, and conservatism at -3 on the opposite side. In terms of word count, 3 represents very creative, 2 represents fairly creative, 1 represents slightly creative, -1 represents slightly conservative, -2 represents fairly conservative, -3 represents very conservative, and 0 represents neither. Similarly, for the personality intensity corresponding to the conscientiousness factor, diligence is assigned at 3, and laziness at -3 on the opposite side. For the personality intensity corresponding to the extraversion factor, extraversion is assigned at 3, and introversion at -3 on the opposite side. For the personality intensity corresponding to the agreeableness factor, cooperativeness is assigned at 3, and exclusiveness at -3 on the opposite side. For the personality intensity corresponding to the neuroticism tendency factor, rationality is assigned at point 3, and emotionality is assigned at point -3 on the opposite side.

[0063] As an example, such as Figure 9 As shown, the index value derivation unit 46 inputs the image 56, for which a storage request has been made, into the index value derivation model 37. Then, the index value 58 is output from the index value derivation model 37. The index value derivation model 37 is constructed, for example, by a convolutional neural network. The index value derivation model 37 is an example of a "learned model" according to the technology of the present invention.

[0064] As an example, such as Figure 10 As shown, in the learning phase of the indicator value derivation model 37, learning data (also called teaching data or training data) 70 is prepared. Learning data 70 consists of a set of learning images 56L and correct answer indicator values ​​58CA. Multiple sets of learning data 70 are prepared. The correct answer indicator value 58CA is an indicator value attached to the learning images 56L by the annotator according to pre-defined annotation definitions; it can be considered data used to check answers. Annotation definitions include, for example, assigning high openness values ​​to learning images 56L that are creative, artistic, and inquisitive; assigning high conscientiousness values ​​to learning images 56L that appear to be meticulous in composition; assigning high extroversion values ​​to outdoor-related learning images 56L or learning images 56L that depict a large number of people other than their families; assigning high agreeableness values ​​to learning images 56L that depict smiling people or families; and assigning low neuroticism values ​​to learning images 56L that are mostly achromatic.

[0065] During the learning phase, the learning image 56L is input into the index value derivation model 37, from which the learning index value 58L is output. This learning index value 58L is compared with the correct answer index value 58CA. Then, based on the comparison result, a loss calculation is performed on the index value derivation model 37 using a loss function. Next, based on the result of the loss calculation, the various coefficients of the index value derivation model 37 (filter coefficients of convolutional layers, etc.) are updated, and the index value derivation model 37 is updated according to these updated settings.

[0066] During the learning phase of the index value derivation model 37, the following series of processes are repeatedly performed while the learning data 70 is being changed: inputting the learning image 56L to the index value derivation model 37, outputting the learning index value 58L from the index value derivation model 37, loss calculation, setting update, and updating the index value derivation model 37. This series of processes continues until the derivation accuracy of the learning index value 58L reaches a preset level. The index value derivation model 37, with the derivation accuracy reaching the preset level, is then stored in the storage device 20B of the information processing server 12. Alternatively, the learning can end after repeating the above series of processes a set number of times, regardless of the derivation accuracy of the learning index value 58L.

[0067] As an example, such as Figure 11 and Figure 12 As shown, the personality determination unit 48 includes a first acquisition unit 751, a second acquisition unit 752, a first selection unit 761, a second selection unit 762, a first derivation unit 771, a second derivation unit 772, a comparison unit 78, and a generation unit 79. Furthermore, the determination assistance information 38 includes a setting range 85 and differential-intensity correspondence information 86.

[0068] The first acquisition unit 751 causes the RW control unit 47 to read the object image information group 90T from the image information DB36. The first acquisition unit 751 acquires the object image information group 90T from the RW control unit 47. The first acquisition unit 751 outputs the acquired object image information group 90T to the first selection unit 761. The object image information group 90T is a collection of object image information 59T consisting of object images 56T related to the object user UT whose personality is determined, object-related information 57T, and object image index values ​​58T. The object image information group 90T contains the object image information 59T of all object images 56T.

[0069] Furthermore, the first acquisition unit 751 causes the RW control unit 47 to read the object attribute information 60T of the object user UT from the image information DB36. The first acquisition unit 751 acquires the object attribute information 60T from the RW control unit 47. The first acquisition unit 751 outputs the acquired object attribute information 60T to the second acquisition unit 752.

[0070] The target user UT is a user U whose image information 59 (index value 58) corresponding to the storage request is stored in the storage area 55 at a predetermined number or more, sufficient to determine personality. Furthermore, the target user UT is a user U whose image information 59 is stored in the storage area 55 at a predetermined number or more, and a user U for whom a predetermined period has elapsed since the last generation of personality determination information 61. The predetermined number is, for example, 100. The predetermined period is, for example, 1 year. Therefore, for user Us whose image information 59 is not yet stored in the storage area 55 at a predetermined number or more, personality determination information 61 is not generated. The personality determination information 61 is updated according to the predetermined period.

[0071] The second acquisition unit 752 causes the RW control unit 47 to read the reference image information group 90R from the image information DB36. More specifically, as an example, such as... Figure 13 As shown, the second acquisition unit 752 references the object attribute information 60T and instructs the RW control unit 47 to read the reference image information group 90R of the reference user UR (other than the object user UT) and whose specific attributes are consistent with or similar to those of the object user UT. The second acquisition unit 752 acquires the reference image information group 90R from the RW control unit 47. The second acquisition unit 752 outputs the acquired reference image information group 90R to the second selection unit 762. The reference image information group 90R is a collection of reference image information 59R consisting of a group of reference images 56R, reference supplementary information 57R, and reference image index values ​​58R related to the reference user UR. The reference image information group 90R contains the reference image information 59R of all reference images 56R.

[0072] Reference users UR that are identical or similar to target user UT in specific attributes are users U who have three or more identical or similar items in each of the following attributes in attribute information 60: gender, age, address, marital status, family structure, pets, and hobbies. In the case of age, if the ages are the same, they are considered identical. Alternatively, they can be considered similar if they are in the same age group (e.g., 30 years old, 50 years old). In the case of family structure, if the number of family members, the gender of the children (e.g., 3 boys, 1 boy, and 2 girls), and the family age group are all the same, they are considered identical. Alternatively, they can be considered similar if one of the following is the same: number of family members, the gender of the children, or the family age group. If multiple pets are registered in target attribute information 60T, and one of those pets is also registered in attribute information 60, they are considered identical. The same applies if multiple hobbies are registered in target attribute information 60T.

[0073] The target user UT is one person, while the reference users UR are multiple people. Therefore, the reference image information 59R contained in the reference image 56R and the reference image information group 90R is much more than the object image information 59T contained in the target image 56T and the object image information group 90T. Furthermore, the reference user UR may include the target user UT. If the number of reference users UR other than the target user UT is, for example, hundreds to thousands or more, then even if the reference user UR includes the target user UT, the influence of the indicator value based on the target user UT on the statistical indicator value 92 is small. Therefore, it is not a problem to include the target user UT in the reference user UR and to include the indicator value of the target user UT as the indicator value when deriving the statistical indicator value 92.

[0074] exist Figure 11 In this process, the first selection unit 761 selects attribute-related object image information 59TA that contains attribute-related object images 56TA related to the attribute from the object image information 59T of the object image information group 90T. The first selection unit 761 outputs the set of selected attribute-related object image information 59TA, i.e., the attribute-related object image information group 90TA, to the first export unit 771. The attribute-related object images 56TA are, for example, object images 56T reflecting a present home, object images 56T reflecting a present pet, and object images 56T reflecting a present state of enjoying a hobby, etc. The first selection unit 761 searches for attribute-related object images 56TA by referring to tags, etc., of the object-related information 57T.

[0075] The second selection unit 762, similarly to the first selection unit 761, selects attribute-related reference image information 59RA that includes attribute-related reference images 56RA from the reference image information 59R of the reference image information group 90R. The second selection unit 762 outputs the set of selected attribute-related reference image information 59RA, i.e., the attribute-related reference image information group 90RA, to the second export unit 772.

[0076] The setting range 85 of the determination assistance information 38 is input into the first export unit 771. The first export unit 771 derives the personality index value 91 from the object image index values ​​58T of the multiple attribute-related object image information 59TA constituting the attribute-related object image information group 90TA. More specifically, as an example, such as Figure 14As shown, the first export unit 771 exports the average value of the object image index values ​​58T after removing values ​​outside the set range 85 as the personality index value 91. The average value is an example of the "representative value" involved in the technology of this invention. That is, the personality index value 91 is a representative value of the multiple object image index values ​​58T derived from multiple object images 56T. The first export unit 771 calculates the average value of each of the five factors (openness, conscientiousness, extraversion, agreeableness, and neuroticism) of the object image index value 58T after removing values ​​outside the set range 85. The first export unit 771 outputs the exported personality index value 91 to the comparison unit 78.

[0077] In the second export unit 772, the setting range 85 of the determination assistance information 38 is input in the same way as in the first export unit 771. The second export unit 772 derives statistical index values ​​92 from the reference image index values ​​58R of the multiple attribute-related reference image information 59RA constituting the attribute-related reference image information group 90RA. More specifically, as an example, such as... Figure 15 As shown, the second export unit 772 exports the average value of the reference image index values ​​58R after removing values ​​outside the set range 85 as a statistical index value 92. The average value is an example of a "representative value" according to the technology of this invention. That is, the statistical index value 92 is a representative value of the multiple reference image index values ​​58R derived from multiple reference images 56R. The second export unit 772 calculates the average value after removing values ​​outside the set range 85 for each of the five factors of the reference image index value 58R. The second export unit 772 outputs the exported statistical index value 92 to the comparison unit 78. Furthermore, in Figure 14 and Figure 15 The example shown illustrates cases where the set range 85 is 30% or higher. Therefore, as indicated by the × mark, values ​​less than 30%, such as 18%, 26%, 3%, 11%, 23%, and 25%, are excluded.

[0078] Thus, the personality index value 91 and the statistical index value 92 are derived essentially using the same method. The same method means that even if the values ​​used to remove or calculate the average differ among the five factors, the order in which values ​​outside the set range of 85 are removed or the average is calculated using the removed values ​​remains the same.

[0079] exist Figure 12 In the comparison section, 78 compares the personality index value 91 with the statistical index value 92. Specifically, as an example, ... Figure 16 As shown, the comparison unit 78 calculates the difference Δ between the personality index value 91 and the statistical index value 92 by subtracting the statistical index value 92 from the personality index value 91. The comparison unit 78 calculates the difference Δ for each of the five factors. The comparison unit 78 outputs the calculated difference Δ as a comparison result 93 to the generation unit 79.

[0080] The differential-intensity correspondence information 86 of the decision assistance information 38 is input into the generation unit 79. For example, ... Figure 17 As shown, the intensity corresponding to the difference Δ is registered in the difference-intensity correspondence information 86. Specifically, the intensity is 3 for the case where the difference Δ is greater than 10 (10 < Δ), 2 for the case where the difference Δ is greater than 5 and less than 10 (5 < 10 ≤ Δ), and 1 for the case where the difference Δ is greater than 0 and less than 5 (0 < Δ ≤ 5). Furthermore, the intensity is 0 for the case where the difference Δ is 0. Additionally, the intensity is -3 for the case where the difference Δ is less than -10 (Δ < -10), -2 for the case where the difference Δ is greater than -10 and less than -5 (-10 ≤ Δ -5), and -1 for the case where the difference Δ is greater than -5 and less than 0 (-5 ≤ Δ < 0). As an example, ... Figure 16 As shown, the generation unit 79 generates personality determination information 61 from the comparison result 93 based on the difference-intensity correspondence information 86.

[0081] The generation unit 79 outputs the generated personality determination information 61 to the RW control unit 47. The RW control unit 47 stores the personality determination information 61 in the image information DB36.

[0082] Next, as an example, refer to Figure 18 and Figure 19 The flowchart shown illustrates the function of the above structure. For example... Figure 3 As shown, the CPU 22A of the user terminal 10 functions as the browser control unit 32 upon the activation of the image AP 30. Furthermore, as... Figure 4 As shown, the CPU 22B of the information processing server 12 functions as a request receiving unit 45, an index value exporting unit 46, an RW control unit 47, a personality determination unit 48, and a distribution control unit 49 upon the activation of the working program 35. Figure 11 and Figure 12 As shown, the personality determination unit 48 includes a first acquisition unit 751, a second acquisition unit 752, a first selection unit 761, a second selection unit 762, a first derivation unit 771, a second derivation unit 772, a comparison unit 78, and a generation unit 79.

[0083] User U uses the camera function of user terminal 10 to capture image 56. After capturing image 56, under the control of browser control unit 32, the storage request for image 56 is sent to information processing server 12.

[0084] In the information processing server 12, the request receiving unit 45 receives the storage request for image 56. Figure 18 (In step ST100, it is marked "Yes"). The storage request is output from the request receiving unit 45 to the index value exporting unit 46, etc.

[0085] like Figure 9 As shown, in the index value export unit 46, image 56 is input to index value export model 37, thereby outputting index value 58 from index value export model 37 (step ST110). Index value 58 and image 56 are output together from index value export unit 46 to RW control unit 47. Then, under the control of RW control unit 47, image 56, accompanying information 57 and index value 58 are associated through image ID and stored as image information 59 in image information DB36 of storage device 20B (step ST120).

[0086] In the case where there is a user UT for whom personality assessment information 61 should be generated ( Figure 19 In step ST200, it is marked "Yes", such as Figure 11 As shown, the RW control unit 47 reads the object image information group 90T and object attribute information 60T from the image information DB36 for the object user UT. The object image information group 90T and object attribute information 60T are output from the RW control unit 47 to the first acquisition unit 751 and acquired by the first acquisition unit 751 (step ST210). The object image information group 90T is output from the first acquisition unit 751 to the first selection unit 761. Furthermore, the object attribute information 60T is output from the first acquisition unit 751 to the second acquisition unit 752.

[0087] like Figure 11 and Figure 13 As shown, the RW control unit 47 reads a reference image information group 90R from the image information DB36 that has attributes consistent with or similar to those of the target user UT. The reference image information group 90R is output from the RW control unit 47 to the second acquisition unit 752 and acquired by the second acquisition unit 752 (step ST220). The reference image information group 90R is then output from the second acquisition unit 752 to the second selection unit 762.

[0088] In the first selection unit 761, attribute-related object image information 59TA that includes attribute-related object image 56TA is selected from the object image information 59T of the object image information group 90T (step ST230). The attribute-related object image information group 90TA, which is a set of attribute-related object image information 59TA, is output from the first selection unit 761 to the first export unit 771.

[0089] Furthermore, in the second selection unit 762, attribute-related reference image information 59RA that includes attribute-related reference image 56RA is selected from the reference image information 59R of the reference image information group 90R (step ST240). The attribute-related reference image information group 90RA, which is a set of attribute-related reference image information 59RA, is output from the second selection unit 762 to the second export unit 772.

[0090] like Figure 14 As shown, in the first export unit 771, the average value of the object image index value 58T, after removing values ​​outside the set range 85, from the exported attribute-related object image information 59TA is used as the personality index value 91 (step ST250). The personality index value 91 is output from the first export unit 771 to the comparison unit 78.

[0091] And, as Figure 15 As shown, in the second export unit 772, the average value of the reference image index value 58R of the exported attribute-related reference image information 59RA, after removing values ​​outside the set range 85, is taken as the statistical index value 92 (step ST260). The statistical index value 92 is output from the second export unit 772 to the comparison unit 78.

[0092] like Figure 16 As shown, in the comparison unit 78, the personality index value 91 is compared with the statistical index value 92, and the difference Δ between the personality index value 91 and the statistical index value 92 is calculated (step ST270). The comparison result 93, including the difference Δ, is output from the comparison unit 78 to the generation unit 79.

[0093] like Figure 16 As shown, in the generation unit 79, the differential-intensity correspondence information 86 is referenced, and personality determination information 61 for the target user UT is generated based on the comparison result 93 (step ST280). The personality determination information 61 is output from the generation unit 79 to the RW control unit 47. Under the control of the RW control unit 47, the personality determination information 61 is stored in the image information DB36 of the storage device 20B (step ST290).

[0094] Personality assessment information 61 is distributed to user terminal 10 and viewed by user U according to user U's distribution request. Furthermore, personality assessment information 61 is considered when selecting images 56 as the basis for a photo album recommended to user U. For example, for user U, who is determined to have an introverted and rational personality, indoor images 56, images 56 without people, and less vibrant images 56 are prioritized as the basis for a photo album recommended to user U. Additionally, personality assessment information 61 can also be considered when selecting content from commercial advertisements provided to user U via a web browser through image AP30.

[0095] As explained above, the personality determination unit 48 of the CPU 22B of the information processing server 12 acquires the personality index value 91 derived from the object image 56T related to the target user UT whose personality is being determined, and compares the personality index value 91 with the statistical index values ​​92 of multiple reference users UR. The generation unit 79 generates personality determination information 61 based on the comparison result 93, thereby determining the personality of the target user UT.

[0096] If the personality of a user UT is determined solely based on the personality index value 91 without comparing it with the statistical index value 92, the personality assessment information 61 will significantly influence the subject reflected in the image 56T. For example, a user U with young children in the family is likely to take many images 56 of children. Such a user U will have higher agreeableness simply because of the large number of images 56 of children.

[0097] In contrast, if personality index value 91 is compared with statistical index value 92, and personality judgment information 61 is generated based on the comparison result 93, then the personality of the target user UT can be determined by comparing it with the statistical personality of multiple reference users UR who serve as benchmarks. Therefore, the personality of user U can be determined with high accuracy based on image 56.

[0098] like Figure 14 As shown, personality index value 91 is a representative value of multiple object image index values ​​58T derived from multiple object images 56T related to the object user UT. Furthermore, as... Figure 15 As shown, statistical index value 92 is a representative value of multiple reference image index values ​​58R derived from multiple reference images 56R related to the reference user UR. Therefore, a simple comparison between personality index value 91 and statistical index value 92 can be made.

[0099] like Figure 14 As shown, the personality index value of 91 is the average of the object image index value 58T after removing values ​​outside the set range of 85. Furthermore, as... Figure 15 As shown, statistical index value 92 is the average of reference image index values ​​58R after removing values ​​outside the set range 85. If the average is calculated without removing values ​​outside the set range 85, the characteristics required for personality determination will be averaged out. Therefore, by calculating the average after removing values ​​outside the set range 85, a more appropriate personality index value 91 and statistical index value 92 representing the characteristics required for personality determination can be derived.

[0100] like Figure 9 As shown, index value 58, namely object image index value 58T and reference image index value 58R, is derived using index value derivation model 37. Therefore, object image index value 58T and reference image index value 58R can be derived simply and with high accuracy.

[0101] The comparison unit 78 compares the personality index value 91 with the statistical index value 92 of the reference user UR, whose specific attributes are consistent with or similar to those of the target user UT. Therefore, compared with the case involving the reference user UR, which has no relation to the target user UT and whose attributes are different from those of the target user UT, the accuracy of personality determination can be further improved.

[0102] The comparison unit 78 acquires the personality index value 91 derived from the attribute-related object image 56TA in the object image 56T. Therefore, it is possible to determine the personality of the object user UT corresponding to the attribute.

[0103] There are multiple possible values ​​for the personality index 91 and the statistical index 92. Therefore, it is possible to determine user U's personality from multiple perspectives.

[0104] [Second Implementation]

[0105] Additionally, personality index value 91 derived from attribute-related object image 56TA was used, but it is not limited to this. As an example, such as... Figure 20 As shown in the personality determination unit 100, the first selection unit 761 and the second selection unit 762 can be omitted. In this case, the first acquisition unit 751 outputs object image information group 90T to the first export unit 771, and the second acquisition unit 752 outputs reference image information group 90R to the second export unit 772. Then, the first export unit 771 exports personality index value 91 from the object image index value 58T of all object images 56T and outputs the exported personality index value 91 to the comparison unit 78. Furthermore, the second export unit 772 exports statistical index value 92 from the reference image index value 58R of all reference images 56R and outputs the exported statistical index value 92 to the comparison unit 78.

[0106] Thus, in the second embodiment, the comparison unit 78 acquires personality index values ​​91 derived from all object images 56T. Therefore, it is possible to determine the personality of the object user UT, which is less restricted by attributes.

[0107] The first and second embodiments described above can be implemented in combination. In this case, the generation unit 79 generates personality determination information 61 based on personality index values ​​91 derived from attribute-related object images 56TA and personality determination information 61 based on personality index values ​​91 derived from all object images 56T. When selecting an image 56 as the basis for recommending a photo album to user U, the image 56 is selected in a balanced manner by referring to both the personality determination information 61 based on personality index values ​​91 derived from attribute-related object images 56TA and the personality determination information 61 based on personality index values ​​91 derived from all object images 56T. For example, overall, the image 56 is selected by referring to the personality determination information 61 based on personality index values ​​91 derived from all object images 56T. Then, for an image 56 configured on a specific page, the personality determination information 61 based on personality index values ​​91 derived from attribute-related object images 56TA is used for selection.

[0108] The strength of personality is not limited to the seven levels shown. There can be fewer than seven levels or more than seven levels. Different setting ranges of 85 can be prepared for each of the five factors. The representative value is not limited to the average shown, but can also be the most frequent value, median, etc.

[0109] The difference Δ is shown as a comparison result 93 between personality index value 91 and statistical index value 92, but it is not limited to this. The ratio obtained by dividing personality index value 91 by statistical index value 92 can also be used as a comparison result 93.

[0110] The personality assessment unit 48 can be activated based on the request of user U, and the personality of the user U who made the request can be assessed as the target user UT. Furthermore, the statistical index values ​​92 can be pre-exported and stored in the storage device 20B for each attribute.

[0111] The indicator values ​​are not limited to the five factors listed above: openness, conscientiousness, extraversion, agreeableness, and neuroticism.

[0112] The hardware structure of the computer constituting the information processing server 12 can be modified in various ways. For example, to improve processing power and reliability, the information processing server 12 can be composed of multiple computers that are separate hardware components. For example, the functions of the request receiving unit 45 and the index value exporting unit 46, as well as the functions of the RW control unit 47, the character determination unit 48 or 100, and the distribution control unit 49, can be distributed among two computers. In this case, the information processing server 12 is composed of two computers. Furthermore, the user terminal 10 can also perform all or part of the functions of the information processing server 12.

[0113] Thus, the hardware structure of the user terminal 10 and the information processing server 12 can be appropriately modified according to the required performance such as processing power, security, and reliability. Moreover, not limited to hardware, the image AP 30 and the operating program 35, etc., can of course be dual-distributed or stored in multiple storage devices for the purpose of ensuring security and reliability.

[0114] In the above embodiments, for example, as the hardware structure of the processing units that perform various processes, such as the browser control unit 32, request receiving unit 45, index value exporting unit 46, RW control unit 47, personality determination unit 48 and 100, distribution control unit 49, first acquisition unit 751, second acquisition unit 752, first selection unit 761, second selection unit 762, first exporting unit 771, second exporting unit 772, comparison unit 78, and generation unit 79, various processors as shown below can be used. These processors include general-purpose processors such as CPUs 22A and 22B that execute software (image AP30 and working program 35) and function as various processing units, as well as processors such as FPGAs (Field Programmable Gate Arrays) whose circuit structure can be changed after manufacturing (i.e., Programmable Logic Devices (PLDs)) and / or ASICs (Application Specific Integrated Circuits) which have circuit structures specifically designed for performing specific processes (i.e., dedicated circuits).

[0115] A processing unit can consist of one of these various processors, or it can consist of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Furthermore, multiple processing units can also be composed of a single processor.

[0116] As examples of multiple processing units comprised of a single processor, firstly, in computers such as client and server computers, a single processor is composed of a combination of one or more CPUs and software, and this processor functions as multiple processing units. Secondly, in systems-on-a-chip (SoC), a processor that implements the overall functionality of a system containing multiple processing units is used through a single integrated circuit (IC) chip. Thus, various processing units are constructed using one or more of the aforementioned processors as their hardware structure.

[0117] Furthermore, as the hardware structure of these various processors, more specifically, circuits composed of circuit elements such as semiconductor elements can be used.

[0118] Based on the above records, one can master the techniques described in the following notes.

[0119] [Note 1]

[0120] An information processing device comprising a processor, The processor performs the following processing: Obtain personality index values ​​derived from object images related to the target user whose personality is being determined; Compare the personality index values ​​with the statistical index values ​​of multiple reference users; The personality of the target user is determined based on the comparison results.

[0121] [Note 2]

[0122] According to the information processing apparatus described in Appendix 1, wherein, The personality index value is a representative value of the index value of each of the multiple object images derived from the multiple object images. The statistical index value is a representative value of the index value of each of the multiple reference images derived from multiple reference images related to the reference user.

[0123] [Note 3]

[0124] According to the information processing apparatus described in Appendix 2, wherein, There are multiple types of personality index values ​​and statistical index values. The personality index value is the average of the index values ​​for each object image after removing values ​​outside the set range. The statistical index value is the average of the index values ​​of each reference image after removing values ​​outside the set range.

[0125] [Note 4]

[0126] According to the information processing apparatus described in appendix 2 or 3, wherein, The indicator values ​​are derived using the learned model.

[0127] [Note 5]

[0128] The information processing apparatus according to any one of appendices 1 to 4, wherein, The processor performs the following processing: The personality index value is compared with the statistical index value of the reference user, and the reference user has the same or similar specific attributes as the target user.

[0129] [Note 6]

[0130] According to the information processing apparatus described in Appendix 5, wherein... The processor performs the following processing: Obtain the personality index value derived from the attribute-related object image associated with the attribute in the object image.

[0131] [Note 7]

[0132] According to the information processing apparatus described in appendix 5 or 6, wherein, The processor performs the following processing: Obtain the personality index values ​​derived from all the object images.

[0133] [Note 8]

[0134] The information processing apparatus according to any one of appendices 1 to 7, wherein, There are multiple values ​​for the personality index and the statistical index.

[0135] The technology of the present invention can also be appropriately combined with the various embodiments and / or variations described above. Furthermore, it is not limited to the embodiments described above; various structures can be adopted as long as they do not depart from the spirit of the invention. Moreover, in addition to programs, the technology of the present invention also relates to storage media for non-transitory program storage and computer program products containing programs.

[0136] The above descriptions and illustrations are detailed explanations of the parts involved in the technology of this invention, and are merely one example of the technology of this invention. For example, the descriptions related to the above structure, function, effect, and effect are only one example of the structure, function, effect, and effect of the parts involved in the technology of this invention. Therefore, without departing from the spirit of the technology of this invention, unnecessary parts can be deleted, new elements can be added, or replacements can be made to the above descriptions and illustrations. Furthermore, to avoid cumbersome descriptions and to facilitate understanding of the parts involved in the technology of this invention, relevant explanations such as common technical knowledge that do not require special explanation in enabling the implementation of the technology of this invention have been omitted from the above descriptions and illustrations.

[0137] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, when "and / or" is used in this specification to represent three or more items together, the same concept as "A and / or B" can also be applied.

[0138] All documents, patent applications and technical standards described in this specification are incorporated herein by reference to the same extent as the specific documents, patent applications and technical standards described therein.

Claims

1. An information processing device comprising a processor, The processor performs the following processing: Obtain personality index values ​​derived from object images associated with object users, where the object users are users whose personalities are being determined; The personality index value is compared with the statistical index values ​​of multiple reference users; and The personality of the target user is determined based on the comparison results.

2. The information processing apparatus according to claim 1, wherein, The personality index value is a representative value of the index value of each of the plurality of object images derived from the plurality of object images. The statistical index value is a representative value of the index value of each of the multiple reference images derived from multiple reference images related to the reference user.

3. The information processing apparatus according to claim 2, wherein, There are multiple types of personality index values ​​and statistical index values. The personality index value is the average of the index values ​​for each object image after removing values ​​outside the set range. The statistical index value is the average of the index values ​​of each reference image after removing values ​​outside the set range.

4. The information processing apparatus according to claim 2, wherein, The metric values ​​are derived using the learned model.

5. The information processing apparatus according to claim 1, wherein, The processor performs the following processing: The personality index value is compared with the statistical index value of the reference user, and the specific attributes of the reference user are consistent with or similar to those of the target user.

6. The information processing apparatus according to claim 5, wherein, The processor performs the following processing: Obtain the personality index value derived from the attribute-related object image associated with the attribute in the object image.

7. The information processing apparatus according to claim 5, wherein, The processor performs the following processing: Obtain the personality index values ​​derived from all the object images.

8. The information processing apparatus according to claim 1, wherein, There are multiple values ​​for the personality index and the statistical index.

9. A method of operating an information processing device, comprising the following steps: Obtain personality index values ​​derived from object images associated with object users, where the object users are users whose personalities are being determined; The personality index value is compared with the statistical index values ​​of multiple reference users; and The personality of the target user is determined based on the comparison results.

10. An operating procedure for an information processing apparatus, which causes a computer to perform a process comprising the following steps: Obtain personality index values ​​derived from object images associated with object users, where the object users are the users whose personalities are being assessed. The personality index value is compared with the statistical index values ​​of multiple reference users; and The personality of the target user is determined based on the comparison results.

Images