Character relationship determination method and electronic equipment

Abstract

The invention discloses a character relationship determination method and electronic equipment. The method can be applied to the field of terminals. The method comprises the steps that a first image is acquired, the first image comprises a first character, and the first character is a character with the character relation to be determined; a second image is obtained, the second image comprises a second figure, and the second figure is an owner of the electronic equipment. According to the image in the electronic equipment, a third person having a first association relationship with the first person is determined, and the third person and the second person have a second association relationship. And determining a character relationship between the first character and the second character according to the first association relationship and the second association relationship. The character relationship between the first character and the second character is mined through the third character which has the association relationship with the first character and the second character at the same time, and the character relationship of the second character can be further expanded.

Description

Technical Field

[0001] This application relates to the field of terminals, and in particular, to a method for determining a person relationship and an electronic device. Background Art

[0002] Electronic devices have gradually become an essential product in people's daily lives. Taking mobile phones as an example, people establish social connections through mobile phones. For example, user A can share a certain content (e.g., through an instant messaging application) from mobile phone A to mobile phone B of user B. In this process, user A needs to find the contact information of user B among numerous contact methods in the instant messaging application to share with user B, and the operation is not convenient enough. Summary of the Invention

[0003] This application provides a method for determining a person relationship and an electronic device, which are used to mine and apply person relationships.

[0004] In a first aspect, this application provides a method for determining a person relationship, which is applied to an electronic device. The method includes: obtaining a first image, where the first image includes a first person, and the first person is the person whose person relationship is to be determined; obtaining a second image, where the second image includes a second person, and the second person is the owner of the electronic device; determining a third person who has a first association relationship with the first person according to the images in the electronic device, and the third person has a second association relationship with the second person; and determining the person relationship between the first person and the second person according to the first association relationship and the second association relationship.

[0005] In the embodiments of this application, when a mobile phone determines the person relationship between the first person appearing in the first image and the owner of the mobile phone, it can determine a third person who has a first association relationship with the first person through the images in the mobile phone, and the third person has a second association relationship with the second person. Then, the person relationship between the first person and the second person can be determined through the first association relationship and the second association relationship. Therefore, by mining the person relationship between the first person and the second person through the third person who has an association relationship with both the first person and the second person, the person relationship of the second person can be further expanded.

[0006] In some embodiments, the electronic device can determine the person relationship according to the images in the gallery.

[0007] For example, the electronic device obtains a third image and a fourth image from the gallery. The third image includes the second person and the third person, and the fourth image includes the third person and the first person. Determine the first association relationship between the second person and the third person according to the distance and / or posture between the second person and the third person in the third image. Determine the second association relationship between the first person and the third person according to the distance and / or posture between the third person and the first person in the fourth image.

[0008] In the embodiments of the present application, if there are group photos of a second person and a third person (i.e., the third image) and a group photo of the third person and the first person (i.e., the fourth image) in the image library at the same time, the third image and the fourth image can be directly obtained, and the association relationship between the persons can be determined based on the distance and / or posture between the persons in the images. The third person can be determined according to the first association relationship and the second association relationship.

[0009] In some other embodiments, the electronic device can also determine the relationship between persons according to the images in the image library and the auxiliary information.

[0010] In the first case, the auxiliary information may include information such as the shooting time and shooting location of the image.

[0011] For example, if person B is included in the first image, the electronic device can obtain the shooting time and / or shooting location of the first image, and determine the relationship between person B and person A (the device owner) according to the shooting time and / or shooting location. For example, if the shooting time is October 2014 and the shooting location is XXX University, then the mobile phone 100 can determine that person B is a classmate of person A.

[0012] In the second case, the auxiliary information may further include information in other applications in the electronic device. The other applications may include an address book, an email application, a calendar application, a memo application, and the like.

[0013] For example, the electronic device obtains the third image in the image library. The third image includes the first person B and the third person C. According to the distance and / or posture between the first person and the third person in the third image, the first association relationship between the first person and the third person in the third image is determined. The electronic device determines that there is relevant information about the third person C in the first application in the electronic device, and determines the second association relationship between the third person C and the second person A according to the description information of the third person C in the first application.

[0014] In the embodiments of the present application, when there is only a group photo of the first person and the third person, i.e., the third image, in the image library, the first association relationship between the first person and the third person can be determined according to the third image, and then a search is performed in the electronic device to find the first application that includes relevant information about the third person C. The relevant information may be, for example, the avatar and / or name of the third person. Assume that the electronic device finds relevant information about the third person in the first application. Then, according to the description information of the third person in the first application, the second association relationship between the third person and the second person is determined.

[0015] Taking the first application as the address book as an example, if the electronic device queries that there is an avatar and / or name of the third person in the address book, then the second association relationship between the third person and the second person is determined according to the description information of the third person in the address book (such as the note information, group information, etc.).

[0016] Taking the first application being an email application as an example, if the electronic device finds the head portrait and / or name of a third person in the email application, it determines the second association relationship between the third person and the second person according to the description information of the third person in the email application (such as, remarks information, grouping information, email address information, etc.).

[0017] Taking the first application being a calendar application as an example, if the electronic device finds the name of a third person in the calendar application, it determines the second association relationship between the third person and the second person according to the description information of the third person in the calendar application (such as, keyword extraction from schedule information).

[0018] It should be noted that this article takes the first application being an address book, an email application, and a calendar application as examples for illustration. In fact, the first application can also be other applications, such as a memo application, and the embodiments of this application will not list them one by one.

[0019] In a possible design, determining a third person having a first association relationship with the first person includes: identifying the scene of the first image, where the scene includes one of a home scene, a party scene, and an office scene. According to the scene recognition result, determining that there is description information of a third person C in the first application in the electronic device, and the description information matches the scene recognition result; according to the description information, determining the first association relationship between the first person and the third person, and the second association relationship between the second person and the third person.

[0020] In the embodiments of this application, when there is no group photo of the first person and other people in the photo gallery, the first image can be scene-recognized to obtain a scene recognition result, and a search can be performed in the electronic device according to the scene recognition result to find the first application with relevant information of the third person C, where the relevant information is, for example, the head portrait and / or name of the third person. Assuming that the electronic device finds relevant information including the third person in the first application, the second association relationship between the third person and the second person is determined according to the description information of the third person in the first application.

[0021] Assuming that the scene recognition result is an office scene, the electronic device can search in the email application (i.e., the first application) to see if there is relevant information of the third person, such as a head portrait and / or name. If so, the second association relationship between the third person and the second person is determined according to the description information of the third person in the email application (such as, remarks information, grouping information, email address information, etc.).

[0022] Assume that the scene recognition result is a home scene. Then, the electronic device can search in the address book (i.e., the first application) to check if there is any relevant information about the third person, such as an avatar and / or a name. If so, based on the description information of the third person in the address book (such as note information, grouping information, etc.), determine the second association relationship between the third person and the second person.

[0023] In a possible design, the relationship between the first person B and the second person A is the first type of relationship. The electronic device can also determine the intimacy between each person in the first type of relationship and the device owner. For example, the first type of relationship is a family relationship, the device owner is A, and the first type of relationship includes B (spouse), C (child), and D (parent). The electronic device can determine the intimacy 1 between B and A, the intimacy 2 between C and A, and the intimacy 3 between D and A. Optionally, the more interaction times, the higher the intimacy. When the electronic device determines that it needs to share the first content with the people in the first type of relationship, it can sort according to the intimacy. For example, B has the highest intimacy, so B's contact information is ranked first, and D has the lowest intimacy, so D's contact information is ranked at the end. Or, when the electronic device sets an emergency contact as a person in the first type of relationship (family relationship), when there is a need for an emergency contact, it first contacts the person with the highest intimacy.

[0024] In a possible design, the electronic device can implement the method for determining the relationship between people in the embodiments of the present application through an algorithm model. For example, after the algorithm model determines the relationship between the first person B and the second person A (such as a family relationship), it can also output a probability value. The probability value is used to indicate the probability that the first person and the second person are in a family relationship. Optionally, if the probability is low (e.g., less than a threshold), a prompt message is output to prompt the user to determine whether the relationship between the first person and the second person (such as a family relationship) is correct. Of course, if the probability is high (e.g., greater than or equal to the threshold), no prompt needs to be output.

[0025] In a possible design, recognizing the scene of the first image includes: recognizing the scene of the first image through at least one of the items held by the first person in the first image, the attachments, the clothing, and the environment where the first person is located.

[0026] In a possible design, the relationship between the first person and the second person is the first type of relationship. The method further includes: receiving a sharing operation for the first content. Based on the first content, determine to share the first content with the people in the first type of relationship. Display the sharing method for the first person, where the sharing method is used to share the first content with the first person.

[0027] In the embodiments of the present application, the mobile phone can apply the first type of relationship to the scene of content sharing and specify to share the first content with the people in the first person.

[0028] In a possible design, the first content includes content related to the fourth person in the first person relationship and does not include content related to the first person B.

[0029] In the embodiments of the present application, even if the first content does not include content related to the first person, since the first person is in the first person relationship, the first content can still be shared with the first person.

[0030] In a possible design, the person relationship between the first person B and the second person A is the first person relationship, and the method further includes: setting the Do Not Disturb mode according to a user operation, and not responding to new messages of persons outside the first person relationship in the Do Not Disturb mode.

[0031] In the embodiments of the present application, the first person relationship can be applied to the scenario of Do Not Disturb mode setting, and the mobile phone only rings, vibrates or pops up a reminder for messages or calls of persons in the first person relationship.

[0032] In a possible design, the person relationship between the first person B and the second person A is the first person relationship, and the method further includes: setting the emergency contacts in the first application as the persons in the first person relationship according to a user operation. In response to an emergency contact operation, contact the first person B.

[0033] In the embodiments of the present application, the first person relationship can be applied to the scenario of emergency contact setting to automatically complete the information of emergency contacts.

[0034] In a possible design, obtaining the second image includes at least one of the following: obtaining the profile picture of the owner of the electronic device; or obtaining a self-taken image in the gallery application; or obtaining the person image with the highest appearance frequency in the gallery application.

[0035] In the embodiments of the present application, there are various ways for the electronic device to obtain the second image including the profile picture of the mobile phone owner. Specifically, one or more of these ways can be flexibly selected according to the actual situation. For example, if there is a profile picture of the owner in the electronic device, the profile picture can be directly obtained as the second image. When there is no profile picture of the owner in the electronic device, a self-taken image in the gallery can be obtained as the second image. Also for example, a self-taken image in the gallery can be directly obtained, and so on. Optionally, the profile picture of the owner can be the profile picture in the "My Business Card" in the electronic device. Among them, the "My Business Card" can be located in a certain application in the electronic device, which can be the contact application, or the settings application, or other applications. The embodiments of the present application do not limit its specific location.

[0036] In a second aspect, an electronic device is further provided, including:

[0037] A processor, a memory, and one or more programs;

[0038] Wherein, the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the processor, cause the electronic device to execute the method steps provided in the first aspect as described above.

[0039] In a third aspect, a computer-readable storage medium is provided, and the computer-readable storage medium is used to store a computer program. When the computer program runs on a computer, the computer is caused to execute the method described in the first aspect as above.

[0040] In a fourth aspect, a computer program product is provided, including a computer program that, when run on a computer, causes the computer to execute the method described in the first aspect as above.

[0041] In a fifth aspect, a chip is further provided. The chip is coupled to a memory in an electronic device and is used to call a computer program stored in the memory and execute the technical solution of the first aspect of the embodiments of the present application. In the embodiments of the present application, "coupled" means that two components are directly or indirectly combined with each other.

[0042] For the technical effects that can be achieved in the second to fifth aspects above, please refer to the description of the technical effects that can be achieved in the corresponding design solutions in the first aspect above. The present application will not repeat them here. Description of the Drawings

[0043] Figure 1A It is a schematic diagram of the hardware structure of the mobile phone 100 provided by an embodiment of the present application;

[0044] Figure 1B It is a schematic diagram of the software structure of the mobile phone 100 provided by an embodiment of the present application;

[0045] Figure 2A It is a schematic diagram of identifying a person in an image by the image segmentation technology provided by an embodiment of the present application;

[0046] Figure 2B It is a schematic diagram of identifying an animal in an image by the image segmentation technology provided by an embodiment of the present application;

[0047] Figure 3 It is a schematic diagram of a currently existing method for determining a person's relationship provided by an embodiment of the present application;

[0048] Figure 4A It is a schematic diagram of an image in the picture library provided by an embodiment of the present application;

[0049] Figure 4BSchematic diagram for identifying the relationship between characters in Image 1 provided by an embodiment of the present application;

[0050] Figure 5 Schematic flowchart of a method for determining the relationship between characters provided by an embodiment of the present application;

[0051] Figure 6 Schematic diagram of the profile picture of the device owner provided by an embodiment of the present application;

[0052] Figure 7 Schematic diagram of a multi-level relationship between characters provided by an embodiment of the present application;

[0053] Figure 8A Schematic diagram of the content sharing interface provided by an embodiment of the present application;

[0054] Figure 8B Schematic diagram of the family sharing settings interface provided by an embodiment of the present application;

[0055] Figure 9A Schematic diagram of the medical emergency card interface provided by an embodiment of the present application;

[0056] Figure 9B Schematic diagram of the settings interface for the family account provided by an embodiment of the present application;

[0057] Figure 9C Schematic diagram of the automatic settings interface for associated persons provided by an embodiment of the present application;

[0058] Figure 10 Schematic diagram of the settings interface for the do not disturb mode provided by an embodiment of the present application;

[0059] Figure 11 Schematic diagram of the interface for identifying the relationship between characters provided by an embodiment of the present application;

[0060] Figure 12 Schematic diagram of a structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0061] Hereinafter, some terms in the embodiments of the present application will be explained to facilitate the understanding of those skilled in the art.

[0062] At least one involved in the embodiments of the present application includes one or more; among them, multiple means greater than or equal to two. In addition, it should be understood that in the description of this specification, terms such as "first" and "second" are only used for the purpose of distinguishing descriptions, and cannot be understood as explicitly or implicitly indicating relative importance, nor can they be understood as explicitly or implicitly indicating order. For example, the first image and the second image do not represent the importance degree of the two or the order of the two, but are only for distinguishing descriptions. In the embodiments of the present application, "and / or" only describes the association relationship and means that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0063] References in this specification to "one embodiment" or "some embodiments" or the like mean that one or more embodiments of this specification include the specific features, structures, or characteristics described in connection with that embodiment. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" and the like that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0064] As used in this specification, depending on the context, the term "when" or "after" can be interpreted to mean "if" or "after" or "in response to determining" or "in response to detecting". Similarly, depending on the context, the phrase "when determining" or "if detecting (the stated condition or event)" can be interpreted to mean "if determining" or "in response to determining" or "when detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)".

[0065] The method for determining the relationship between characters provided by the embodiments of the present application is applicable to electronic devices. For example, the electronic device can be a portable electronic device such as a mobile phone, a tablet computer, a laptop computer, etc., or a wearable device such as a watch or a bracelet, or a virtual reality (VR) device, an augmented reality (AR) device, a mixed reality (MR) device, etc. In short, the embodiments of the present application do not limit the specific type of the electronic device. For the convenience of description, this article mainly takes the electronic device as a mobile phone as an example.

[0066] Figure 1AA schematic diagram showing the hardware structure of an electronic device is presented. The electronic device can be, for example, a mobile phone 100. As Figure 1A shown, the electronic device may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0067] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors. Among them, the controller may be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions. A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can save the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0068] Among them, the processor 110 may execute the process of the method for determining the relationship between characters provided in the embodiments of the present application. For example, the processor 110 may obtain a first image, which includes a first person, and the first person is the person whose relationship needs to be determined. The processor 110 may also obtain a second image, which includes a second person, and the second person is the owner of the electronic device. After the processor 110 obtains the second image, it may further determine a third person associated with the first person based on the images in the electronic device, and determine the relationship between the first person and the second person through the third person. The following introduction will elaborate on this process in detail.

[0069] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0070] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be respectively coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface to implement the touch function of the electronic device 100.

[0071] The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple groups of I2S buses. The processor 110 can be coupled to the audio module 170 through the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface to implement the function of answering a call through a Bluetooth headset.

[0072] The PCM interface can also be used for audio communication to sample, quantize, and encode analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through the PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface to implement the function of answering a call through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0073] The UART interface is a general-purpose serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example: The processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.

[0074] The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate through the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the electronic device 100.

[0075] The GPIO interface can be configured through software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, the display screen 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0076] The USB interface 130 is an interface compliant with the USB standard specification, which can specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used for data transmission between the electronic device 100 and peripheral devices. It can also be used to connect headphones to play audio through the headphones. This interface can also be used to connect other electronic devices, such as AR devices, etc.

[0077] It can be understood that the interface connection relationships illustrated among the various modules in the embodiments of this application are only illustrative descriptions and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 can also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0078] The wireless communication function of the electronic device can be implemented through antenna 1, antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the electronic device can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example: Antenna 1 can be multiplexed as the diversity antenna of the wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0079] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc. applied to the electronic device. The mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves through antenna 1, filter, amplify, etc. the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 can be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 can be provided in the same device.

[0080] The wireless communication module 160 may provide solutions for wireless communications applied to an electronic device, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), and so on. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, frequency-modulate them, amplify them, and convert them into electromagnetic waves via the antenna 2 for radiation.

[0081] In some embodiments, the antenna 1 of the electronic device is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, enabling the electronic device to communicate with a network and other devices through wireless communication technologies.

[0082] The display screen 194 is used to display the display interface of an application, etc. The display screen 194 includes a display panel. In some embodiments, the electronic device may include one or more display screens 194.

[0083] The electronic device 100 may implement a shooting function through an ISP, the camera 193, a video codec, a GPU, the display screen 194, and an application processor, etc. Among them, the ISP is used to process the data fed back by the camera 193.

[0084] The internal memory 121 may be used to store computer-executable program codes, and the executable program codes include instructions. The processor 110 executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory 121. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and software codes of at least one application program, etc. The data storage area may store data generated during the use of the electronic device (such as images, videos, etc.). In addition, the internal memory 121 may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash memory, etc.

[0085] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor 110 through the external memory interface 120 to implement the data storage function. For example, files such as pictures and videos are saved in the external memory card.

[0086] The electronic device can implement audio functions through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0087] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0088] The speaker 170A, also known as the "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to hands-free calls and other external speaker scenarios through one or more speakers 170A.

[0089] The receiver 170B, also known as the "earpiece", can be one or more, and is used to convert an audio electrical signal into a sound signal. When the electronic device 100 answers a call or a voice message, the voice can be listened to by bringing the receiver 170B close to the human ear.

[0090] The microphone 170C, also known as the "microphone" or "transmitter", is used to convert a sound signal into an electrical signal.

[0091] The headphone jack 170D is used to connect a wired headphone.

[0092] The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194.

[0093] The gyroscope sensor 180B can be used to determine the motion posture of the electronic device. In some embodiments, the angular velocity of the electronic device around three axes (i.e., the x, y, and z axes) can be determined through the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake during shooting.

[0094] The barometric pressure sensor 180C is used to measure barometric pressure. In some embodiments, the electronic device calculates the altitude based on the barometric pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

[0095] The magnetic sensor 180D includes a Hall sensor. For example, the electronic device can use the magnetic sensor 180D to detect the opening and closing of the flip case.

[0096] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device in various directions (generally three axes). When the electronic device is stationary, the magnitude and direction of gravity can be detected.

[0097] The distance sensor 180F is used to measure distance. The electronic device can measure distance through infrared or laser.

[0098] The proximity light sensor 180G can include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The light-emitting diode can be an infrared light-emitting diode. The electronic device emits infrared light outward through the light-emitting diode. The electronic device uses the photodiode to detect the infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device. When insufficient reflected light is detected, the electronic device can determine that there is no object near the electronic device.

[0099] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device can adaptively adjust the brightness of the display screen 194 according to the sensed ambient light brightness.

[0100] The fingerprint sensor 180H is used to collect fingerprints.

[0101] The temperature sensor 180J is used to detect temperature.

[0102] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as the "touch screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operations to the application processor to determine the type of touch event.

[0103] The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signals of the vibrating bone mass of the human vocal part.

[0104] The button 190 includes a power-on button, volume buttons, etc. The button 190 can be a mechanical button or a touch button. The electronic device can receive button inputs and generate key signal inputs related to the user settings and function controls of the electronic device. The motor 191 can generate vibration prompts. The motor 191 can be used for incoming call vibration prompts or for touch vibration feedback. The indicator 192 can be an indicator light, which can be used to indicate the charging status, power change, or can also be used to indicate messages, missed calls, notifications, etc. The SIM card interface 195 is used to connect the SIM card. The SIM card can be in contact with and separated from the electronic device by being inserted into or removed from the SIM card interface 195.

[0105] It can be understood that Figure 1A the components shown do not specifically limit the electronic device. The electronic device in the embodiments of the present application may include more or fewer components than Figure 1A those shown. In addition, Figure 1A the combination / connection relationship between the components shown can also be adjusted and modified.

[0106] Figure 1B This is a schematic diagram of the software structure of the electronic device provided by the embodiments of the present application. The electronic device can be, for example, a mobile phone. As Figure 1B shown, the software structure of the electronic device can be a layered structure. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers is through software interfaces. The layered architecture can be, for example, an event-driven architecture, a microkernel architecture, a microservices architecture, or a cloud architecture. The embodiments of the present application take the system of the layered architecture as an example to exemplarily illustrate the software structure of the electronic device. In some embodiments, the system is divided into five layers, from top to bottom are the application layer, application framework layer, Android runtime and system libraries, kernel layer, and hardware layer.

[0107] The application layer may include a series of application packages. As Figure 1B shown, the application packages may include applications such as a camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message, etc.

[0108] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer includes some predefined functions. As Figure 1BAs shown in the figure, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, etc. The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data, and make this data accessible to application programs. The data may include videos, images, audio, dialed and received calls, browsing history and bookmarks, phone books, etc. The view system includes visible controls, such as controls for displaying text, controls for displaying pictures, etc. The view system can be used to build application programs. The display interface can be composed of one or more views. For example, a display interface including a text message notification icon may include a view for displaying text and a view for displaying pictures. The phone manager is used to provide the communication function of the electronic device. For example, the management of call status (including connection, disconnection, etc.). The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, and so on. The notification manager enables application programs to display notification information in the status bar, can be used to convey notification-type messages, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as a notification of a background-running application program, and can also be a notification that appears on the screen in the form of a dialog window. For example, prompt text information in the status bar, emit a prompt sound, the electronic device vibrates, the indicator light flashes, etc.

[0109] The Android Runtime includes core libraries and a virtual machine. The Android runtime is responsible for the scheduling and management of the Android system. The core libraries consist of two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core libraries of Android. The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection. The system libraries can include multiple functional modules. For example: surface manager, Media Libraries, 3D graphics processing libraries (such as: OpenGL ES), 2D graphics engine (such as: SGL), etc. The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications. The media library supports the playback and recording of various common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc. The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc. The 2D graphics engine is the drawing engine for 2D drawing.

[0110] The kernel layer is the layer between the hardware and the software. The kernel layer includes at least a display driver, a camera driver, an audio driver, and a sensor driver.

[0111] The hardware layer includes various hardware, such as, a wifi transceiver, a camera, a display screen, etc.

[0112] It can be understood that Figure 1B the software structure shown does not constitute a specific limitation on the software structure of the electronic device. The software structure of the electronic device in the embodiments of the present application may include more or fewer modules than Figure 1B shown, for example, more or fewer layers than Figure 1B shown, or a certain layer may include more or fewer modules than Figure 1B shown, etc. In addition, Figure 1B the combination / connection relationship between the modules in

[0113] Next, taking a mobile phone as an example, the method for determining the relationship between characters provided in the embodiments of the present application will be introduced in combination with the accompanying drawings.

[0114] Currently, the image segmentation technology is becoming increasingly mature. For example, as Figure 2A shown, when there are images of multiple people in an image, the people in the image can be recognized through the image segmentation technology. Or, as Figure 2B, when there are multiple images of animals in an image, a cat can be recognized through image segmentation technology. However, if we want to further reflect the relationships among the people in the image, it cannot be achieved by only recognizing individual objects.

[0115] One possible implementation is to obtain the relationships among people based on graph learning. For example Figure 3 , obtain multiple subgraphs of relationships among people, and each subgraph of relationships among people includes target nodes composed of the unique account identifiers of users. For the first subgraph of relationships among people in the multiple subgraphs of relationships among people, in response to determining that there is an association relationship between at least one node in the first subgraph of relationships among people and at least one node in the second subgraph of relationships among people, obtain a target graph of relationships among people composed of the first subgraph of relationships among people and the second subgraph of relationships among people. Obtain the similarity between the target nodes of the first subgraph of relationships among people and the target nodes of the second subgraph of relationships among people in the target graph of relationships among people, and determine the relationship between the user corresponding to the first subgraph of relationships among people and the user corresponding to the second subgraph of relationships among people based on the similarity.

[0116] Another possible implementation is to determine the relationships among people and perform contact recommendations by creating a pattern model. The pattern model can operate on the interaction data stored in the database to generate rules based on the repeated patterns of the identified features. The rules generated by the model can be applied to the current context of the mobile device to determine a set of rules related to the current context. The features can be derived from the relevant rules and input into a machine learning model adapted to the user's specific interactions. The machine learning model can provide one or more suggestions for the application to be used and provide the suggestions for selection on the user interface. For example, based on the input features, the machine learning model can rank the rules according to the importance of the recommended application recipient or both. The suggestions for the application, the recipient, or both can be extracted from the ranked rules and can be displayed on the user interface in the order of the user selection probabilities determined by the machine learning model.

[0117] However, for the people appearing in the images in the image gallery, neither of the above two methods can well recognize the relationships among the people in the image.

[0118] In view of this, the embodiments of the present application provide a method for determining the relationships among people. By obtaining a first image and a second image, determine a first person and a second person, and the second person is the owner of the electronic device. According to the images in the electronic device, determine a third person who has a first association relationship with the first person, and the third person has a second association relationship with the second person. According to the first association relationship and the second association relationship, determine the relationship between the first person and the second person. Therefore, by mining the relationship between the first person and the second person through the third person who is simultaneously associated with the first person and the second person, the relationships among the people of the second person can be further expanded.

[0119] For example, please refer to Figure 4A , in the photo gallery application of a mobile phone, there are two images. One image includes person A and person C, and the other image includes person C and person B. The mobile phone can determine the relationship between person A and person B through the intermediate person C.

[0120] Taking Figure 4A Image 1 in (a) as an example, Image 1 includes person A and person C. If it is necessary to determine the relationship between person A and person C, one possible implementation is that the mobile phone determines the distance and / or pose between person A and person C on Image 1 through image recognition, and determines the relationship between person A and person C according to the distance and / or pose.

[0121] Another possible implementation is that the mobile phone can determine the scene of Image 1 through image recognition, and determine the relationship between person A and person C according to the scene. For example, if the scene in Image 1 is an office scene, then the relationship between person A and person C in Image 1 is a colleague relationship. Optionally, when the mobile phone recognizes the image scene, it can be implemented in the manner shown in Figure 4B . As shown in Figure 4B , people detection can be performed on Image 1 to respectively determine person A and person C, and then the handheld items and accessories of each person in Image 1 are respectively detected, and the scene is judged through the handheld items and accessories. The handheld item is the item in the hand of the person in the image, and the accessory is the object that the person in the image relies on or the object in contact with the user. For example, as shown in Figure 4B , the mobile phone recognizes the mobile phone and tablet computer in the hand of person A, and the umbrella in the hand of person B, and determines that the scene is a beach scene or an outdoor scene. The mobile phone can also perform global semantic recognition on Image 1, and the semantic recognition result of Image 1 is "happy". Furthermore, the feature extraction module extracts features from the recognized content, and models the relationship between people through the extracted features to determine the relationship between person A and person C, such as a friend relationship.

[0122] The above is an example with Image 1 in Figure 4A (a). For Image 2 in Figure 4A (b), it is the same principle.

[0123] Please refer to Figure 5 , which is a schematic flowchart of the method for determining the relationship between people provided by an embodiment of the present application. This method can be applied to a mobile phone, and the mobile phone has the structure of Figure 1A and / or Figure 1B . As shown in Figure 5 , the process includes:

[0124] S501. The mobile phone 100 obtains a first image.

[0125] Users can obtain images in various ways and store them in the image library. For example, they can take pictures using the front or rear cameras of mobile phone 100 and save the obtained images in the Joint Photographic Experts Group (JPEG) format in the image library. Or, users can also take screenshots on mobile phone 100 and store the screenshots in the image library. Or, users can also obtain images from third-party software such as "WeChat" and "QQ" and store them in the image library. For example, users can obtain pictures shared by WeChat friends. Since the user's WeChat friends may include the user's family members, friends, or colleagues, the user may obtain images taken by family members, friends, or colleagues. Or, users can obtain images shared by their family members, friends, or colleagues through a shared photo album in the cloud, and so on.

[0126] The images stored in the image library include portrait images, which refer to images taken by the user alone and group photos of multiple people, etc. For example, group photos of people with scenic spots or iconic buildings during travel. Selfies of people at home or group photos with family members. Group photos of people with family members, friends, or colleagues in a restaurant. Selfies of people in the office or group photos with colleagues. Portrait images include human features. For example, images including the facial, body, etc. features of one or more people. Taking person A as the central or reference person in the relationship, portrait images include images taken by person A alone, and can also include group photos of person A with person B and person A with person C. Or, the group photos in the portrait images may not include person A, but only include the group photo of person B and person C, or only include the photo of person B alone or the photo of person C alone, etc.

[0127] Furthermore, the environments where portrait images are located can be divided into different scenes. For example, they include home scenes, party scenes, and office scenes. Among them, home scenes include the living room, dining room, bedroom, etc. in the user's home. Portrait images taken in home scenes can include relatives such as the user's spouse, parents, and children. Party scenes include public places such as restaurants and cinemas, and there are often text descriptions in these places, such as "hot pot", "cinema", etc. Office scenes are generally the user's workplace, and portrait images often include the user's colleagues, etc.

[0128] Among multiple images stored in the image gallery, there may be multiple persons, such as Persons B, C, D, and E. These persons appear in the images stored in the image gallery. However, generally, the relationships between these persons and the owner of the mobile phone 100, Person A, are not clear. For example, the mobile phone 100 can obtain a first image that includes Person B. Maybe there is only a single photo of Person B in the image gallery, or there is only a group photo of Person B and other persons in the image gallery. From the perspective of the image, Person B and the owner Person A are separated and there is no direct connection. This application aims to determine the relationship between Person B and the owner of the mobile phone 100, and the determination process will be introduced in detail below.

[0129] S502. The mobile phone 100 obtains a second image, and the second image includes a second person, and the second person is the owner of the mobile phone 100.

[0130] If the relationship between Person B and the owner of the mobile phone 100 needs to be determined, it is necessary to first determine the owner from the images stored in the mobile phone 100. Only on the premise of determining the image corresponding to the owner of the mobile phone 100 can the relationship between the owner's persons be further determined according to the images. Person B can be called the first person, and the owner of the mobile phone 100 can be called the second person. That is to say, the relationship between Person B and Person A is determined with the second person as the reference person or the central person in the person relationship. The image including the second person can be called the second image. The second image includes, but is not limited to, the images in the image gallery application, and the second image can be determined in different ways.

[0131] In some embodiments, such as Figure 6 , the owner of the mobile phone 100 sets an owner avatar in the "My Business Card". If the mobile phone 100 detects the owner avatar, it can obtain the avatar. Considering that the owner may set the owner avatar with a non-person image when setting the owner avatar, for example, a landscape photo, an animal photo, etc. are used to set the owner avatar. The mobile phone 100 can detect whether the owner avatar is a person image. If it is a person image, the mobile phone 100 obtains the owner avatar as the second image. Optionally, the "My Business Card" can be located in a certain application of the electronic device. For example, it can be the contact application, or the settings application, or other applications. The embodiments of the present application do not limit its specific location.

[0132] In some embodiments, the mobile phone 100 can also detect whether there are self - portrait images in the gallery application and obtain a self - portrait image as the second image. In some cases, if the user has pre - set a self - portrait image set in the gallery, the images stored in the self - portrait image set can be considered as self - portrait images of the second person. Or, in other cases, when the user takes an image using the front - facing camera of the mobile phone 100, the image will be stored. The image can be stored as a file in a format such as Joint Photographic Experts Group (JPEG), and a flag bit is added to the file. The flag bit is used to indicate that the image is taken by the front - facing camera. In this case, the mobile phone 100 can also determine, based on the flag bit, that the image taken by the front - facing camera with the flag bit added in the gallery can be considered as a self - portrait image. The position of the flag bit in the file is not limited in the embodiments of the present application. Additionally, considering that the image taken by the front - facing camera may not necessarily be a portrait image, that is, the image may not include a human object, after the mobile phone determines the image taken by the front - facing camera based on the flag bit, it can also determine whether the image includes a human object. If so, it is determined that the image is a self - portrait image of a person.

[0133] In some embodiments, the mobile phone 100 can obtain the person image with the highest frequency of appearance in the gallery application as the second image. For example, among the images stored in the gallery application, the number of person images taken of a certain person is the largest. Generally speaking, the person image with the highest frequency of appearance in the gallery should be the user's person image, so the mobile phone 100 can obtain this image as the second image.

[0134] It should be understood that when the mobile phone 100 determines the second image, it can flexibly select among the above - mentioned implementation manners. For example, if there is a user's avatar (such as "my business card") in the electronic device and the avatar is a person avatar, the avatar can be directly obtained as the second image. When the mobile phone 100 detects that there is no user's avatar in the electronic device or the avatar is not a person avatar, it can further obtain a self - portrait image in the gallery as the second image. Another example is that a self - portrait image in the gallery can be directly obtained. Different methods are not strictly mutually exclusive and can be flexibly combined according to the actual situation.

[0135] S503. The mobile phone 100 determines a third person C having a first association relationship with the first person B according to the images in the mobile phone 100.

[0136] To further determine the relationship between person A and person B, it can be described in two cases: whether the image in the mobile phone 100 includes a group photo of person A and person B or not. If the image includes a group photo of person A and person B, then it can be based on Figure 4BDirectly identify the relationship between Person A and Person B in the group photo in the shown manner. If the group photo of Person A and Person B is not included in the image, it may be necessary to determine the relationship between Person A and Person B through other means.

[0137] In the embodiment of the present application, for the case where the group photo of Person A and Person B is not included in the image of the mobile phone 100, the relationship between Person A and Person B is determined by introducing a third person. That is to say, if a third person can be determined according to the image in the mobile phone 100, and the third person has an association relationship with Person A, such as a first association relationship, and also has a second association relationship with Person B, then the relationship between Person A and Person B can be indirectly determined based on the first association relationship and the second association relationship. Next, how to determine the third person and the association relationships between the third person and Person A and Person B will be introduced.

[0138] Case 1: The photo library includes a group photo of the third person and the first person, and also includes a group photo of the third person and the second person.

[0139] The mobile phone 100 obtains the third image and the fourth image in the photo library. Among them, the third image includes a group photo of Person A and the third person, and the fourth image includes a group photo of the third person and Person B. In this way, for the third image, the mobile phone 100 can determine the distance and / or posture between Person A and the third person in the third image. The mobile phone 100 can extract the person features of Person A and the third person in the third image through a preset neural network model and determine the distance between the person features of Person A and the third person in the third image. If the distance between Person A and the third person in the third image is relatively close, it can be considered that Person A and the third person have an association relationship. Or, the mobile phone 100 can also determine the posture between Person A and the third person through the extracted person features of Person A and the third person. If the posture between Person A and the third person is hugging, shoulder-slapping, leaning, holding hands, etc., it can be considered that Person A and the third person have an association relationship. Further, the mobile phone 100 can determine the first association relationship between Person A and the third person according to the distance and / or posture between Person A and the third person in the third image. For example, if the distance between Person A and the third person is very close and the posture is hugging, the first association relationship between Person A and the third person can be determined as "spouse", "child", etc. If the distance between Person A and the third person is very close and the posture is "shoulder-slapping, leaning", the first association relationship between Person A and the third person can be determined as "friend", etc. If the distance between Person A and the third person is relatively close, and, the first association relationship between Person A and the third person can be determined as "colleague", etc.

[0140] For the fourth image, the process of determining the second association relationship between Person B and the third person is similar to the process of determining the relationship between Person A and the third person, and will not be elaborated here.

[0141] Case 2: The image library includes a group photo of the third person and the first person, but does not include a group photo of the third person and the second person.

[0142] The mobile phone 100 can obtain a third image, which includes a group photo of the third person and Person B. The mobile phone 100 can determine the first association relationship between the third person and Person B based on the obtained third image in the same way as in Case 1 above, which will not be elaborated here. For the second association relationship between the third person and Person A, since Person A is the owner of the mobile phone 100, the mobile phone 100 may store the contact information and the avatar of the third person. The mobile phone 100 can determine the first association relationship between the first person and the third person according to the third image, and then search in the mobile phone 100 to find a first application that includes relevant information about the third person C, such as the avatar and / or name of the third person. Suppose the mobile phone 100 finds relevant information about the third person in the first application, then determine the second association relationship between the third person and the second person according to the description information of the third person in the first application.

[0143] Taking the address book as an example, since the mobile phone 100 already knows the facial feature information of the third person through the third image, if the mobile phone 100 determines that the address book stores the avatar of the third person based on this facial feature information, it can further determine the second association relationship between the third person and Person A according to the group and / or note where the third person is located in the address book. For example, the mobile phone 100 can match an avatar that is consistent with the facial feature information of the third person in the third image, and the group information and / or note information of the third person is "family", then it can be determined that the second association relationship between Person A and the third person is "family".

[0144] Taking the email application as an example, the email application may include the avatars and / or names of users such as the sender and the recipient. Suppose the third person sends an email to the mobile phone 100 of Person A, then the mobile phone 100 can query that there is an avatar and / or name of the third person in the email application, and then determine the second association relationship between the third person and Person A according to the description information of the third person in the email application, such as the note information, group information, and email address information. For example, the mobile phone 100 queries that there is an avatar and / or name of the third person in the email application, and the note information recorded in the description information is "colleague", then it can be determined that the second association relationship between Person A and the third person is "colleague".

[0145] Taking the calendar application as an example, Person A can record schedules in the calendar application. The schedules may include, for example, event information of some persons (such as the third person C) related to Person A. For example, playing football with friend C on October 1st. Therefore, if the mobile phone 100 can query the name of the third person C in the calendar application, it can determine the second association relationship between the third person and Person A according to the description information of the third person in the calendar application. For example, if the description information of the third person is playing football with friend C, then the mobile phone 100 can extract the keyword of the third person from the description information, such as "friend", and the mobile phone 100 can determine that the third person C and Person A are in a "friend" relationship.

[0146] It should be noted that the above takes the first application as the address book, email application, and calendar application as examples for illustration. In fact, the first application can also be other applications, such as the memo application, and the embodiments of the present application will not list them one by one.

[0147] Case 3: The photo gallery does not include a group photo of the third person and the first person, nor a group photo of the third person and the second person.

[0148] In this case, the mobile phone 100 can identify the scene of the first image. The scene of the first image includes one of a home scene, a party scene, and an office scene. The mobile phone 100 can extract the feature information of the first image through a preset neural network model, and determine the scene of the first image according to the environment where the first person is located in the first image. Among them, the scene can be determined as a home scene through the feature information such as the sofa, dining table, kitchen utensils, and bed in the first image. The scene can be determined as a party scene through the extracted feature information such as the dining room and wine glasses. The scene can be determined as an office scene through the extracted feature information such as the office desk and printer.

[0149] The mobile phone 100 can also determine the scene of the first image through the items held, attached, or worn by person B in the first image. For example, if the item held by person B is "chopsticks", "book", "toy", "remote control", or the attached item is "pillow", "blanket", or the wear is "casual", etc., then the scene of the first image can be considered as a home scene. Another example, if the item held by person B is "wine glass", or the wear is "formal", the scene of the first image can be considered as a party scene. Another example, if the item held by person B is "keyboard", "computer", "printer", the attached item is "office chair", and the wear is "formal", the scene of the first image can be considered as an office scene.

[0150] When there is no group photo of Person B and other people in the photo gallery, the first image can be subjected to scene recognition to obtain a scene recognition result. Based on the scene recognition result, a search can be performed in the mobile phone 100 to find a first application that has information related to a third person C, such as the head portrait and / or name of the third person C. Assuming that the electronic device finds information related to the third person C in the first application, then based on the description information of the third person C in the first application, a second association relationship between the third person and the second person is determined.

[0151] Taking the first application as the address book as an example, assuming that the scene recognition result for the first image is a home scene, the mobile phone 100 can search in the address book to see if there is information about the third person, such as the head portrait and / or name. If it exists, then based on the description information of the third person (such as, note information, group information, email address information, etc.), a second association relationship between the third person and the second person is determined. The mobile phone 100 can determine a first contact that matches the scene recognition result from the note information in the address book according to the scene recognition result, and determine this first contact as the third person. That is to say, by performing scene recognition on the first image, the first person can be associated with the scene in the first image. Then, if it is noted in Person A's address book that the third person is also associated with the same scene, and the address book notes the first association relationship between the third person and Person A, then since Person B and the third person are associated with the same scene, the first association relationship between Person B and the third person, as well as the second association relationship between Person A and the third person, can be determined.

[0152] Taking the first application as the email application as an example, assuming that the scene recognition result for the first image is an office scene, the mobile phone 100 can search the emails exchanged with Person A in the email application to see if there is information about the third person, such as the head portrait and / or name of the third person. If there is the head portrait and / or name of the third person in the email application, then based on the corresponding description information of the third person (such as, note information, group information, email address information, etc.), a second association relationship between the third person and the second person is determined.

[0153] S504. The mobile phone 100 determines the relationship between the first person B and the second person A according to the first association relationship and the second association relationship.

[0154] The mobile phone 100 can determine the relationship between Person A and Person B according to the first association relationship and the second association relationship. For example, if the first association relationship between a third person and Person B is "child", and the second association relationship between Person A and the third person is "husband and / or wife", then it can be determined that the relationship between Person A and Person B is "family member". Another example, if the first association relationship between a third person and Person B is "friend", and the second association relationship between Person A and the third person is "friend", then it can be determined that the relationship between Person A and Person B is "friend". Another example, if the first association relationship between a third person and Person B is "colleague", and the second association relationship between Person A and the third person is also "colleague", then it can be determined that the relationship between Person A and Person B is "colleague", and so on.

[0155] In the above embodiment, when the mobile phone 100 obtains a first image that includes Person B, it determines the relationship between Person A (the phone owner) and Person B according to the intermediate person C. In other embodiments, the mobile phone 100 can also determine the relationship between Person B and Person A according to the image information of the first image. For example, the mobile phone 100 can obtain the shooting time and / or shooting location of the first image, and determine the relationship between Person B and Person A according to the shooting time and / or shooting location. For example, if the shooting time is October 2014 and the shooting location is XXX University, then the mobile phone 100 can determine that Person B is a classmate of Person A.

[0156] After obtaining the relationship between persons through the above technical solution, the mobile phone can store the relationship between persons. For example, please refer to Figure 7, which is a schematic diagram of the personal relationship provided by an embodiment of the present application. The personal relationships involved in the present application include family members, friends, and colleagues. Among them, family members can be further divided into spouses, parents, brothers, sisters, children, etc. Taking person A as the reference person or central person in the personal relationship, that is, person A is the second person in the above solution, multiple levels of personal relationships of person A can be derived. For example, taking person A as the central person or reference person, the first-level personal relationship of person A can be derived. The first-level personal relationship includes person B, person C, and person D, where person B, person C, and person D can be the third persons in the above solution. Among them, person B can be the spouse of person A, person C can be the parents of person A, and person D can be the friend of person A. Based on the first-level personal relationship, the second-level personal relationship of person A can be determined. The second-level personal relationship includes persons E, F, and G who have an indirect relationship with person A. Among them, persons E, F, and G can be the second persons in the above solution. Among them, person E can be the father or mother of person B, person F can be the friend of person B, and person G can be the father or mother of person C. That is to say, according to the solution in the embodiment of the present application, if it is necessary to determine the relationship between persons E, F, and G and the second person A, the relationships between the third persons B, C, and D and persons E, F, and G can be found, and the relationships between persons E, F, and G and person A can be determined indirectly through the third persons B, C, and D.

[0157] After the mobile phone obtains the personal relationship, the personal relationship can be applied. The following are some application scenarios.

[0158] Application scenario 1: Content sharing.

[0159] Taking the sharing of an image by mobile phone 100 as an example, when the mobile phone receives an operation on an image in the gallery application, it can display an interface such as Figure 8A . For the convenience of users to share quickly, the mobile phone can recommend sharing objects within the Figure 8A interface. For example, as shown in Figure 8A , the contact information of the sharing objects, namely person B, person C, and person D, is displayed on the mobile phone.

[0160] A possible implementation method is that the mobile phone determines the sharing objects according to the intimacy. For example, the mobile phone has previously determined the intimacy between each person and the owner of the mobile phone, and can select the persons with higher intimacy as sharing candidates. For example, Figure 8AThe sharing objects include Person B, Person C, and Person D. Among them, Person B, Person C, and Person D are persons with a relatively high level of intimacy with the device owner. Optionally, a possible way for the mobile phone to determine the intimacy between each person and the device owner is that when the mobile phone determines that there are more images of a certain person in the photo gallery, it can determine that the intimacy between this person and the device owner is relatively high. Or, when the mobile phone determines that there are more group photos of a certain person and the device owner in the photo gallery, it can determine that the intimacy between this person and the device owner is relatively high. Or, when the mobile phone determines that the number of interactions with a certain person is relatively high, it can determine that the intimacy between this person and the device owner is relatively high. In short, the embodiments of the present application do not limit the way of determining intimacy.

[0161] Another possible implementation is that the mobile phone determines the sharing objects according to the content and the relationship between the people in the image to be shared (for example, the relationship between the people determined by the method described above). For example, Figure 8A in the image to be shared, there are two people, an adult holding a child. Through image recognition, the electronic device can determine that the two people in the image (i.e., the adult and the child) have a family relationship. Therefore, the mobile phone determines that the sharing objects are the people in the family relationship. Therefore, the mobile phone Figure 8A displays the contact information of Person B, Person C, and Person D in the family relationship in the interface. Taking Person B as an example, the contact information of Person B includes the contact information of Person B in a certain instant messaging application. The instant messaging application can be "Mutual Transmission", "Huawei Share", "AirDrop", "Messages", "Mail", etc. in the mobile phone, or third-party software such as "WeChat", "QQ", etc. in the mobile phone. The embodiments of the present application do not make any limitations. Or, Person A can also upload the image to a group set up in the shared photo album in the cloud, and only the users in the group can access the image.

[0162] Optionally, continuing with Figure 8A as an example, after the mobile phone determines that the sharing objects include Person B, Person C, and Person D according to the content and the relationship between the people in the image to be shared, it can also sort according to the intimacy between each person and Person A (the device owner). For example, Figure 8A assuming that the intimacy of Person B is the highest, the contact information of B is ranked first, and the intimacy of Person D is the lowest, then the contact information of D is ranked at the end.

[0163] In addition, continuing to refer to Figure 8A on the mobile phone, there are also icons of applications such as "AirDrop", "Messages", "Mail", etc. displayed. If the user clicks on the icon of the "Messages" application, the mobile phone can open the interface of the "Messages" application (not shown in the figure), and recommend sharing objects to the user in this interface, for example, the contact information of Person B, Person C, and Person D.

[0164] Such as Figure 8B, the user can also automatically add the recognized person relationships to the groups in content sharing. For example, the user can add the person recognized as having a "family member" relationship with the second person to the "family sharing" group, or can add the person recognized as having a "colleague" relationship with the second person to the "colleague sharing" group. Alternatively, the user can also manually add person relationships and can view the added person relationships.

[0165] After the mobile phone 100 obtains the person relationship between person A and person B through the technical solution provided by the embodiment of the present application, it can be applied to scenario one. The mobile phone 100 receives a sharing operation for the first content. Among them, the first content can include, for example, Figure 3 the pictures in the user's photo gallery as shown, and in addition, can also include the text, voice, etc. stored in the mobile phone 100. The sharing operation can include operations such as the user clicking, dragging, and selecting the content in the interface of the mobile phone 100 for sharing. After the mobile phone 100 receives the sharing operation of the user for the first content, according to the first content, it determines the person to whom the first content is to be shared among the first person relationships. Among them, the first person relationship can refer to the first-level person relationship in the multi-level person relationship as shown in Figure 6 or the second-level person relationship in the multi-level person relationship, which is not specifically limited in the embodiment of the present application. For example, if the content to be shared by the user is a family photo, it can be determined to share the content to the "family members" in the first-level person relationship or the second-level person relationship. If the content to be shared by the user is work-related, it can be determined to share the content to the "colleagues" in the first-level person relationship or the second person relationship. After the user selects the sharing object of the first content, the mobile phone 100 displays the sharing methods of person B. The sharing methods include sending by text message, sending by third-party software, uploading to the cloud, etc. The user can select a sharing method from the optional sharing methods of person B and send the first content to person B through this sharing method.

[0166] Application scenario two, associated person setting.

[0167] As shown in Figure 9A , the "medical emergency card" function can ensure that if the user has an accident and loses consciousness and the mobile phone is in an unlocked state, others can view the content therein by clicking the "Emergency Situation" - "Medical Emergency Card" option. Person A can set personal information in the "medical emergency card" function. For example, person A can set information such as his / her height, weight, blood type, allergy history (not shown in the figure), special disease situation (not shown in the figure), organ donor, preferred language, etc. The "Emergency Contact" option is also set in the "medical emergency card" function. When person A has an accident, the contact information of the emergency contact can be viewed through the "Emergency Contact" option.

[0168] One possible implementation is that the mobile phone determines emergency contacts based on intimacy. For example, the mobile phone determines the person with a higher intimacy as an emergency contact. Regarding the method for the mobile phone to determine the intimacy between each person and the phone owner, please refer to the previous description and will not be repeated here.

[0169] Another possible implementation is that the mobile phone can determine emergency contacts based on the relationship between people. For example, the mobile phone can obtain the relationship of person A (such as family relationship), and add the people in the relationship as emergency contacts to the "Emergency Contacts" option to achieve automatic setting of associated persons.

[0170] Such as Figure 9B ,"Family Account" is used to bind two accounts, which can be payment accounts or other accounts, and the embodiments of the present application do not make limitations.

[0171] One possible implementation is that the mobile phone binds a family account based on intimacy. For example, the mobile phone determines the account of the person with a higher intimacy as the family account.

[0172] Another possible implementation is that the mobile phone can bind a family account based on the relationship between people. For example, if it is recognized that person B is the "mom" of person A, then the mobile phone can search for the contact information of person B in the address book and set the contact information of person B as the "Family Account". The user can set one or more "Family Accounts", and the present application does not make limitations on this.

[0173] Such as Figure 9C the mobile phone can perform automatic setting of associated persons.

[0174] One possible implementation is that the mobile phone determines associated persons based on intimacy. For example, the mobile phone determines the associated persons of the person with a higher intimacy.

[0175] Another possible implementation is that the mobile phone can determine associated persons based on the relationship between people. For example, Figure 9C in, the mobile phone determines that a certain contact is an assistant based on the relationship between people, and can set "Assistant" as the associated person.

[0176] Application scenario three: Do Not Disturb mode setting. Such as Figure 10 the mobile phone can set the "Do Not Disturb mode" function. In the Do Not Disturb mode, the mobile phone only rings, vibrates or pops up a reminder when a contact in the "Contacts" list calls or sends a message to the user, and mutes other contacts who call or send a message to the user.

[0177] One possible implementation is that the mobile phone determines the contacts with a higher intimacy. In the Do Not Disturb mode, when the mobile phone receives calls or messages from these contacts, it can ring, vibrate or pop up a reminder to avoid missing calls from people with a higher intimacy.

[0178] Another possible implementation is that the mobile phone determines contacts based on the relationship between people. For example, it determines the contacts belonging to the family relationship. In the Do Not Disturb mode, when the mobile phone receives calls or messages from these contacts, it can ring, vibrate, or pop up a reminder to avoid missing calls from family members.

[0179] Application Scenario 4: Identification of the relationship between people. The mobile phone 100 can implement the method for determining the relationship between people in the embodiments of the present application through an algorithm model. For Figure 11 For person B in the image in (a), after the algorithm model determines the relationship between the first person B and person A (the device owner) (such as a family relationship), a probability value can also be output. The probability value is used to indicate the probability that the first person B and the second person A are in a family relationship. Optionally, if the probability is low (for example, less than a threshold), a prompt message is output to prompt the user to determine whether the relationship between the first person and the second person is correct. For example, as Figure 11 In (a), when the mobile phone receives an operation (such as a swiping up operation), the mobile phone 100 displays an interface as shown in Figure 11 (b). The prompt message "Family?" is output in this interface, indicating that the mobile phone recognizes that the first person B and person A are in a family relationship, but the probability is low. The user can manually set the relationship between person B and person A, such as setting it as a family relationship or other relationships. Of course, if the probability is high (for example, greater than or equal to the threshold), no prompt needs to be output.

[0180] Based on the same concept, Figure 12 An electronic device 1200 provided by the present application is shown. The electronic device 1200 may be the tablet computer described above. As Figure 12 shown, the electronic device 1200 may include: one or more processors 1201; one or more memories 1202; a communication interface 1203, and one or more computer programs 1204. The above components can be connected through one or more communication buses 1205. Among them, the one or more computer programs 1204 are stored in the above memory 1202 and are configured to be executed by the one or more processors 1201. The one or more computer programs 1204 include instructions, and the above instructions can be used to execute the relevant steps of the tablet computer in the corresponding embodiments above. The communication interface 1203 is used to implement communication with other devices. For example, the communication interface may be a transceiver.

[0181] In the embodiments provided by the present application above, the method provided by the embodiments of the present application is introduced from the perspective of an electronic device (such as a tablet computer) as the execution subject. To implement each function in the method provided by the embodiments of the present application above, the electronic device may include a hardware structure and / or software modules, and implement the above functions in the form of a hardware structure, software module, or a combination of a hardware structure and a software module. Whether a certain function among the above functions is executed in the form of a hardware structure, software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0182] In the above embodiments, depending on the context, the term "when..." or "after..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if detecting (the stated condition or event)" may be interpreted to mean "if determining..." or "in response to determining..." or "when detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)". Additionally, in the above embodiments, relational terms such as first, second, etc. are used to distinguish one entity from another entity, without restricting any actual relationship and order between these entities.

[0183] Reference to "an embodiment" or "some embodiments" etc. described in this specification means that a specific feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, statements such as "in an embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc. that appear in different places in this specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprising", "including", "having" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0184] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc. Without conflict, the solutions of the above embodiments can be combined and used.

[0185] It should be noted that a part of the patent application document of this patent contains content protected by copyright. Except for making copies of the content of the patent document of the patent office or the recorded patent document, the copyright owner reserves the copyright.

Claims

1. A method for determining a person relationship, characterized in that, applied to an electronic device, the method includes: Obtain a first image, where the first image includes a first person B, and the first person B is the person whose relationship needs to be determined; Obtain a second image, where the second image includes a second person, and the second person is the owner A of the electronic device; According to the images in the electronic device, determine a third person C who has a first association relationship with the first person B, and the third person C has a second association relationship with the second person A; According to the first association relationship and the second association relationship, determine the person relationship between the first person B and the second person A.

2. The method according to claim 1, characterized in that, The obtaining of the second image includes at least one of the following: Obtain the profile picture of the owner of the electronic device; or, Obtain a self-taken image in the gallery application; or, Obtain the person image with the highest appearance frequency in the gallery application.

3. The method according to claim 1 or 2, characterized in that, According to the images in the electronic device, determining a third person C who has a first association relationship with the first person B includes: Obtain a third image in the gallery of the electronic device, where the third image includes the first person B and the third person C; According to the distance and / or posture between the first person and the third person on the third image, determine the first association relationship between the first person and the third person; Before determining the person relationship between the first person B and the second person A according to the first association relationship and the second association relationship, the method further includes: Obtain a fourth image in the gallery of the electronic device, where the fourth image includes the third person C and the second person A; According to the distance and / or posture between the third person and the second person on the fourth image, determine the second association relationship between the third person and the second person.

4. The method according to claim 1 or 2, characterized in that, According to the images in the electronic device, determining a third person C who has a first association relationship with the first person B includes: Obtain a third image in the gallery of the electronic device, where the third image includes the first person B and the third person C; According to the distance and / or posture between the first person and the third person on the third image, determine the first association relationship between the first person and the third person; Before determining the person relationship between the first person B and the second person A according to the first association relationship and the second association relationship, the method further includes: Determine that there is relevant information of the third person C in the first application in the electronic device; According to the description information of the third person C in the first application, determine the second association relationship between the third person C and the second person A.

5. The method according to claim 1 or 2, characterized in that, Determining a third person C who has a first association relationship with the first person B includes: Identify the scene of the first image, where the scene includes one of a home scene, a party scene, and an office scene; Based on the scene recognition result, determine the description information of the third person C existing in the first application in the electronic device, where the description information matches the scene recognition result; Before determining the relationship between the first person B and the second person A according to the first association relationship and the second association relationship, the method further includes: According to the description information, determine the first association relationship between the first person and the third person, and the second association relationship between the second person and the third person.

6. According to the method described in claim 1, characterized in that, Determining the third person C having a first association relationship with the first person B according to the image in the electronic device includes: Obtain a first image, where the third person C is included in the first image; According to the shooting time and / or shooting location of the first image, determine the first association relationship between the third person C and the first person.

7. According to the method described in claim 1, characterized in that, The method further includes: Determine the first intimacy between the first person and the second person, the second intimacy between the first person and the third person, and the third intimacy between the second person and the third person.

8. According to the method described in claim 1, characterized in that, The relationship between the first person and the second person is the first relationship, and the method further includes: Determine the probability that the first person and the second person belong to the first relationship; When it is determined that the probability is less than the threshold, output a prompt message, where the prompt message is used to prompt that the probability that the first person and the second person belong to the first relationship is low.

9. According to the method described in claim 8, characterized in that, The method further includes: Receive a user operation, where the user operation is used to determine the relationship between the first person and the second person.

10. According to the method described in claim 5, characterized in that, Identifying the scene of the first image includes: Identify the scene of the first image through at least one of the handheld object, attached object, clothing, and the environment where the first person is located in the first image.

11. According to the method described in any one of claims 1 to 10, characterized in that, The relationship between the first person B and the second person A is the first relationship, and the method further includes: Receive a sharing operation for the first content; According to the first content, determine to share the first content with the people in the first relationship; Display the sharing method of the first person, where the sharing method is used to share the first content with the first person.

12. According to the method described in claim 11, characterized in that, The first content includes content related to the fourth person in the first relationship and does not include content related to the first person B.

13. According to the method described in any one of claims 1 - 12, characterized in that, The relationship between the first person B and the second person A is the first type of relationship, and the method further includes: Setting the device to the Do Not Disturb mode according to a user operation, in which new messages from persons not within the first type of relationship are not responded to.

14. The method according to any one of claims 1 - 13, wherein, The relationship between the first person B and the second person A is the first type of relationship, and the method further includes: Setting, according to a user operation, an emergency contact in a first application as a person within the first type of relationship; Contacting the first person B in response to an emergency contact operation.

15. An electronic device, wherein, it includes: A display screen; One or more processors; A memory; One or more programs; wherein the one or more programs are stored in the memory, and the one or more programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method steps according to any one of claims 1 - 14.

16. A computer-readable storage medium, wherein, it includes computer instructions that, when running on an electronic device, cause the electronic device to perform the method according to any one of claims 1 - 14.

Images