Notification system and method for notifying user based on facial matching

Abstract

When there is a match between the images, the notification system notifies the user. The notification system includes a network interface to receive images from a plurality of user devices including a first user device and a second user device, a memory to store a database and instructions, and a processor. The processor executes instructions to function as: an image processing unit that processes a received image; a face recognition unit that recognizes one or more faces in each of the received images; a face matching unit that checks whether a face in the first image from the first user device matches a face in the second image from the second user device; a metadata processing unit that processes metadata in each of the received images; and a notification unit that notifies the first user device and the second user device in a case where the face in the first image matches the face in the second image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of U.S. Provisional Patent Application Serial No. 17 / 854,476, filed on June 30, 2022, the entire contents of which are hereby incorporated by reference herein. Technical Field

[0003] The present disclosure relates to notification systems and methods for notifying a user based on a face match, and more particularly to notification systems and methods for notifying a user based on a face match in a digital image file. Background Art

[0004] Photos capturing deceased or still alive persons are stored in digital storage in analog form (e.g., photos developed on paper or in negatives) or in digital form (e.g., digital images with extensions of jpg, jpeg, bmp, tiff, gif, png, esp, raw, psd, xcf, ai, cdr, etc.). Such photos can remind the photo holder of the person captured in the photo. However, when the same photo is shared with family members or friends, the photo holders cannot contact each other based on the same photo when they live apart from each other. In addition, the photo on paper usually does not have any written information, and the digital form of the photo on paper usually does not include metadata about the photo on paper.

[0005] Furthermore, photographs, whether in analog or digital form, are two-dimensional in nature and cannot provide three-dimensional information contained therein. Summary of the invention

[0006] The images uploaded by the users may include one or more identical faces, which means that the users are relatives or friends. Thus, by identifying faces in uploaded images and comparing faces in uploaded images, the users are able to connect with other users when there are one or more matching faces. Furthermore, by generating a three-dimensional virtual representation of the image, the user may be able to experience the environment depicted in the image and communicate with the people depicted in the image.

[0007] According to various aspects of the present disclosure, when there is a match between images, a notification system notifies a user. The notification system includes: a network interface that receives images from multiple user devices including a first user device and a second user device; a memory that stores a database and instructions; and a processor. The processor executes instructions to serve as: an image processing unit that processes received images; a facial recognition unit that identifies one or more faces in each received image; a facial matching unit that checks whether a face in a first image from a first user device matches a face in a second image from a second user device; a metadata processing unit that processes metadata in each received image; and a notification unit that notifies the first user device and the second user device when a face in the first image matches a face in the second image.

[0008] In aspects, the processor further executes the instructions to function as a relation unit configured to associate faces with corresponding metadata.

[0009] In various aspects, in a case where the received image does not have metadata therein, the metadata processing unit is further configured to send a request for metadata to a corresponding user device and receive metadata from the corresponding user device.

[0010] In various aspects, the facial recognition unit is further configured to group the same faces in the received images from each user device and set representative faces of the same faces. The facial matching unit is further configured to calculate the difference between a first representative face in the received image from the first user device and a second representative face in the received image from the second user device. In the case where the difference is between a first threshold and a second threshold, the facial matching unit is further configured to calculate a similarity score between the metadata of the first representative face and the second representative face. In the case where the similarity score is greater than or equal to a third threshold, the notification unit notifies the first user device and the second user device.

[0011] In various aspects, the metadata includes personal information, geographic information, and image information.

[0012] In various aspects, the database includes user information for each user. The first user device and the second user device are notified based on the user information.

[0013] According to aspects of the present disclosure, a notification method notifies a user when there is a match between images. The notification method includes receiving images from a plurality of user devices including a first user device and a second user device, processing the received images, identifying one or more faces in each processed image, checking whether a face in a first image from the first user device matches a face in a second image from the second user device, processing metadata in each received image, and notifying the first user device and the second user device if a face in the first image matches a face in the second image.

[0014] In various aspects, the notification method further includes associating the face with corresponding metadata.

[0015] In various aspects, the notification method further includes, in a case where the received image does not have the metadata therein, sending a request for the metadata to the corresponding user device, and receiving the metadata from the corresponding user device.

[0016] In various aspects, facial matching is checked by grouping identical faces in received images from each user device and setting a representative face of the identical faces. Facial matching is further checked by calculating a difference between a first representative face in a received image from a first user device and a second representative face in a received image from a second user device. Facial matching is checked by calculating a similarity score between metadata of the first representative face and the second representative face in a case where the difference is between a first threshold and a second threshold. The first user device and the second user device are notified in a case where the similarity score is greater than or equal to a third threshold.

[0017] In various aspects, the notification method further includes storing a database including user information of each user, and notifying the first user device and the second user device based on the user information.

[0018] According to aspects of the present disclosure, a non-transitory computer-readable storage medium includes instructions that, when executed by a computer, cause the computer to perform a notification method for notifying a user when a match exists between images. The notification method includes receiving images from a plurality of user devices including a first user device and a second user device, processing the received images, identifying one or more faces in each processed image, checking whether a face in a first image from the first user device matches a face in a second image from the second user device, processing metadata in each received image, and notifying the first user device and the second user device if a face in the first image matches a face in the second image.

[0019] According to various aspects of the present disclosure, a virtual reality system creates a virtual three-dimensional (3D) representation of an image. The virtual reality system includes a processor, a memory, the memory being coupled to the processor and storing one or more instructions. The instructions, when executed by the processor, cause the virtual reality system to function as: an access unit, the access unit being configured to access a first image from a database, the first image being input into the database by a user; a virtual 3D generation unit, the virtual 3D generation unit being configured to perform image processing on the first image to detect at least one object depicted in the first image and generate a virtual 3D representation of the at least one object; a receiving unit, the receiving unit being configured to receive a command for modification via a user interface; and a modification unit, the modification unit being configured to modify the virtual 3D representation based on the command.

[0020] In various aspects, the virtual 3D generation unit is further configured to generate a 3D mesh of at least one object. The virtual 3D generation unit employs an artificial intelligence module to detect at least one of an edge, a node, and a face of the at least one object to generate a plurality of features of the 3D mesh. The command comprises an audio command. The instructions, when executed by the processor, further cause the virtual reality system to function as a language processing unit, the language processing unit being configured to parse the audio command to identify one of a plurality of features of the virtual 3D representation expressed in the audio command. The modification unit modifies the identified feature of the plurality of features of the virtual 3D representation according to the audio command.

[0021] In various aspects, the audio command is used to identify a new object or feature, and the modification unit adds the new feature or object expressed in the audio command to the virtual 3D representation.

[0022] In various aspects, the instructions, when executed by the processor, further cause the virtual reality system to function as an animation unit configured to animate a person to speak or move in the virtual 3D representation when the at least one object includes the person.

[0023] In various aspects, the access unit is further configured to access a second image from a database, the second image being input into the database by another user and being related to the first image. The virtual 3D generation unit is further configured to process the second image to detect at least one object depicted in the second image and to determine at least one difference between at least one object depicted in the first image and the second image in the second image, and the modification unit is further configured to modify the virtual 3D representation of the at least one object based on the difference.

[0024] According to aspects of the present disclosure, a method creates a virtual three-dimensional (3D) representation of an image. The method includes accessing a first image from a database, the first image being input into the database by a user; processing the first image to detect at least one object depicted in the first image; and generating a virtual 3D representation of the at least one object. The method also includes receiving a command to modify the virtual 3D representation via a user interface, and modifying the virtual 3D representation based on the command.

[0025] In various aspects, generating a 3D virtual representation includes generating a 3D mesh of at least one object. Generating the 3D mesh of at least one object includes detecting at least one of edges, nodes, or faces of the at least one object to generate a plurality of features of the 3D mesh using artificial intelligence.

[0026] In various aspects, the command comprises an audio command, and the method further comprises parsing the audio command to identify one of a plurality of features of the virtual 3D representation, or to identify a new feature to be added to the virtual 3D representation expressed in the audio command. Modifying the virtual 3D representation comprises modifying the identified one of a plurality of features of the virtual 3D representation, or adding the new feature to the virtual 3D representation, according to the audio command.

[0027] In various aspects, the method further includes, when the at least one object includes a person, animating the person to speak or move in the virtual 3D representation.

[0028] In aspects, a virtual 3D representation of at least one object is modified based on the difference.

[0029] According to aspects of the present disclosure, a non-transitory computer-readable storage medium includes instructions that, when executed by a computer, cause the computer to perform a method for creating a virtual three-dimensional (3D) representation of an image. The method includes accessing a first image from a database, the first image being input into the database by a user; processing the first image to detect at least one object depicted in the first image, and generating a virtual 3D representation of the at least one object. The method also includes receiving a command to modify the virtual 3D representation via a user interface, and modifying the virtual 3D representation based on the command.

[0030] The details of one or more aspects of the present disclosure are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of the techniques described in the present disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Various aspects and features of the present disclosure are described below with reference to the drawings, wherein like numerals represent like or corresponding elements in each of the several views:

[0032] Figure 1 is a block diagram of a notification system according to aspects of the present disclosure;

[0033] Figure 2 is a graphical illustration of a digital image structure stored in a notification system according to aspects of the present disclosure;

[0034] Figure 3 is metadata stored in a notification system according to aspects of the present disclosure;

[0035] Figure 4 and Figure 5 is a flow chart of a notification method according to aspects of the present disclosure;

[0036] Figure 6 is a block diagram of a virtual reality system according to aspects of the present disclosure;

[0037] Figures 7 to 9 is a flow chart of a method for generating a virtual three-dimensional representation according to aspects of the present disclosure; and

[0038] Fig.10 is a block diagram of a computing device in accordance with aspects of the present disclosure. DETAILED DESCRIPTION

[0039] In the present disclosure, a notification system and method are provided for providing notifications to two or more users when their photos include the same face, so that related family or friends can contact each other. For example, when one or more images that have been shared by a user include family members or friends, the user is likely to be a close or distant family member or friend. However, the holders of the image cannot contact each other based solely on the possession of the image. Based on the notification sent to the user by the notification system of the present invention, they can contact each other.

[0040] In addition, a virtual reality (VR) system and method are provided in the present disclosure for providing a VR space for related family or friends to connect with each other in the VR space. When a person captured in an image has passed away, the holder of the image cannot meet or communicate with the person based solely on the possession of the image. However, the VR system can provide a function that enables the user of the image to meet and communicate with even deceased family members or friends.

[0041] In this disclosure, unless otherwise specified, "image" or "images" is used to refer to digital images, and "virtual reality" ("VR") is used to include virtual reality, augmented reality, and mixed reality. Based on the notification, users can meet and communicate with each other in the VR space. The terms "system", "server", "computer", and "user device" refer to computing devices capable of performing arithmetic and logical calculations. The terms "a", "an", and "one" may be used to mean one or more.

[0042] exist Figure 1 A notification server 100 according to aspects of the present disclosure is provided. When one or more images of each user include the same face or person, the notification server 100 sends a notification to both users. By providing notifications to the users, they can connect with each other via the same face or the same person in the image. In this regard, the notification server 100 sends notifications to the multiple user devices 1801-180 n Images are received from a plurality of users, or a plurality of users upload images to the image repository 110 of the notification server 100. The image repository 110 may be an internal or external storage medium or a storage medium in the cloud.

[0043] The notification server 100 includes an image processing unit 120 that processes images stored in the image repository 110. The image processing unit 120 may resize the stored images so that each resized image has the same resolution. The image processing unit 120 may also adjust the brightness of the stored images so that the resulting images have similar brightness. In addition, the image processing unit 120 may process the stored images to distinguish between foreground and background. In this way, the image processing unit 120 may perform pre-processing on the stored images in preparation for the functionality of the facial recognition unit 140. In one aspect, the pre-processed images may be stored in the image repository 110. In another aspect, the image repository 110 may have a storage medium for storing images received via the user devices 1801-180 n A first memory for images received / uploaded from a plurality of users, and a second memory for storing pre-processed images.

[0044] The notification server 100 further includes a metadata processing unit 130 that is capable of extracting metadata from images stored in digital format in the image repository 110, the extensions of which may include jpg, jpeg, bmp, tiff, gif, png, esp, raw, psd, xcf, ai, cdr, etc. This list of extensions is not exhaustive but is provided as an example. Typically, images in digital format include metadata, which is information about, for example, geographic location and settings such as aperture, shutter speed, ISO number, depth of focus, and dots per inch (DPI). The metadata processing unit 130 extracts such metadata.

[0045] In one aspect, the metadata processing unit 130 may add metadata to the stored image. For example, in the case where the image is originally in an analog format (e.g., a paper photograph or negative) and is subsequently digitized by a scanner or image capture device, the digitized image may not include metadata, or even when the digitized image includes metadata, the geographic location information contained in the metadata in the digitized image may not be the geographic location where the original photograph was captured. In another case where a paper photograph is captured or digitized by an image capture device, the geographic location of the captured image may not be the geographic location of the original photograph. In these cases, the metadata processing unit 130 may provide a user interface requesting the user to enter relevant information about the original photograph in an analog format. For example, the relevant information may be the geographic location information of the original photograph in the analog format.

[0046] After the metadata processing unit 130 extracts or adds metadata, the facial recognition unit 140 performs facial recognition processing on the digital image preprocessed by the image processing unit 120. Specifically, the facial recognition unit 140 sets an area for the face shown in the image and extracts features of the face. Figure 2 , the image 200 includes two sections: a metadata section 210 and an image section 220. The image section 220 includes two persons in the foreground. The metadata processing unit 130 extracts metadata from the metadata section 210, and the face recognition unit 140 sets a region 230 for each face and recognizes the face by extracting features from each face.

[0047] In one aspect, the metadata processing unit 130 may add the holder information of the holder of the image to the metadata. The holder information may include the name and contact information (e.g., address, phone number, email address, etc.) of the holder who uploaded the image. The holder information may further include a permission flag that allows the notification server 100 to contact the holder when a matching face is found in an image uploaded by another holder.

[0048] The metadata processing unit 130 may add time information indicating the capture time of the image. The time information may be automatically added when the image is captured, or manually added when the paper image is digitally scanned.

[0049] In one aspect, the facial recognition unit 140 can utilize a machine learning system and a machine learning algorithm in the process of identifying a face in an image. In various aspects, the machine learning system and the machine learning algorithm can be trained via a supervised or unsupervised manner. For example, a previously captured image can be provided with annotations of facial regions and faces therein, and the machine learning system and the machine learning algorithm are trained by strengthening weights. The machine learning system can include, but is not limited to, a neural network, a deep neural network, a recursive neural network (RNN), deep learning by a neural network, a generative adversarial network (GAN), Bayesian regression, naive Bayes, Monte Carlo methods, nearest neighbors, least squares, mean and support vector regression, long short-term memory (LSTM), and other data science, artificial intelligence, and machine learning techniques.

[0050] After identifying faces in an image, metadata processing unit 130 may receive facial information specific to each identified face from the user. For example, facial information may include the identified person's name, gender, parents' names, siblings' names, date of birth, place of birth, spouse's name, children's names, date of death, burial place, etc. Thus, when an image includes two people, such as Figure 2 As shown, the user can provide corresponding facial information for each recognized face.

[0051] In another aspect, metadata of image 200 stored in metadata section 210 may be inherited to facial information of each recognized face in image 200. Personal information of each recognized face may be added to the facial information by the user.

[0052] Now return to reference Figure 1 , the notification server 100 includes a relationship unit 150, which associates each recognized face with corresponding metadata and facial information. In one aspect, the relationship unit 150 can establish a database by storing the relationship between each recognized face and the corresponding metadata and facial information. The database also stores the original image, and the facial image in the original image, and the relationship between them. The database can be stored for each user. In this way, users can have their own database of friends, colleagues, family members, relatives, acquaintances, etc. In one aspect, the database can be a relational database, a NoSQL database, a cloud database, a key-value database, a hierarchical database, or any other suitable database.

[0053] After processing all images uploaded by one user and identifying all faces in each image, one or more identical faces may exist in the user's database. In this case, the relationship unit 150 may group the identical faces into a group and set a representative face representing the identical faces of the group. The identical face may refer to a face of a person captured in different directions or showing different facial expressions. The representative face may show the entire face of a person in a substantially forward facing position in the group of identical faces. The database may store identical faces hierarchically under the representative face.

[0054] In one aspect, since a person's face ages over time, a machine learning algorithm can manipulate and perform morphological processing so that the face in the image can be transformed into a face of a reference age (e.g., 20, 30, 40, etc.). By doing so, the representative face can represent a face of the reference age.

[0055] refer to Figure 3 , the database 300 can be established by the relationship unit 150 based on an image. The top level is the image, and includes geographic location information and image information, which are saved in the metadata part of the image or can be added by the owner of the image. The top level also includes owner information of the owner of the image, which includes name, contact information and notification logo.

[0056] The next relevant layer includes all faces captured in the image and recognized by the facial recognition unit 140, Face1-Face n Each face may be a representative face of the same face in the image. Each face is associated with personal information of the face. Personal information is information specific to a face, including name, gender, date of birth, place of birth, date of death, burial place, marital information, family information, etc.

[0057] After processing all images uploaded by the user, the relationship unit 150 may combine the databases of each image. For example, the faces of each image may be combined. For example, when the same face is found in two or more databases of images uploaded by the user, the relationship unit 150 may: select global representatives among the same faces, which are representative faces in two or more databases; and merge the metadata of the same face into one global database together with the global representative faces. In other words, when the user has already added metadata and facial information for one face, the user does not have to add facial information for subsequent identical faces. In the case where the metadata of two images including the same face are different, only the facial information of the two databases may be merged.

[0058] In another case where the facial information of a face in one of the databases is different from the facial information of a face in another database, the metadata processing unit 130 may display a notification indicating the inconsistency between the facial information of the same face, and display a user interface to resolve such inconsistency. Then, the user corrects the inconsistency so that the relationship unit 150 may be able to update the facial information of one global representative face to have the correct information. After combining all databases, the relationship unit 150 may establish a global database for each user.

[0059] Now return to reference Figure 1 , the image repository 110 further includes a face matching unit 160 and a notification unit 170. After processing all images from all users, the face matching unit 160 may compare the recognized face of the image of one user with the recognized face of the image of another user. In other words, the face matching unit 160 compares the global database of one user with the global database of another user. Specifically, the face matching unit 160 may compare the global representative face in the global database of one user with the global representative face in the global database of another user. In the following, without losing the general context, "representative face" will be used for "globally representative face".

[0060] In one aspect, representative faces may be standardized so that they have substantially the same face size and / or substantially similar brightness. The face matching unit 160 may also utilize machine learning systems and machine learning algorithms by strengthening weights when comparing faces or representative faces. In another aspect, machine learning systems and machine learning algorithms may be trained via a supervised or unsupervised manner. For example, previously acquired faces may be provided with annotations, and machine learning systems and machine learning algorithms may be trained by strengthening weights. Machine learning systems may include, but are not limited to, neural networks, deep neural networks, recursive neural networks (RNNs), deep learning by neural networks, generative adversarial networks (GANs), Bayesian regression, naive Bayes, Monte Carlo methods, nearest neighbors, least squares, mean and support vector regression, long short-term memory (LSTM), and other data science, artificial intelligence, and machine learning techniques.

[0061] In a further aspect, the face matching unit 160 may compare the global representative faces in the global database after age matching by performing morphological processing on faces of different ages so that the ages of the two faces match or are at least similar to each other.

[0062] Since not every image of a face is exactly the same, when the difference between two representative faces is less than a first threshold, the face matching unit 160 can determine that the two representative faces are the same. When the difference is greater than the first threshold but less than a second threshold (the second threshold is greater than the first threshold), the face matching unit 160 can determine that the two faces are very similar, but may not be able to determine that they are the same. In this case, the face matching unit 160 can also compare the metadata associated with each face. Specifically, the face matching unit 160 compares the personal information of one representative face (e.g., a person's name, gender, parents' names, siblings' names, date of birth, place of birth, spouse's name, children's name, date of death, burial place, etc.) with the personal information of another representative face. When the personal information matches each other, the face matching unit 160 can determine that the representative faces match and consider the representative faces to be the same person. In one aspect, the face matching unit 160 may not compare every part of the metadata, but compare a part of the metadata, such as a person's name or birthday. In this regard, the face matching unit 160 can prioritize the metadata, and metadata with a higher priority can have a higher weight than metadata with a lower priority when compared. When metadata with higher priorities match each other, the face matching unit 160 may determine that the representative faces are the same, which means that the two users have the same friends, family members, or acquaintances.

[0063] When the difference is greater than the second threshold, the two faces are considered different from each other. The face matching unit 160 then moves to the next representative face of each user for face comparison.

[0064] When the two faces of the two users are determined to match, the notification unit 170 may send a notification about the matched faces to the two users. Since the holder information in the database may have the notification permission of the user, when both users allow such notification, the two users may be notified. When there is no permission of one user, the face matching unit 160 may not notify the user even if the global database of the user has the same face as that in the global database of another user.

[0065] Figure 4 4 is a flow chart of a notification method 400 when a face captured in an image of one user matches a face captured in an image of another user according to aspects of the present disclosure. In step 410 of the notification method 400, an image repository of a user uploads and a notification system receives a digital image, the digital image including an image portion and a metadata portion (e.g., Figure 2 210 and 220).

[0066] In step 420, one or more faces in the image portion of each uploaded image are identified. Prior to such facial identification, the uploaded images may be pre-processed so that the pre-processed images may have substantially the same facial size and / or substantially similar brightness.

[0067] In step 430, metadata for each uploaded image is extracted from the metadata portion. The metadata may include geographic location information of the location where the image was captured, as well as setting information such as aperture, shutter speed, ISO number, depth of focus, and dots per inch (DPI). In one aspect, in the event that the metadata portion does not include additional information or the metadata needs to be updated, the additional information may be added to the metadata.

[0068] Additionally, facial information may be added to each recognized face in each image in step 430. Since the metadata in the metadata portion of each image does not provide sufficient information about the recognized faces, the user may be asked to enter information about the recognized faces. The facial information may include name, gender, parents' names, siblings' names, date of birth, place of birth, spouse's name, children's names, date of death, burial place, etc. for each recognized face.

[0069] In step 440, the identified faces and corresponding metadata and facial information are correlated with each other and the relationship may be saved in a database. A database may be saved for each user, and a user's database includes all faces captured in all images uploaded by the user.

[0070] When the same face exists in the user's database, in step 450, the same face is grouped together, and a representative face of the same face can be selected. That is, in step 460, the entries of the same face are combined into one entry of the representative face in the user's database. In particular, for the facial information of the representative face, the facial information of the same face can be merged. The representative face can show the entire face facing basically forward. In addition, the entries of the facial information of the same face other than the representative face can be removed from the user's database, and other images of the same face can be saved under the representative face in the user's database. After the grouping in step 450, in step 460, the final version of the user's database can be saved together with the metadata and facial information.

[0071] By repeatedly performing steps 410 - 460 for each user, a database may be built for each user without redundancy of representative faces.

[0072] In step 470, a user, namely user iEach representative face (or first face) in the database of (or first user) is associated with another user, namely user j The first user is compared with each representative face (or second face) in the database of the first user (or second user). In one aspect, the difference between the features of the first face and the features of the second face is calculated.

[0073] In step 480, it is determined whether the difference is less than or equal to a first threshold. The first threshold may be predetermined for the same face. If the difference is less than the first threshold, the first face and the second face may be considered to be the same. In this case, in step 490, the two users may be notified of the same face.

[0074] In one aspect, the database includes notification permission information. By setting notification permissions to off, the user is not bothered by notifications when a face stored in the user's database matches a face stored in another user's database. Additionally, by setting notification permissions to off, the user may not be able to send notifications to other users for one or more faces stored in the user's database.

[0075] When it is determined in step 480 that the difference is not greater than the first threshold, executing Figure 5 Steps 510-530 are performed to check whether the first representative face is similar to the second representative face. In step 510, it is determined whether the difference is less than or equal to a second threshold, which is greater than the first threshold. The second threshold can be predetermined and used to determine similar faces.

[0076] In the case where it is determined that the difference is not less than or equal to the second threshold, the first representative face is considered different from the second representative face, and the notification method 400 ends with comparing the first representative face and the second representative face. Otherwise, in step 520, a difference score between the metadata of the first representative face and the metadata of the second representative face is calculated. The metadata may include the geographic location and personal information of the representative face. In one aspect, the difference score may be calculated with different priorities or weights of items in the personal information. For example, name and hometown may have a higher weight than marriage or death date.

[0077] When calculating the individual difference score for each item, the individual difference score can be the difference between the items. For example, "Smith" and "Smithson" are different but similar. Similarly, "Robinson" and "Robison" are different but similar. In addition, the difference between "Smith" and "Smithson" may be greater than the difference between "Robinson" and "Robison". Regarding dates, the individual difference score can be the difference between two dates. The individual difference score can be multiplied by the corresponding weight, and all individual difference scores are added to obtain a difference score.

[0078] In step 530, the difference score is compared to a third threshold. In the case where it is determined that the difference score is less than or equal to the third threshold, in step 490, the first representative face and the second representative face are considered to be the same, and the corresponding two users are notified. If not, the first representative face and the second representative face are considered different, and the notification method 400 ends with comparing the first representative face and the second representative face.

[0079] By performing steps 410-490 and 510-530, the notification method 400 compares the first representative face of the first user with the second representative face of the second user. The notification method 400 is further performed so that each representative face of the first user is compared with each representative face of the second user. In addition, the notification method 400 is also performed between every two possible users of all users. In this way, any pair of users having images with the same face are notified by the notification method 400.

[0080] refer to Figure 6 A virtual reality system (VRS) 600 for creating a virtual three-dimensional (3D) representation of an image is provided. The VRS 600 can enable a user to input an image into the VRS 600, detect at least one object in the image, and generate a virtual 3D representation of the at least one object in the image, and the user can interact with or view the virtual 3D representation via any virtual 3D display device known to those of ordinary skill in the art. The VRS 600 includes an access unit 610, a virtual 3D generation unit 620, a receiving unit 630, a modification unit 640, a natural language processing (NPL) unit 650, and an animation unit 660.

[0081] The access unit 610 can access a first image from a database, the first image being accessed by a user via a user device (eg, via Figure 11) is input into a database by a user device 170-1 of the user device 170-1. In various aspects, the access unit 610 can access a second image related to the first image. Each of the first image and the second image can include metadata, and the access unit 610 can process the metadata to determine whether the first image and the second image are related. For example, where the metadata of the first image indicates the location where the first image was captured and the metadata of the second image also indicates the same location, the access unit 610 can verify that the first image and the second image are related. In addition, in a configuration where the first image can depict the exterior of a building and the second image can depict at least a portion of the interior of the building, the access unit 610 verifies that the images relate to the same house.

[0082] The virtual 3D generation unit 620 may perform image processing on the first image to detect one or more objects depicted in the first image and generate a virtual 3D representation of the one or more objects. The one or more objects may be at least one of an environment (e.g., a house, a mountain, a lake, a boat, a street, etc.), a person, an animal, or an object (e.g., a sofa, a lamp, a book, a shovel). The virtual 3D generation unit 620 may generate a 3D mesh of the object and generate a plurality of features of the 3D mesh.

[0083] Alternatively, the virtual 3D generation unit 620 can use tessellation to develop a virtual 3D representation of an object. Multiple features of an object include size, color, texture, etc., and each feature or object is identified and classified. For example, a feature may be classified as a 7-foot-high wall with geometric printed wallpaper. The virtual 3D generation unit 620 classifies the features so that each feature can be selected or identified by a user, or the modification unit 640 can further refine the virtual 3D representation. In another example, in the case where the image depicts a playground, the virtual 3D generation unit 620 is able to identify swings, slides, or climbing obstacles and classify them. In various aspects, the first image or the second image can depict a fictitious environment, person, animal, or article. For example, the first image can depict a scene from a movie or a stage play, the scene having a set and / or being animated, and the virtual 3D generation unit renders a virtual 3D representation of a fictitious environment, person, animal, or article.

[0084] The virtual 3D generation unit 620 may include an artificial intelligence module that processes images to generate a 3D mesh or mosaic of an object. The virtual 3D generation unit 620 may utilize machine learning algorithms, computer vision, convolutional neural networks, etc. to process images and / or generate a 3D mesh or mosaic of an object. The artificial intelligence module is further configured to detect at least one of the edges, nodes, or faces of an object depicted in the image. The edges, nodes, or faces may be used to map and generate a 3D mesh or mosaic of an object. The artificial intelligence module, machine learning, computer vision, etc. may predict the missing edges, nodes, or faces of an object not depicted in the image. For example, if the image depicts the front of a sofa, the artificial intelligence module determines the edges, nodes, and / or faces, as well as other features of the sofa, including color, size, texture, etc., and determines what the sofa will look like if viewed from another perspective (e.g., from behind). The virtual 3D generation unit 620 then generates additional edges, nodes, or faces of the object to complete the virtual 3D representation of the object.

[0085] The virtual 3D generation unit 620 may have been trained with a training set or database of images of objects with various views. The training set can teach the artificial intelligence model to predict and refine how each object should be rendered in the virtual 3D representation. For example, the training set may include a set of images of different tables and a corresponding virtual 3D representation of each different table, so that the virtual 3D generation unit 620 can predict how the table depicted in the first image may appear in the virtual 3D representation.

[0086] In one aspect, the virtual 3D generation unit 620 can generate a 3D mesh or mosaic from multiple images depicting the same area. For example, one image shows a living room, another image shows a kitchen, and another image shows the house where the living room and kitchen are located. The artificial intelligence module can integrate them together to generate a 3D representation of the house by filling in the gaps not shown in the images.

[0087] In another aspect, the virtual 3D generation unit 620 may be configured to generate a 3D representation from a movie clip, such as Casablanca or The Godfather, and integrate the 3D representation into the environment shown in the image. The source of the 3D representation is not limited to images and movie clips, but may be extended to any other feasible real or virtual object.

[0088] In further aspects, the virtual 3D generation unit 620 may incorporate or link to historical facts. For example, if a building was destroyed during World War II, the virtual 3D generation unit 620 may incorporate that fact into the building so that the generated VR building may be viewed as destroyed even though the VR building was shown intact prior to World War II. Alternatively, historical facts may be linked to the VR building so that when the user's mouse reaches the VR building, a window pops up with a description of the building over time during World War II. Historical facts may include specific time periods, geographic areas, and any other information.

[0089] In yet another aspect, the virtual 3D generation unit 620 may access geographic and temporal information in metadata of the image and generate a VR space by linking historical facts to objects in the VR space based on the geographic and temporal information in the corresponding metadata as described above.

[0090] In yet another aspect, the virtual 3D generation unit 620 may integrate a personal VR that has been personally generated by a user and is private to the user into the VR space. For example, when the user has generated a building (e.g., a house, a playground, a school, a mountain, etc.) and a character (e.g., a person) in the personal VR space and wishes to add the personal VR space to the VR space, the virtual 3D generation unit 620 may integrate the personal VR space at an appropriate location. When the personal VR space is integrated, the state of the integrated personal VR space may become public so that each user may access the integrated personal VR space.

[0091] In various aspects, a user may be able to set an integrated portion of a personal VR space as a VR space to be private, such that other users may be able to access the integrated personal VR space only upon receiving an invitation to the integrated personal VR space.

[0092] Only in the case where there is a conflict between the VR space and the personal VR space, which means that the corresponding parts in the personal VR space and the VR space are not identical to each other or do not correspond to each other, the virtual 3D generation unit 620 may request the user to resolve the conflict.

[0093] The receiving unit 630 enables a user to provide a command to modify the virtual 3D representation. The command may be a voice command. The receiving unit 630 may include an audio input device or a microphone for sensing the voice command. The VRS 600 is thus able to receive a user command to modify the virtual 3D representation and enable the user device to display the modified virtual 3D representation according to the command. The NLP unit 650 may parse the voice command and apply a natural language processing algorithm to the parsed voice command to identify one of a plurality of features of the virtual 3D representation expressed in the voice command.

[0094] The modification unit 640 modifies the feature according to the identified feature. The modification unit 640 can modify the virtual 3D representation in near real time according to the command.

[0095] Returning to the previous example of the playground, the user can use a voice command "The swing is black with green chains, and the slide is tall and has a metallic texture." The NLP unit 650 parses the voice command into a sentence, such as: "The swing is black with green chains. The slide is tall and has a metallic texture." Then, the modification unit 640 identifies the swing in the virtual 3D representation and modifies the features of the swing so that the swing is black with green chains. The modification unit 640 also modifies the virtual 3D representation of the slide so that the slide is tall and has a metallic appearance.

[0096] The voice commands may include commands to add new features or objects to the virtual 3D representation. Returning again to the previous example of the playground, the voice command may include the statement "add a bench, flanked by flowers on the side, opposite the slide." The NLP unit 650 processes the voice command and recognizes the objects and instructions, i.e., "add a bench," "flanked by flowers on the side (of the bench)," the bench being placed "opposite the slide." The modification unit 640 then modifies the virtual 3D representation to include a bench, flanked by flowers on the side, opposite the bench. In this way, the user can arrange, modify, or add objects or features, and thereby better approximate the 3D virtual representation depicted by the image.

[0097] In various aspects, the command may include a command to modify the virtual 3D representation by processing a second image related to the first image. The virtual 3D generation unit 620 then analyzes the second image and identifies objects or features depicted in both the first image and the second image, and renders the virtual 3D representation to incorporate additional features, including additional edges, nodes, or faces of one or more objects. The virtual 3D generation unit 620 thus identifies the difference between the second image and the first image, and modifies the virtual 3D representation based on the detected difference. The virtual 3D generation unit 620 modifies the first image to include the difference, wherein the difference between the second image and the first image provides additional details of one or more objects or features from the first image, beyond any previous estimate or prediction of the appearance of the one or more objects or features. The virtual 3D generation unit 620 may interpolate between the first image and the second image to obtain a refined prediction of the appearance of one or more objects, and modify the 3D mesh of the virtual 3D representation as needed.

[0098] For example, a user may input an image depicting the user's grandmother sitting in the grandmother's living room at the grandmother's house, and the VRS 600 generates a virtual 3D representation of the grandmother and the living room as depicted in the image. The VRS 600 enables the user to further modify and refine the virtual 3D representation as desired, for example, by inputting commands to process an image of the kitchen of the grandmother's house, which is open to the living room. The virtual 3D generation unit 620 then renders the virtual 3D representation so that the virtual model of the grandmother's house includes the living room and the kitchen. In this way, the virtual 3D representation can be further refined, extended, enlarged, or developed with additional VR structures. In this way, a building, structure, object, or environment can be generated approximately as a virtual 3D representation.

[0099] The user may provide a voice command to increase the virtual length of a sofa in a living room represented in the virtual 3D representation. The modification unit 640 then modifies the sofa according to the command.

[0100] The VRS 600 may include an animation unit 660 configured to animate a virtual 3D representation of a human or animal. The animation unit 660 may animate the virtual 3D representation of a person to speak or move in the virtual 3D representation. The animation unit 660 may utilize an artificial intelligence chatbot configured to talk with the user in the virtual 3D representation. In the previous example, the animation unit 660 animates the grandmother to move in the living room or talk with the user in the virtual 3D representation via the artificial intelligence chatbot.

[0101] The user can modify the virtual voice or sound of the artificial intelligence chatbot using the modification unit 640. The user can input an audio recording related to the voice or sound of the virtual 3D representation object. For example, the user can input an audio recording of the user's grandmother speaking. The modification unit 640 then renders the virtual sound so that the animation unit 660 can approximate the grandmother's voice via the artificial intelligence chatbot.

[0102] By repeatedly modifying the virtual 3D representation via commands, voice commands, additional images, or voice recordings, the user is able to approximate a memory of one or more objects depicted in the image.

[0103] VRS 600 can transmit the virtual 3D representation to the user device so that the user can view the virtual 3D representation on the display device of the user device. A rendering unit (e.g., a graphics processing unit, a processor, or a combination of both) of the user device can render the virtual 3D representation for display on the display device.

[0104] The VRS 600 may also include a facial recognition unit (e.g., Figure 1A facial recognition unit 140 configured to recognize one or more faces in the first image; and a facial matching unit (e.g., Figure 1 The VRS 600 may also include a metadata processing unit (e.g., Figure 1 The VRS 600 may include a notification unit (e.g., Figure 1 The VRS 600 may include a notification unit 170 configured to notify a first user and a second user associated with the first image and the second image if a face in the first image matches a face in the second image. In this manner, the first user and the second user may view a virtual 3D representation of the first image. The first user and the second user may then each be able to modify the virtual 3D representation by providing commands to the VRS 600.

[0105] Advantageously, if the first user and the second user each input an image of another relative, friend or relationship (e.g., a great-grandparent), this can allow lost relatives, friends or relationships (e.g., old friends, family friends, business partners, etc.) to reconnect. They can each experience the virtual 3D representation of the image and further modify the virtual 3D representation using their image and / or memories associated with the virtual 3D representation.

[0106] refer to Figures 7 to 9 , a flow chart of an exemplary computer-implemented method 700 for generating a virtual 3D representation of an image according to aspects of the present disclosure is shown. Figures 7 to 9 The operations of method 700 are shown in a particular order, but the operations need not all be performed in the specified order, and some operations may be performed in another order. For simplicity, method 700 will be described below in which VRS 600 performs the operations. However, in various aspects, Figures 7 to 9 The operations of may be performed in part by VRS 600, and in part by any other suitable computing device. Such variations are contemplated to be within the scope of the present disclosure.

[0107] Initially, at step 710, the VRS accesses a first image from a database. At step 720, the VRS processes the first image to detect at least one object depicted in the first image. At step 730, the VRS generates a virtual 3D representation of the at least one object depicted in the first image. At step 740, the VRS receives user input including instructions or commands to modify the virtual 3D representation.

[0108] Step 740 may include: Fig. 9 and Fig.10 In step 810, the VRS parses the voice command. In step 820, the VRS identifies features or objects of the virtual 3D representation to be modified as expressed in the parsed voice command. In step 830, the VRS modifies the identified features or objects of the virtual 3D representation. In step 840, the VRS identifies new features or objects to add to the virtual 3D representation expressed in the parsed voice command. In step 850, the VRS modifies the virtual 3D representation to include the new features or objects expressed in the parsed voice command.

[0109] Can be Fig. 9 910, the VRS receives the second image and processes the second image to detect at least one object in the second image depicted in the first image. In step 920, the VRS determines a difference between the objects depicted in the second image and the first image. In step 930, the VRS modifies the virtual 3D representation based on the determined difference between the objects as depicted in the second image and the first image.

[0110] The method 700 may further include: Figure 4 The method 700 may include displaying a virtual 3D representation to a plurality of users. The method 700 may include animating a person or animal in the virtual 3D representation. The method 700 may include enabling a person in the virtual 3D representation to converse with a user via an artificial intelligence chatbot.

[0111] Fig.10 A representative embodiment according to the present disclosure Figure 1 Notification server 100 or Figure 6 A block diagram of a computing device 1000 of a VRS 600 of the present invention is provided. As non-limiting examples, the computing device 1000 may include a server computer, a desktop computer, a laptop computer, a notebook computer, a sub-notebook computer, a netbook computer, a network tablet computer, a set-top computer, a handheld computer, an Internet appliance, a mobile smartphone, a tablet computer, a personal digital assistant, a video game console, an embedded computer, and an unmanned vehicle. Those skilled in the art will recognize that many smartphones are suitable for the system described herein. Suitable tablet computers include tablet computers with brochure, slate, and convertible configurations known to those skilled in the art.

[0112] In some embodiments, computing device 1000 includes an operating system that is configured to execute executable instructions. An operating system is, for example, software including programs and data that manages the hardware of the device and provides services for the execution of application programs. Those skilled in the art will recognize that, as non-limiting examples, suitable server operating systems include FreeBSD, OpenBSD, Linux, Mac OS X Windows and Those skilled in the art will recognize that suitable personal computer operating systems include, by way of non-limiting example, Mac OS and UNIX-like operating systems such as In some embodiments, the operating system is provided by cloud computing. Those skilled in the art will also recognize that, by way of non-limiting example, suitable mobile smartphone operating systems include OS, Research In BlackBerry Windows OS, Windows OS, and

[0113] In some embodiments, the computing device 1000 may include a storage device 1010. The storage device 1010 is one or more physical devices for storing data or programs on a temporary or permanent basis. In some embodiments, the storage device 1010 may be a volatile memory and requires power to maintain the stored information. In some embodiments, the storage device 1010 may be a non-volatile memory and retains the stored information when the computing device 1000 is not powered. In some embodiments, the non-volatile memory includes a flash memory. In some embodiments, the non-volatile memory includes a dynamic random access memory (DRAM). In some embodiments, the non-volatile memory includes a ferroelectric random access memory (FRAM). In some embodiments, the non-volatile memory includes a phase change random access memory (PRAM). In some embodiments, as non-limiting examples, the storage device 1010 includes a CD-ROM, a DVD, a flash memory device, a disk drive, a tape drive, an optical drive, and a cloud computing-based storage device. In some embodiments, the storage device 1010 may be a combination of devices such as those disclosed herein.

[0114] The computing device 1000 further includes a processor 1030, an expansion device 1040, a display 1050, an input device 1060, and a network card 1070. The processor 1030 is the brain of the computing device 1000. The processor 1030 executes instructions to implement the tasks or functions of the program. When the user executes the program, the processor 1030 reads the program stored in the storage device 1010, loads the program onto the RAM, and executes the instructions specified by the program.

[0115] Processor 1030 may include a microprocessor, a central processing unit (CPU), an application specific integrated circuit (ASIC), an arithmetic coprocessor, a graphics processor, or an image processor, each of which is an electronic circuit within a computer that executes the instructions of a computer program by performing basic arithmetic, logic, control, and input / output (I / O) operations specified by the instructions.

[0116] In an embodiment, the expansion device 1040 may include several ports, such as one or more Universal Serial Bus (USB), IEEE 1394 ports, parallel ports, and / or expansion slots, such as Peripheral Component Interconnect (PCI) and PCI express (PCIe). The expansion device 1040 is not limited to this list, but may include other slots or ports that may be used for appropriate purposes. The expansion device 1040 may be used to install hardware or add additional functionality to the computer, which may facilitate the purpose of the computer. For example, a USB port may be used to add additional storage to the computer, and / or IEEE 1394 may be used to receive motion / still image data.

[0117] In some embodiments, the display 1050 can be a cathode ray tube (CRT), a liquid crystal display (LCD), or a light emitting diode (LED). In some embodiments, the display 1050 can be a thin film transistor liquid crystal display (TFT-LCD). In some embodiments, the display 1050 can be an organic light emitting diode (OLED) display. In various embodiments, the OLED display is a passive matrix OLED (PMOLED) or an active matrix OLED (AMOLED) display. In some embodiments, the display 1050 can be a plasma display. In some embodiments, the display 1050 can be a video projector. In some embodiments, the display can be interactive (e.g., with a touch screen or a sensor such as a camera, a 3D sensor, LiDAR, a radar, etc.), which can detect user interactions / gestures / responses, etc.

[0118] In yet other embodiments, display 1050 is a combination of devices such as those disclosed herein.

[0119] The user may input and / or modify data via input device 1060, which may include a keyboard, a mouse, or any other device with which the user may input data. Display 1050 displays data on a screen of display 1050. Display 1050 may be a touch screen so that display 1050 may be used as an input device.

[0120] The network card 1070 is used to communicate with other computing devices wirelessly or via a wired connection. Through the network card 1070, a digital image can be received from a user device, and a notification of a facial match can be sent to the user device.

[0121] The computing device 1000 may further include a graphics processing unit (GPU) 1080, which typically accelerates graphics rendering. Since the GPU 1080 is capable of processing many pieces of data simultaneously and in parallel, the GPU 1080 may also be used for machine learning systems and AI algorithms. The GPU 1080 may collaborate with the processor 1030 to enhance machine learning or AI algorithms for recognizing faces, objects, and backgrounds in images and generating and modifying virtual 3D representations.

[0122] Any method, program, algorithm or code described herein can be converted into a programming language or computer program, or expressed in a programming language or computer program. As used herein, the terms "programming language" and "computer program" include any language for specifying instructions to a computer, and include (but are not limited to) the following languages ​​and their derivatives: assembly language, Basic, batch files, BCPL, C, C++, C#, Delphi, Fortran, Java, JavaScript, machine code, operating system command language, Pascal, Perl, PL1, scripting languages, Visual Basic, the metalanguage of the program itself, and all first, second, third, fourth, fifth or further generations of computer languages. Also included are databases, other data models and any other metalanguages. There is no difference between the languages ​​that are interpreted or compiled. There is no difference between the compiled version and the source version of the program. Therefore, referring to a program, in which a programming language can exist in more than one state (such as source, compiled, object or linked), is referring to any and all such states. Mentioning a program can cover the actual instructions and / or the intention of those instructions.

[0123] In one or more instances, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or codes on a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a non-transitory computer-readable medium, which corresponds to a tangible medium, such as a data storage medium (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store the desired program code in the form of instructions or data structures and can be accessed by a computer).

[0124] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), GPUs, or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any other physical structure suitable for implementing the described techniques. Likewise, the techniques may be fully implemented in one or more circuits or logic elements.

[0125] It should be understood that the various aspects disclosed herein may be combined in combinations different from those specifically presented in the detailed description and the drawings. It should also be understood that, depending on the instance, certain actions or events of any process or method described herein may be performed in a different order, may be added, combined, or omitted entirely (e.g., all described actions or events may not be necessary to perform the technique). In addition, although certain aspects of the present disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of the present disclosure may be performed by a combination of units or modules associated with, for example, the above-described servers and computing devices.

Claims

1. A notification system that notifies a user when a match exists between images, the notification system comprising: a network interface configured to receive images from a plurality of user devices including a first user device and a second user device; A memory storing a database and instructions; A processor that executes the instructions to: an image processing unit, the image processing unit being configured to process the received image; a facial recognition unit configured to recognize one or more faces in each received image; a face matching unit configured to check whether a face in a first image from the first user device matches a face in a second image from the second user device; a metadata processing unit configured to process metadata in each received image; as well as A notification unit configured to notify the first user device and the second user device in a case where the face in the first image matches the face in the second image.

2. The notification system of claim 1, wherein the processor further executes the instructions to: A relation unit is configured to associate a face with corresponding metadata.

3. The notification system according to claim 1, wherein in a case where the received image does not have metadata therein, the metadata processing unit is further configured to send a request for metadata to a corresponding user device and receive metadata from the corresponding user device. 4 . The notification system according to claim 1 , wherein the facial recognition unit is further configured to group the same faces in the received images from each user device and set a representative face of the same faces.

5. The notification system of claim 4, wherein the facial matching unit is further configured to calculate a difference between a first representative face in the received image from the first user device and a second representative face in the received image from the second user device.

6. The notification system of claim 5, wherein in a case where the difference is between a first threshold and a second threshold, the facial matching unit is further configured to calculate a similarity score between the metadata of the first representative face and the second representative face. 7 . The notification system according to claim 6 , wherein in a case where the similarity score is greater than or equal to a third threshold, the notification unit notifies the first user device and the second user device.

8. The notification system of claim 1, wherein the metadata includes personal information, geographic information, and image information.

9. The notification system according to claim 1, wherein the database includes user information of each user. 10 . The notification system according to claim 9 , wherein the first user device and the second user device are notified based on the user information.

11. A notification method for notifying a user when there is a match between images, the notification method comprising: receiving an image from a plurality of user devices including a first user device and a second user device; processing the received image; identifying one or more faces in each processed image; checking whether a face in a first image from the first user device matches a face in a second image from the second user device; Processing metadata in each received image; as well as The first user device and the second user device are notified in a case where the face in the first image matches the face in the second image.

12. The notification method according to claim 11, further comprising: Associate faces with corresponding metadata.

13. The notification method according to claim 11, further comprising: In a case where the received image has no metadata therein, a request for metadata is sent to a corresponding user device, and metadata is received from the corresponding user device. 14 . The notification method according to claim 11 , wherein the face match is checked by grouping the same faces in the received images from each user device and setting a representative face of the same faces.

15. The notification method of claim 14, wherein the facial match is further checked by calculating a difference between a first representative face in the received image from the first user device and a second representative face in the received image from the second user device.

16. The notification method according to claim 15, wherein in a case where the difference is between a first threshold and a second threshold, the facial match is checked by calculating a similarity score between the metadata of the first representative face and the second representative face. 17 . The notification method according to claim 16 , wherein the first user equipment and the second user equipment are notified in a case where the similarity score is greater than or equal to a third threshold.

18. The notification method according to claim 11, further comprising: A database is stored, the database including user information for each user. The notification method according to claim 18 , wherein the first user equipment and the second user equipment are notified based on the user information.

20. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform a notification method for notifying a user when a match exists between images, the notification method comprising: receiving an image from a plurality of user devices including a first user device and a second user device; processing the received image; identifying one or more faces in each processed image; checking whether a face in a first image from the first user device matches a face in a second image from the second user device; Processing metadata in each received image; as well as The first user device and the second user device are notified in a case where the face in the first image matches the face in the second image.

21. A virtual reality system for creating a virtual three-dimensional (3D) representation of an image, the virtual reality system comprising: processor; and a memory coupled to the processor and storing one or more instructions that, when executed by the processor, cause the virtual reality system to function as: an access unit configured to access a first image from a database, the first image being input into the database by a user; a virtual 3D generation unit configured to perform image processing on the first image to detect at least one object depicted in the first image and to generate a virtual 3D representation of the at least one object; a receiving unit configured to receive a command for modification via a user interface; as well as A modification unit is configured to modify the virtual 3D representation based on the command.

22. The virtual reality system of claim 21, wherein the virtual 3D generation unit is further configured to generate a 3D mesh of the at least one object.

23. The virtual reality system of claim 22, wherein the virtual 3D generation unit employs an artificial intelligence module to detect at least one of edges, nodes, and faces of the at least one object to generate a plurality of features of the 3D mesh.

24. The virtual reality system of claim 23, wherein the commands comprise audio commands.

25. The virtual reality system of claim 24, wherein the instructions, when executed by the processor, further cause the virtual reality system to function as: a language processing unit configured to parse the audio command to identify a feature of the plurality of features of the virtual 3D representation expressed in the audio command, and Wherein the modification unit modifies the identified feature of the plurality of features of the virtual 3D representation according to the audio command.

26. A virtual reality system according to claim 25, wherein the audio command is used to identify a new object or feature, and The modification unit adds the new feature or object expressed in the audio command to the virtual 3D representation.

27. The virtual reality system of claim 25, wherein the instructions, when executed by the processor, further cause the virtual reality system to function as: An animation unit configured to animate a person to speak or move in the virtual 3D representation when the at least one object includes the person.

28. The virtual reality system of claim 21, wherein the access unit is further configured to access a second image from the database, the second image being input into the database by another user and being related to the first image, wherein the virtual 3D generation unit is further configured to process the second image to detect at least one object depicted in the second image and to determine at least one difference depicted in the second image between the at least one object in the first image and the second image, and Wherein the modification unit is further configured to modify the virtual 3D representation of the at least one object based on the difference.

29. A method for creating a virtual three-dimensional (3D) representation of an image, the method comprising: accessing a first image from a database, the first image being input into the database by a user; processing the first image to detect at least one object depicted in the first image; generating a virtual 3D representation of the at least one object; receiving, via a user interface, a command to modify the virtual 3D representation; as well as The virtual 3D representation is modified based on the command.

30. The method of claim 29, wherein generating the 3D virtual representation comprises generating a 3D mesh of the at least one object.

31. The method of claim 30, wherein generating a 3D mesh of the at least one object comprises detecting at least one of edges, nodes, or faces of the at least one object to generate a plurality of features of the 3D mesh using artificial intelligence.

32. The method of claim 31 , wherein the command comprises an audio command, Wherein the method further comprises: parsing the audio command to identify one of the plurality of features of the virtual 3D representation, or to identify a new feature to be added to the virtual 3D representation expressed in the audio command; and Wherein modifying the virtual 3D representation comprises modifying the identified feature of the plurality of features of the virtual 3D representation or adding the new feature to the virtual 3D representation according to the audio command.

33. The method of claim 32, further comprising: When the at least one object includes a person, the person is animated to speak or move in the virtual 3D representation.

34. The method of claim 32, further comprising: accessing a second image from the database, the second image being input into the database by the user and being related to the first image; as well as processing the second image to detect at least one object depicted in the first image and determining a difference between the at least one object in the first image and the second image, Wherein the virtual 3D representation of the at least one object is modified based on the difference.

35. A non-transitory computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform a method for creating a virtual three-dimensional (3D) representation of an image, the method comprising: accessing a first image from a database, the first image being input into the database by a user; processing the first image to detect at least one object depicted in the first image; generating a virtual 3D representation of the at least one object; receiving, via a user interface, a command to modify the virtual 3D representation; as well as The virtual 3D representation is modified based on the command.

Images