A method including the generation of augmented reality visuals associated with a personalized object.

Abstract

A method including the generation of augmented reality visuals associated with a personalized object: The disclosed method (10) includes acquiring an image stream (ET6), identifying a carrier object in the image stream (ET7), identifying a personalized object in the image stream (ET7), and providing access to an image stream transformation model associated with this personalized object, such as an image stream transformation model that generates augmented reality visuals in the image stream, where the identified carrier object and the personalized object correspond to a pair of associated objects (ET8).

Description

Technical Field

[0001] Embodiments and implementations of the present invention relate to a data processing method, and more particularly to a method including the generation of augmented reality visuals in an image stream.

Background Art

[0002] It is necessary to make good use of virtual aesthetic decorations (such as a color filter for a photograph or an augmented reality object inserted into a video image stream) that are unique and exclusive to an individual user's virtual identity.

[0003] The expression "augmented reality" is understood to have its conventional meaning ascribed thereto. That is, briefly speaking, it is a technology that enables virtual objects such as 2D or 3D images to be embedded in a sequence of real-world images in a realistic manner, usually calculated in real time by a computer system.

[0004] The expression "virtual identity" is understood to mean, for example, content selected by a user for sharing with an audience or community via social media, that is, online applications that enable the creation and exchange of user-generated content, based on the concepts and technologies of the Internet, for example.

[0005] Specifically, due to the continuous improvement of the performance of augmented reality technology, virtual decorations, that is, in a broad sense, virtual decorative objects such as virtual gems, are used to generate sufficiently attractive results for decorating and personalizing a user's virtual identity.

Summary of the Invention

Means for Solving the Problems

[0006] Some of the embodiments and implementations defined below provide a unique and exclusive generative experience, personalized by augmented reality aesthetics for the user's virtual identity.

[0007] According to one embodiment, a method is provided in this regard, and the method is - Acquisition of the image stream, - Identification of carrier objects in the image stream, - Identification of personalized objects in the image stream, - When the identified carrier object and personalization object correspond to a pair of associated objects, access to the image stream transformation model associated with the personalization object is provided, Includes.

[0008] According to one implementation, providing access to an image stream transformation model includes the execution of the transformation model, which generates an augmented reality visual in the image stream, and the display of the image stream in which the augmented reality visual is contained.

[0009] In other implementations, providing access to the image stream transformation model is: Loading the image stream transformation model into the memory of the device that acquired the image stream, or For example, unlocking the use of the image stream transformation model in a way associated with the user profile, It can include...

[0010] For example, image streams can be acquired in real time by cameras, and image streams containing augmented reality visuals can be displayed in real time, among other things, by LCD, LED, or OLED screens (these well-known acronyms refer to liquid crystal displays, light-emitting diodes, and organic light-emitting diodes, respectively).

[0011] For example, the step of identifying a carrier object can be performed using conventional object recognition techniques, such as those implemented by an object recognition module belonging to a photo filtering engine as described in U.S. Patent Application Publication No. 2016 / 0203586A1 (dated July 14, 2016).

[0012] For example, the step of identifying a personalized object can be performed using one of the following: a machine learning technique called "SnapML," which is an extension of "Lens Studio," a design tool distributed by Snap Inc. and also known as the social media application "Snapchat"; another technique such as image tracking; or the input of a code associated with the purchase of the personalized object.

[0013] For example, augmented reality visuals can be generated using conventional technologies, such as the technology described in Korean Patent Application Publication No. 20230038604A (dated March 20, 2023), which enables moving virtual objects relative to a real environment and manipulating them in other ways, or the technology described in Korean Patent Application Publication No. 20190089091A (dated July 29, 2019), which enables generating a 3D model of an object of interest and manipulating it based on changes in its position within the field of view of an image capture device.

[0014] According to one implementation, the augmented reality visual includes at least one portion superimposed on the personalized object within the image stream in order to transform the aesthetics of the personalized object.

[0015] In one implementation, the augmented reality visual is configured to dynamically display, in an image stream, virtual art designs on a carrier object, such as animations of artwork of several artistic makeup looks, and the designs are emitted from the personalized object.

[0016] In one implementation, the identification of a carrier object includes the identification of a marker unique to a given type of carrier object, such as a face marker if the carrier object is a face, and the identification of a personalization object includes the detection of the location of the personalization object within a suitable region of the carrier object, determined by the location of this unique marker.

[0017] According to one implementation, the preliminary step includes short-range wireless communication of preliminary information by a contactless transponder contained in the personalized object, and the communicated preliminary information can determine at least one pair of associated objects.

[0018] The term "near-field wireless communication" is generally understood to refer to the technology defined in the ISO 14443 standard, which is usually abbreviated as NFC (Near-Field Communication). The term "contactless transponder" is generally understood to refer to a tag-type NFC chip that can communicate with a reader using a power supply method that involves inductive coupling to the electromagnetic field generated by the reader.

[0019] According to one implementation, the communicated preliminary information can further instruct, after the preliminary step, the operation of the following steps of the method, namely, in particular, the acquisition of an image stream, the identification of an object, the generation and display of an augmented reality visual.

[0020] Therefore, in this implementation, the augmented reality experience associated with the object is triggered by contactless NFC communication, which typically occurs when an NFC reader approaches a personalized object. This allows for an immediate experience, providing the user with a pleasant surprise and a sense of exclusivity.

[0021] According to one implementation, the personalization object is a tag designed to adhere to human skin, such as a body jewel.

[0022] According to one implementation form, the carrier object is a human face.

[0023] A computer program product is also provided, and when the program is executed by a computer such as a smartphone, the computer program product includes instructions for causing the computer to implement the method as defined above.

[0024] A computer-readable recording medium is also provided, and when they are executed by a computer such as a smartphone, the computer-readable recording medium includes instructions for causing the computer to implement the method as defined above.

[0025] According to another aspect, a personalized object is also defined, and the personalized object is designed to implement the method as defined above.

[0026] According to another aspect, a system is also defined, and the system includes a computer such as a smartphone and a personalized object as defined above configured to interact to implement the method as defined above, for example.

[0027] For example, since the personalized object is a tag designed to be adhesively bonded to human skin and intended to be worn on the face, the personalized object provides access to an image stream conversion model associated with the personalized object, such as a conversion model that generates an augmented reality visual in the acquired image stream when the identified carrier object and the personalized object correspond to a pair of associated objects including the carrier object consisting of a human face and the personalized object consisting of this tag.

[0028] For example, since the tag is intended to be attached near one eye, the identification of the personalized object is suitable for a method that includes detecting the location of the personalized object in a suitable area of ​​the carrier object determined by the facial marker position, i.e., in the area near the eye.

[0029] For example, the tag includes a contactless transponder designed for short-range wireless communication of preliminary information, which can determine at least one pair of associated objects and can instruct the implementation of the method, in particular the operation of acquiring an image stream and the operation of the identification of the objects.

[0030] In other words, embodiments and implementations of the present invention provide a body jewel having an NFC chip that enables the wearer to trigger a personalized experience using the front or rear camera on their smartphone, for example, on a social media application. This unique and exclusive experience is accessible only to the person who owns the tag and is wearing it on their face. Thanks to the magical feeling of augmented reality, the user can push back the limits of the physical world and enter a surprising and astonishing world of beauty that is intimately connected to them.

[0031] The experience is reserved for the tag owner, particularly for the identification step of the personalized object.

[0032] The tag itself can open up the world of body jewelry, for example, an exclusive experience associated with that trademark.

[0033] Other advantages and features of the present invention will become apparent from a detailed description of embodiments and implementations, which are not limited in any way, as well as from the accompanying drawings below. [Brief explanation of the drawing]

[0034] [Figure 1] Embodiments and implementations of the present invention are illustrated. [Figure 2] Embodiments and implementations of the present invention are illustrated. [Figure 3] Embodiments and implementations of the present invention are illustrated. [Figure 4] Embodiments and implementations of the present invention are illustrated. [Figure 5] Embodiments and implementations of the present invention are illustrated. [Figure 6] Embodiments and implementations of the present invention are illustrated. [Modes for carrying out the invention]

[0035] Figure 1 illustrates an example of a live experience for a female or male user utilizing this unique and exclusive augmented reality embellishment.

[0036] In step ET1, the user acquires a personalized object, for example, by purchasing it. The personalized object is a decorative accessory, such as an adhesive tag intended to be attached to a carrier object.

[0037] Therefore, in the first example, the personalized object could be an adhesive tag intended to decorate a carrier object such as a vehicle.

[0038] Figure 2 illustrates a preferred example of a Personalized Object (BIJ) that is a cosmetic accessory intended to be worn on a part of the user's body (VIS), such as a body jewel, i.e., a tag designed to adhere to human skin.

[0039] Body jewels (BIJ) can, in practice, be intended to be attached on the one hand to a specific part of the body, for example, on the face (VIS), or in some cases on the wrist or shoulder, as well as to an appropriate area (REG) of the body, for example, on the contour of the right eye on the face (VIS).

[0040] For example, French Patent Application Publication No. 2966704A1 (dated May 4, 2012), French Patent Application Publication No. 3007955A1 (dated January 9, 2015), and International Publication No. 2013164521A1 (dated November 7, 2013) describe body jewel technology, body jewels referred to as body ornaments, and processes for manufacturing such items.

[0041] Referring again to Figure 1, in step ET2 of experience 10, the user attaches the body jewel BIJ to an appropriate area REG on their body, for example, near their right eye.

[0042] To obtain a favorable immediate user experience 10 that brings a pleasant surprise and a sense of exclusivity, the augmented reality experience associated with the object is triggered in the preliminary step ET4 by contactless NFC communication that normally occurs when the NFC reader approaches the personalized object in the read step ET3.

[0043] In this regard, the BIJ tag includes a contactless transponder suitable for near-field communication (NFC) for preliminary information.

[0044] An NFC reader can typically be a computer, such as a smartphone, that is configured for this functionality.

[0045] On the other hand, the preliminary information can determine at least one pair of BIJ-VIS associated objects to be used at the end of identification step ET7, which is described below.

[0046] On the other hand, the preliminary information can instruct the subsequent steps ET6, ET7, and ET8 of the experience to be activated after the preliminary step ET4.

[0047] In this regard, the preliminary information may include, for example, a request ET5 to open a social media application, or an invitation to download one if the social media application is not already installed on the smartphone acting as an NFC reader.

[0048] A social media application, such as the one named "Snapchat," is opened in a commercialized private features mode, as described in U.S. Patent Application Publication No. 2016203586A1 (dated July 14, 2016), with respect to accessing personal photo filters proposed by third-party vendors, in cooperation with the aforementioned social media application "Snapchat."

[0049] Next, in step ET6, the user records a person (or a carrier object possessing a Personalized Object BIJ from the private function of a social media application) within the field of view of a photographic sensor, such as the front or rear camera of the smartphone. Thus, in step ET6, a continuous image stream is acquired by the camera in real time.

[0050] Firstly, in detection step ET7, the commercialized private functionality of the social media application is configured to identify the corresponding carrier object VIS in the acquired image stream, i.e., to detect the presence of a face VIS in the camera's field of view in this example.

[0051] For example, carrier objects can be identified using conventional object recognition techniques, such as being implemented by an object recognition module belonging to a photo filtering engine described in, for example, U.S. Patent Application Publication No. 2016 / 0203586A1.

[0052] In this regard, the identification of a carrier object VIS may include the identification of markers specific to a given type of carrier object, such as face markers (traditionally the position of the pupil, cheekbone, nose, mouth, etc.), if the carrier object is a face VIS.

[0053] Secondly, in detection step ET7, the commercialized private functionality of the social media application is configured to identify the corresponding personalized object BIJ in the acquired image stream, i.e., to detect the presence of a body jewel BIJ in the camera's field of view.

[0054] Identifying a personalized object (BIJ) may involve detecting that the personalized object's location is actually situated within the appropriate region (REG) of the carrier object (VIS). This location can be determined relative to the location of the aforementioned unique marker.

[0055] For example, the identification step ET7 of the personalized object BIJ can be performed using a machine learning technique called "SnapML," which is an extension of the design tool "Lens Studio," also known as the social media application "Snapchat," distributed by Snap Inc.; another technique such as image tracking; or by entering a code associated with the purchase of the personalized object.

[0056] If the identified carrier object and personalization object correspond to a pair of associated objects, for example, communicated with preliminary information ET4, the experience may favorably include, in the image stream, the generation of an augmented reality visual associated with this personalization object BIJ ET8.

[0057] For example, augmented reality visuals can be generated using technologies such as those described in Korean Patent Application Publication No. 20230038604A (dated March 20, 2023), which enable the movement and manipulation of virtual objects relative to the real environment, or technologies described in Korean Patent Application Publication No. 20190089091A (dated July 29, 2019), which enable the generation of a 3D model of an object of interest and its manipulation based on changes in its position within the field of view of an image capture device.

[0058] For example, a user can choose to record a sequence of image streams containing augmented reality visuals, which are displayed in real time on the user's smartphone, typically on an LCD, LED, or OLED screen (these well-known acronyms refer to liquid crystal displays, light-emitting diodes, and organic light-emitting diodes, respectively).

[0059] Alternatively, step ET8 includes providing access to the image stream transformation model associated with the detected personalized objects, if step ET7 successfully detected a pair of objects.

[0060] The phrase "providing access to the transformation model" is understood to mean not only the "live" applications of augmented reality visuals, such as those described above and later described with reference to Figures 3-6, but also other possibilities of access, such as loading the transformation model into the memory of the device that acquired the image stream, or unlocking the model for subsequent use.

[0061] Therefore, loading an image stream transformation model, for example, may include downloading program code that implements the model from a web server via an internet connection such as Wi-Fi or 4G, 5G, or LTE, and adding the downloaded program code to a library of transformation models contained in the smartphone's memory.

[0062] Furthermore, unlocking the use of image stream transformation models, for example, can be associated with user profiles, taking into account the possibility of using them on various devices to which the profile is connected. Transformation models may already be included in a library of transformation models, or they may be accessed online via streaming or a one-time download, or they may even be loaded into local memory as described above.

[0063] Furthermore, in the detection step (step ET9), if the commercialized private functionality of the social media application does not identify the carrier object VIS in the image stream, or does not identify the personalization object BIJ in the image stream, or even if the identified carrier object and personalization object do not correspond to the associated VIS-BIJ, or optionally, if the identified personalization object BIJ is not located in the appropriate region REG of the identified carrier object VIS, then the augmented reality visual will not be generated or displayed in the image stream (step ET10).

[0064] Therefore, the user can witness an exclusive experience 10, because this experience is reserved for its owner who has chosen to wear the body jewel (or personalization object) on the appropriate area (or carrier object) of their body.

[0065] The exclusive experience can be associated with BodyJewel's trademark aesthetic world, and thanks to the magical sensation of augmented reality, users can push back the limits of the physical world and enter a surprising and astonishing world of beauty that is intimately connected to them.

[0066] Furthermore, in step ET8, where the augmented reality effect is displayed, the effect can be randomly selected from a given number (e.g., 5) different themes. Since the user does not know which augmented reality effect they will receive, a sense of scarcity and excitement is created.

[0067] Refer to Figures 3–6 for further details. These figures illustrate four examples of themes for augmented reality visuals that generate aesthetic ornaments designed to adorn a user's virtual identity. All of these examples correspond to the case where the personalization object is a body jewel BIJ designed to be positioned around the contour of one eye.

[0068] Figure 3 shows examples a, b, c, d, e, f of augmented reality visuals 30 that include at least one portion superimposed on a personalized object BIJ to transform the aesthetics of the personalized object BIJ.

[0069] Therefore, examples of augmented reality visuals 30 can be used to develop and extend the shape of a body jewel in space, for example, to represent a world corresponding to a trademark, for example, the appearance of Japanese calligraphy.

[0070] Augmented reality visuals 30 can be advantageously animated to track the user's facial movements in the image stream in real time, in a way that is itself a known conventional method.

[0071] Figure 4 shows examples a, b, c, d, e, and f of augmented reality visuals 40 that dynamically display animations of virtual art designs on the user's face in an image stream, such as animations that may extend from body jewels.

[0072] In these examples 40, explosions of color of various shapes and energies may erupt from the center of the eye or from body jewels.

[0073] Figure 5 shows examples a, b, c, and d of augmented reality visuals 50 that dynamically display animations of virtual art designs on the user's face in an image stream, such as animations that may extend from body jewels.

[0074] In these 50 examples, a drawing with the visual texture of a watercolor is generated on the face of the user wearing the body jewel. This drawing could represent, for example, a trademark world-renowned animal, such as a carp (an ornamental fish symbolizing Japanese culture), which may appear to swim in from behind the jewel.

[0075] Figure 6 shows steps a, b, c, d, e, and f of the storyboard for an augmented reality visual 60, which dynamically displays an animation of a virtual art design on the user's face, extending from a body jewel, in an image stream.

[0076] In this storyboard 60, scene a corresponds to the user's actual face wearing the body jewel. Scene b corresponds to a tear generated in the corner of the user's eye by augmented reality. Scene c has the effect of the tear spreading and drawing the shape of a camellia tree with liquid ink flowing through it. Scene d has the ink and tear dry and the flower begins to bloom. Scene e has the flower grown to its final size. In scene f, advantageously, in response to the user's movement, the largest of the flowers could burst open and drop its petals. For example, this movement could be a wink, a nod, or something else.

[0077] In all the examples described with reference to Figures 3 to 6, it should be noted that users can interact with the augmented reality visuals and further animate them by using facial expressions, such as eye, eyebrow, and mouth movements.

[0078] This interaction brings the body jewels on the user's face to life by magically animating them according to the artistic direction chosen by the brand and its corresponding trend board. Possible animations could include, for example, swirling particles, streaks of light, flying butterflies, randomly shimmering body jewel colors, colored marks breaking down into particles that form calligraphic patterns, and so on, to create a sense of personalization and dynamism. [Explanation of symbols]

[0079] 10 ways 30, 40, 50, 60 Augmented Reality Visuals

Claims

1. - Acquisition of the image stream (ET6), - Identification of the carrier object (VIS) in the aforementioned image stream (ET7), - Identification of the Personalized Object (BIJ) in the aforementioned image stream (ET7), - If the identified carrier object and the identified personalization object correspond to a pair of associated objects, access to the image stream transformation model associated with the personalization object is provided (ET8), Method (10), including the following.

2. The provision of access to the image stream transformation model (ET8) comprises the execution of the transformation model, which generates augmented reality (AR) visuals in the image stream (ET8), and the display of the image stream in which the augmented reality visuals (30, 40, 50, 60) are contained. The method according to claim 1, including the method described in claim 1.

3. The method according to claim 2, wherein the augmented reality visual (AR) includes at least one portion superimposed on the personalized object (BIJ) in the image stream so as to transform the aesthetics of the personalized object (30).

4. The method according to claim 2 or 3, wherein the augmented reality visual (AR) is a virtual art design on the carrier object, and is configured to dynamically display animations of the virtual art design emitted from the personalized objects (40, 50, 60) in the image stream.

5. - The identification of the carrier object (VIS) includes the identification of a marker specific to a given type of carrier object, such as a face marker if the carrier object is a face. - The identification of the personalized object (BIJ) includes detecting the location of the personalized object in a suitable region (REG) of the carrier object (VIS) determined by the location of the unique marker, The method according to any one of claims 1 to 4.

6. The method according to any one of claims 1 to 5, wherein the preliminary step (ET4) includes short-range wireless communication of preliminary information by a transponder contained in the personalized object (BIJ), the communicated information can determine at least one pair of associated objects (BIJ, VIS).

7. The method according to claim 6, wherein the communicated preliminary information can further instruct the operation (ET5) of subsequent steps of the method, which begin after the preliminary step (ET4) with acquiring the image stream (ET6).

8. The method according to any one of claims 1 to 7, wherein the personalized object (BIJ) is a tag designed to adhere to human skin.

9. The method according to any one of claims 1 to 8, wherein the carrier object (VIS) is a human face.

10. A computer program product comprising, when the program is executed by a computer, an instruction causing the computer to implement the method according to any one of claims 1 to 9.

11. A computer-readable recording medium, which, when executed by a computer, includes an instruction causing the computer to implement the method described in any one of claims 1 to 9.

12. A personalized object (BIJ) designed to implement the method described in any one of claims 1 to 9.

13. A system comprising a computer and a personalized object (BIJ) configured to interact with it to implement the method described in any one of claims 1 to 9.

Images