Method for communicating data from a virtual fitting system for an accessory using a digitally represented living being

Abstract

The present invention relates to a method and device for communicating data from a virtual fitting system (1) for at least one accessory by a digitally represented living being based on virtual reality, i.e., a system that generates a digital result combining a three-dimensional (3D) digital representation of at least one part of a living being (10) and a 3D digital representation of an accessory. Figure 1 for the abstract

Description

Title of the invention: Method for communicating data from a virtual fitting system for an accessory by a digitally re-presented living being Technical field

[0001] The present invention relates to methods and systems for communicating data from a system for virtual fitting of at least one accessory by a digitally represented living being. Technological background

[0002] Applications or services are known that allow a person to try on an accessory virtually. These applications or services are based on augmented reality, that is, for example, on a combination of two-dimensional (2D) digital images captured of the person that are combined with a three-dimensional (3D) rendering of an accessory. Note that augmented reality can also be obtained by a combination of a volumetric image and a rendering of a 3D scene.

[0003] The combination of 2D and 3D image data of applications based on augmented reality limits the display of the image resulting from the combination of this data to a single point of view or at best to different points of view whose view axes are very close to each other.

[0004] Current virtual fitting applications or services thus allow a person to form an opinion concerning the purchase of this accessory by viewing on a screen an image formed by the combination of 2D images of this person and the accessory.

[0005] These applications or services based on augmented reality are usually limited to rendering images resulting from a front view of a user. Indeed, a capture of image data representing a front view of this user is generally made because the user usually looks at the camera which performs this image data capture. The image resulting from the combination of this captured image data and data representing an accessory is then a 2D image of this user representing a front view of the user. Applications or services based on augmented reality are therefore suitable when the accessory in question is worn on the person's face, for example makeup.On the other hand, when the accessory is worn on one side of the person, for example earrings, these applications or services are not well suited because they only offer 2D digital representations while the person may want to have front views. and aside, for example, before deciding on its purchase.

[0006] One of the problems with these virtual fitting applications or services based on augmented reality is therefore that the resulting images do not allow the person to obtain a visualization of an accessory that they would wear if they purchased it from different points of view, for example from the front, from the side or from a three-quarter view. In addition, these virtual fitting applications or services based on augmented reality do not allow the digital representations of the person and the accessory to be contextualized at the discretion of the person viewing on a screen the visual result provided by these applications or services, whereas it would be advantageous in terms of sales if this person could position this visual result in different contexts of brightness or ambiance and / or provide context and different environments to highlight the projection of oneself and the object, for example.

[0007] Summary of the present invention

[0008] An object of the present invention is to solve at least one of the drawbacks of the technological background.

[0009] According to a first aspect, the present invention relates to a method for communicating data from a system for virtual fitting of at least one accessory by a digitally represented living being based on virtual reality, i.e. a system which generates a digital result combining a three-dimensional digital representation of at least one part of the living being (person or animal) and a 3D digital representation of an accessory.

[0010] The method comprises the following steps: - capture, by a first device, of image data representative of at least part of the living being from different points of view; - transmission, by the first device and to a second device implementing at least in part a blockchain, of first data representative of a calculation and reconstruction request to obtain second data representative of a view of a three-dimensional digital twin carrying said at least one accessory according to a point of view, the digital twin comprising mesh data representative of a three-dimensional mesh structure represented, in a three-dimensional geometric space, by a set of vertices of polygonal shapes forming a structure, each polygonal shape sharing at least one side with another polygonal shape, the three-dimensional digital twin further comprising texture data representing a texture of the surfaces of the three-dimensional mesh structure,the first data comprising the captured image data and anonymization data enabling a link to be created between the captured image data and an owner of the captured image data while maintaining the anonymity of the owner; , - transmission to the second device of accessory data representative of a three-dimensional representation of said at least one accessory in the three-dimensional geometric space; - encoding, by the second device, the captured image data and the query anonymization data in the blockchain; - calculating, by the second device, the mesh data by modifying mesh data of an initial digital twin according to morphological information extracted from the captured image data, and texture data of the digital twin obtained from texture data extracted from the captured image data; - calculation, by the second device, of the second data representative of a view according to the point of view of the digital twin carrying said at least one accessory from the mesh data and the texture data of the digital twin and the accessory data; - transmission of the second data by the second device to a third device; and - calculation, by the third device, of a rendering of the view of the digital twin carrying said at least one accessory according to the point of view from the second data.

[0011] According to a variant, the captured image data which correspond to a captured image must verify at least one quality criterion in order to be retained.

[0012] According to a variant, the first data are authenticated by the blockchain before being transmitted.

[0013] According to a variant, the captured image data, the anonymization data and a link data allowing a connection between the anonymization data and the captured image data are stored in separate memories and / or registers.

[0014] According to a variant, the accessory data is transmitted by the second or third device.

[0015] According to a variant, the request also comprises view configuration data representative of a point of view of the digital twin carrying said at least one accessory and / or of configuration of a view context of the digital twin carrying said at least one accessory.

[0016] According to a variant, the captured image data comprises information representative of image contents and / or descriptive information of geometry of said at least one part of the living being in the three-dimensional geometric space.

[0017] The descriptive geometry information allows the appropriation of the space of said at least one part of the living being.

[0018] According to a variant, the captured image data corresponds to an image or to several images.

[0019] According to a variant, the first data is encoded in the blockchain in the form of a non-fungible token.

[0020] According to a variant, the mesh data and the texture data of the digital twin are calculated by CPU (from the English “Central Processing Unit”) and / or in GPU (from the English “Graphics Processing unit”) and / or in quantum computing.

[0021] According to a second aspect, the present invention relates to a system for virtual fitting of at least one accessory by a digitally represented living being comprising: - a first device comprising communication means, at least one image data capture means and at least one processor provided to execute instructions for implementing at least one step of the method according to the first aspect of the present invention; - a second device comprising communication means, means for implementing at least part of a blockchain and at least one processor provided for executing instructions for implementing at least one step of the method according to the first aspect of the present invention; - a third device comprising communication means, a screen and at least one processor provided for executing instructions for the implementation of at least one step of the method according to the first aspect of the present invention.

[0022] According to a variant, the first device and the third device are the same device.

[0023] According to a variant, the first device (101) and the third device (103) are: - the same telephone; or - the same pair of glasses equipped with at least one camera and said screen; or - the same pair of ocular contact lenses equipped with at least one camera and said screen then adaptable to a user's view.

[0024] According to a third aspect, the present invention relates to a computer program which comprises instructions adapted for executing the steps of the method according to the first aspect of the present invention, in particular when the computer program is executed by at least one processor.

[0025] Such a computer program may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in a partially compiled form, or in any other desirable form.

[0026] According to a fourth aspect, the present invention relates to a computer-readable recording medium on which a program is recorded. computer system comprising instructions for carrying out the steps of the method according to the first aspect of the present invention.

[0027] On the one hand, the recording medium may be any entity or device capable of storing the program. For example, the medium may comprise a storage means, such as a ROM memory, a RAM memory, a CD-ROM or a microelectronic circuit type ROM memory, or a magnetic recording means or a hard disk.

[0028] Furthermore, this recording medium may also be a transmissible medium such as an electrical or optical signal, such a signal being able to be conveyed via an electrical or optical cable, by conventional or hertzian radio or by self-directed laser beam or by other means. The computer program according to the present invention may in particular be downloaded from an Internet-type network.

[0029] Alternatively, the recording medium may be an integrated circuit in which the computer program is incorporated, the integrated circuit being adapted to perform or to be used in performing the method in question. Brief description of the figures

[0030] Other characteristics and advantages of the present invention will emerge from the description of the particular and non-limiting exemplary embodiments of the present invention below, with reference to the appended figures 1 to 8, in which:

[0031] [Fig-1] schematically illustrates a data communication environment of a system for virtual fitting of at least one accessory by a digitally represented living being, according to a particular and non-limiting exemplary embodiment of the present invention;

[0032] [Fig.2] illustrates a flowchart of the different steps of a method for communicating data from a virtual fitting system for at least one accessory by a digitally represented living being, according to a particular and non-limiting example of the present invention;

[0033] [Fig.3] schematically illustrates an example of a human-machine interface which is displayed on a screen of the telephone according to a particular and non-limiting example of the present invention;

[0034] [Fig.4] schematically illustrates a view of a digital twin, according to a particular and non-limiting exemplary embodiment of the present invention.

[0035] [Fig.5] schematically illustrates a view of a digital twin, according to a particular and non-limiting exemplary embodiment of the present invention;

[0036] [Fig.6] schematically illustrates a view of a digital twin, according to a particular and non-limiting exemplary embodiment of the present invention;

[0037] [Fig.7] schematically illustrates a view of a digital twin, according to an example particular and non-limiting embodiment of the present invention;

[0038] [Fig.8] schematically illustrates a view of a digital twin, according to a particular and non-limiting exemplary embodiment of the present invention.

[0039] Description of the examples of embodiment

[0040] A method and a device for communicating data from a system for virtual fitting of at least one accessory by a digitally represented living being will now be described in the following with joint reference to Figures 1 to 8. The same elements are identified with the same reference signs throughout the description which follows.

[0041] [Fig.l] schematically illustrates a data communication environment 1 of a data communication system of a virtual fitting system of at least one accessory by a digitally represented living being, according to a particular and non-limiting exemplary embodiment of the present invention.

[0042] The environment 1 comprises for example a first device 101 comprising a means for capturing image data.

[0043] According to an exemplary embodiment of the present invention, the first device comprises at least one camera corresponding for example to: - an infrared camera; - RGB type image acquisition camera (from the English “Red, Green, Blue” or in French “Rouge, vert, bleu”); - camera platform (“Rig” in English) (called Lightfield) or Volumetric camera Rig of RGB type; - LIDAR type acquisition camera (“light detection and ranging” or “laser imaging detection and ranging” in English); or - image acquisition camera associated with one or more devices (for example one or more LEDs (from the English “Light-Emitting Diode” or in French “Electroluminescent Diode”) emitting light in the infrared or in the near infrared band.

[0044] According to exemplary embodiments of the present invention, the first device 101 corresponds to a smartphone or a tablet or a laptop.

[0045] According to an exemplary embodiment of the present invention, the first device 101 is a scanning system comprising a plurality of cameras synchronized with each other to capture several images at different viewpoints of at least a portion of the living being. Captured image data then represents these images captured at different viewpoints.

[0046] According to an exemplary embodiment of the present invention, the first device is a mobile communication device comprising a camera.

[0047] According to variants, the captured image data then correspond either to one image or to a plurality of images obtained by successive captures of images from the camera according to different points of view.

[0048] In the case of a camera platform (sometimes referred to as "Lightfield" in English and "volumetric image data" in French), the cameras are positioned in a circular or planar manner on the physical capture device and capture the living being in a single iteration or several (if motion capture).

[0049] According to an exemplary embodiment of the present invention, the first device 101 comprises at least one volumetric sensor making it possible to obtain volumetric image data (in English “lightfield”).

[0050] The environment 1 further comprises a second device 102, a third device 107 and several databases or registers 103, 104 and 105.

[0051] The second device 102 and the databases 103 to 105 constitute nodes of a network configured for hosting data or transactions of a blockchain 106 formed of several blocks. The second device 102 and the databases 103 to 105 are connected via a “cloud” 100 which comprises equipment for forming a network of entities connected to each other.

[0052] According to the present invention, the second device 102 is adapted to implement at least in part a block chain 106.

[0053] According to an exemplary embodiment of the present invention, the second device 102 is a set of scripts nested between them and written, for example in Python language.

[0054] This exemplary embodiment is advantageous because it allows rapid implementation of the method and facilitated development of the method's functionalities.

[0055] According to exemplary embodiments of the present invention, the third device 103 is a computer, possibly portable, a tablet or any other device suitable for displaying images from image data generated by a rendering engine such as virtual reality or augmented reality or mixed reality glasses (making it possible to vary the real on the virtual and vice versa).

[0056] According to a variant, the first device 101 and the third device 103 are implemented in the same device.

[0057] According to an exemplary embodiment of this variant, the first device 101 and the third device 103 are implemented in the same telephone.

[0058] According to a variant, the first device 101 and the third device 103 are implemented by the same pair of glasses equipped with at least one camera and one screen.

[0059] According to a variant, the first device 101 and the third device 103 are implemented by the same pair of ocular contact lenses equipped with at least one camera and said screen then adaptive to a user's view.

[0060] The present invention is not limited to this communication architecture but extends to any architecture which would comprise, for example, several devices providing the same functions as the second device 102, several devices providing the same functions as the second device 103, several databases providing the same functions as the database 103, several databases providing the same functions as the database 104 and / or several databases providing the same functions as the database 105.

[0061] The devices 101 and 107 are for example connected in communication with the “cloud” 100 via a wired connection (for example, Ethernet and / or a fiber optic connection) and / or via a wireless connection of the Wifi® type (according to IEEE 802.11 or one of the variations of IEEE 802.11 or via a wireless connection of the 4G and / or 5G and / or 6G cellular network type).

[0062] The devices 101 and 107 are thus configured to transmit data to the “cloud” 100 and / or to receive data from the “cloud” 100.

[0063] The mobile communication infrastructure enabling wireless communication of data between the devices 101 and 107 and the “cloud” 100 comprises, for example, one or more communication devices (not shown in [Fig. 1]) of the relay antenna type (cellular network). In a communication mode using such a network architecture, the data are, for example, received from the “cloud” 100 or transmitted to the “cloud” 100 by the devices 101 and 107 via one or more relay antennas (each relay antenna being, for example, connected to the “cloud” 100 via a wired link).

[0064] The wireless communication system allowing the exchange of data between the devices 101 and 107 and the “cloud” 100 corresponds for example to a cellular network type communication system, for example an LTE (Long-Term Evolution), LTE-Advanced, LTE 4G or 5G type network.

[0065] Any data transmitted to the “cloud” 100 is intended either for the device 102 or for one of the databases 103 to 105 of the “cloud” 100 and any data transmitted by the second device to the third device 103 is transmitted via the “cloud” 100.

[0066] The second device 102 and the databases 103 to 105 are advantageously configured for the implementation of the blockchain 106.

[0067] The blockchain 106, also called distributed ledger technology, is a digital tool for storing and communicating data without a central control body. The blockchain 106 is configured to allow entities implementing the blockchain 106 (in the occurrence of the devices 102 or the databases 103 to 105), to share data directly, without an intermediary. The blockchain 106 is a register or a large database which has the particularity of being shared simultaneously with all these entities, all equally holders of this register, and which also all have the capacity to enter data there, according to specific rules set by a computer protocol secured by means of cryptography.

[0068] Any transaction (transmission of data to or from the blockchain 106) requires the identification of the user of the blockchain 106 (in this case the devices 101 and 107) and is also carried out by a cryptographic method, for example by private / public key exchange.

[0069] Each transaction is sent to a network (also called a storage “node”) of remote devices or calculations (for example computers or servers) located in different locations in the “cloud” 100.

[0070] Each node hosts a copy of the database in which the history of transactions carried out is recorded. All stakeholders (i.e., for example, the device 102 and the databases 103 to 105) can access it simultaneously.

[0071] The security system is based on a consensus mechanism of all nodes for each new transaction. The encrypted data of the transaction is decrypted and authenticated by data centers (also called "miners"). For each transaction thus validated, the encrypted data is processed and the shared database is updated.

[0072] Thus, if a transaction issued by a user corresponds to an encoding of data in the block chain 106, that is to say to the addition of encrypted data in the encrypted database, the encrypted data is decrypted and authenticated by the miners. Once the transaction is validated, the encrypted data is added to the shared database in the form of a new block of encrypted data of the block chain 106. A copy of this new block is transmitted to all the nodes of the network. The integration of the new blocks is chronological, indelible and unfalsifiable.

[0073] If a transaction issued by a user corresponds to a decoding of data from the blockchain 106, i.e. obtaining encrypted data from the encrypted database, the encrypted data is decrypted and authenticated by the miners. Once the transaction is validated, the encrypted data is retrieved from the shared database in the form of a block (from the blockchain) of encrypted data. Each new block added to the blockchain 106 (or obtained from the blockchain 106) is linked to the previous one (except for the first block in the chain).

[0074] Decentralization of security management prevents tampering with transactions.

[0075] Blockchain technology is known to those skilled in the art. For example, it is described in a summary report from the French National Assembly dated December 2018 and entitled “Report of the joint information mission on blockchain and its uses: a sovereignty issue”.

[0076] Terms relating to blockchain have also been standardized, notably in the following standards: ISO 22739:2020, ISO / TR 23244:2020, ISO / TR 23455:2019 and ISO / TR 23576:2020.

[0077] [Fig.2] illustrates a flowchart of the different steps of a method for communicating data from a system for virtual fitting of at least one accessory by a digitally represented living being, according to a particular and non-limiting example of the present invention.

[0078] The method makes it possible to create a three-dimensional digital twin JN of a part of a living being 10, of a set of parts of a living being 10 or of a living being 10 in its entirety, and to calculate a view of this digital twin JN carrying a digitally represented accessory.

[0079] The living being 10 is either a human being or an animal.

[0080] The method is based on the so-called blockchain technology and is implemented in the communication environment 1.

[0081] Certain steps of the method are for example implemented by the first device 101. Other steps of the method are for example implemented by the second device 102 and / or by at least one of the databases 103, 104 and / or 105 and other steps of the method are for example implemented by the third device 107.

[0082] In a first step 201, the first device 101 captures DIC image data representative of at least part of the living being 10 from different points of view.

[0083] Captured image data DIC of at least one part of the living being 10 is understood to be image data that represents viewpoints of a single part of the living being 10, for example the head, an arm or a leg of this living being 10, of several parts of the living being 10, for example the head and the torso, or the arms and the torso, or the legs or any other combination of parts of the living being 10 or even the whole of the living being 10.

[0084] According to a variant, the captured image data DIC comprises information representative of image contents and / or descriptive information of geometry of said at least one part of the living being in a three-dimensional geometric space.

[0085] According to an exemplary embodiment, the captured DIC image data is stored in a compressed format, for example HEVC (ISO / IEC 23008-2 High Efficiency Video Coding, ITU-T Recommendation H.265, https: / / www.itu.int / rec / T-REC-H.265-202108-P / en), VVC (ISO / IEC 23090-3 Versatile Video Coding, ITU-T Recommendation H.266, https: / / www.itu.int / rec / T-REC-H.266-202008-Een) or MPEGi (ISO / IEC 23090 Coded Representation of Immersive Media).

[0086] The captured DIC image data can also be stored in glTF (graphics libraries Transmission Format) or USD (Universal Scene Description) formats.

[0087] According to a variant of the first step 201, the captured image data DIC which correspond to a captured image must verify at least one quality criterion in order to be retained.

[0088] This variant is advantageous because it makes it possible to obtain DIC image data which can be used for calculating the digital twin JN.

[0089] According to an exemplary embodiment, a quality criterion is one of the following criteria or a combination of at least two of the following criteria: - an image colorimetry level above a threshold; - an image contrast level above a threshold; - an image brightness level above a threshold; - an image sharpness level above a threshold; - a ratio between the height and width of the image between a low value and a high value.

[0090] According to a variant, when the first device 101 is a scanning system comprising a plurality of cameras, each camera is positioned spatially to be able to capture a required number of images which verify said at least one criterion.

[0091] In a second step 202, the first device 101 transmits to the second device 102, first data DI representative of a calculation and reconstruction request to obtain second data D2 representative of a view of the three-dimensional digital twin JN carrying at least one accessory according to a point of view.

[0092] The three-dimensional digital twin JN, also called realistic avatar, is a set of three-dimensional digital data which represents in the metaverse a part, a set of part(s) of the living being 10 or the living being 10 in its entirety which is represented by the captured image data DIC.

[0093] The three-dimensional digital twin JN comprises mesh data DM (“mesh” in English) representative of a three-dimensional mesh structure represented in a three-dimensional geometric space by a set of vertices of polygonal shapes forming a structure, each polygon shape, for example of triangles or quadrangles, sharing at least one side with another polygonal shape. Quadrangles are particularly advantageous because they allow for more precise deformation of the mesh structure.

[0094] The three-dimensional geometric space is a space in which the part or set of parts of the living being 10 or the living being 10 as a whole is geometrically represented from the DIC image data and the camera(s) used to capture this DIC image data.

[0095] The three-dimensional digital twin JN further comprises texture data DT representing a texture of the surfaces of the three-dimensional mesh structure, i.e., the texture generated from the texture data DT which is mapped (projected) onto the three-dimensional mesh structure when a rendering of a view of the three-dimensional digital twin JN is calculated.

[0096] The first data DI comprises the captured image data DIC and anonymization data DA making it possible to create a link between the captured image data DIC and an owner of the captured image data DIC while maintaining the anonymity of said owner.

[0097] According to a variant of the second step 202, the first DI data must be authenticated by the blockchain before being transmitted.

[0098] This variant makes it possible to secure access to the blockchain in which image data of living beings and anonymization data of this image data are recorded.

[0099] According to an exemplary embodiment of this variant, the first DI data (captured image data DIC and anonymization data DA) are advantageously recorded, stored or entered in the blockchain in the form of a non-fungible token, noted NFT (from the English “Non-Fungible Token”). An NFT is a cryptographic token corresponding to a certificate of authenticity of the first DI data attached to it, in this case the captured image data DIC and the anonymization data DA. A digital signature is advantageously associated with the NFT, which digital signature authenticates for example the first device 101 or the owner of the first device 101.

[0100] This digital signature is generated during the creation or generation of the NFT in the blockchain 106. Such an operation is called “minting” in English (or “minting” in French) and consists of associating the NFT with the first data D1, which data comprises the history of the first data D1, all of this data being recorded in one or more blocks of the blockchain 106.

[0101] The generation of the NFT (minting operation) associated with the first data D1 is for example triggered by the owner of the first device 101 via, for example via a mobile application installed on the first device 101.

[0102] According to this exemplary embodiment, the first data DI (captured image data DIC and anonymization data DA) are authenticated via an NFT recorded in the blockchain 106 and associated with a history of first data D1. If the first data D1 are authenticated via an NFT then the request is formed with these first data D1 and the request is transmitted to the second device 102.

[0103] In a third step 203, DAC accessory data representative of a three-dimensional representation of said at least one accessory in the three-dimensional geometric space are transmitted to the second device 102.

[0104] This type of DAC data is known to those skilled in the art. They represent any type of object, makeup. We often speak of a 2D / 3D layer which incorporates a material, a transparency, a relief and which is calculated by a shading and rendering process.

[0105] According to an exemplary embodiment of the third step 203, the first device 101 transmits the DAC accessory data to the second device 102.

[0106] According to an exemplary embodiment of the third step, the third device 107 transmits the DAC accessory data to the second device 102.

[0107] These last two examples of embodiment are advantageous because they allow a person to interact with a view of his digital twin JN and to be able to obtain, for example, other views of this digital twin JN carrying at least one accessory. The interaction can be to modify the accessory, to reposition it on the digital twin JN, to change its color and texture, to modify its shape. The interaction on the digital twin by a user can be to modify its morphology, color, details, hairstyle, hair, skin texture.

[0108] The first data D1 and / or the data DAC are for example transmitted according to a wireless connection via a terrestrial cellular network infrastructure such as a 4G (or 5G) network based on the LTE (Long Term Evolution) standard defined by the 3GPP consortium.

[0109] According to another example, the first data D1 and / or the data DAC are transmitted via a wired connection (for example Ethernet), for example when the first device 101 or the third device 107 corresponds to a laptop computer connected to a LAN network (from the English “Local Area Network” or in French “Réseau local”) in Ethernet for example.

[0110] The first data D1 of the request and / or the DAC data are for example generated by a mobile application installed on the first device 101.

[0111] The request and / or the DAC data are for example generated at the initiative of the user of the first device 101 or of the third device 107 via the human-machine interface (HMI) of a mobile application, which corresponds for example to a Graphical and tactile HMI via the touch interface of the screen of the first device 101 or the third device 107.

[0112] According to an exemplary embodiment, the anonymization data DA comprises a telephone number.

[0113] This exemplary embodiment is advantageous because it makes it possible to quickly obtain anonymization data which is unique and which uniquely identifies a person.

[0114] The DA anonymization data is for example automatically added to the first DI data of the request by the mobile application.

[0115] In a fourth step 204, the second device 102 receives the request and encodes the captured image data DIC and the anonymization data DA of the received request in at least one new block of the block chain 106.

[0116] According to a variant of the second step 202, the request also comprises DPV view configuration data representative of a point of view of the digital twin JN carrying said at least one accessory and / or of configuration of a view context of the digital twin JN carrying said at least one accessory, i.e. scene lighting parameters in which the digital twin evolves or even data which define a choice of scene, etc.

[0117] According to a variant of the method, the captured image data DIC and the anonymization data DA of the request are stored in the blockchain 106 in association with an NFT used to authenticate these captured image data DIC and the anonymization data DA.

[0118] For example, a new block comprising the captured image data DIC is added to the blockchain 106 and a second block comprising the anonymization data DA is added to the blockchain 106.

[0119] According to an exemplary embodiment, the captured image data (DIC), the anonymization data (DA) and a link data (DL) allowing a connection between the anonymization data (DA) and the captured image data (DIC) are stored in separate memories.

[0120] For example, the captured image data DIC is stored in the database 103 of the cloud 100 and the anonymization data DA is stored in the database 104 of the cloud 100. According to this example, the link data DL is also stored in the database 105 of the cloud 100.

[0121] This embodiment is particularly advantageous because the captured image data DIC and the anonymization data DA are protected in the blockchain 106 by cryptographic means and these captured image data DIC and anonymization data DA are still linked together by a link data DL which is stored in the database 105, thus preserving the anonymity of the owner. captured image data DIC and data exchanges between blockchain users. This example allows compliance with the General Data Protection Regulation (GDPR) because the captured image data DIC and the anonymization data DA received by the second device are stored in separate memories (databases 103 and 104 for example). These memories (databases 103 and 104 for example) can be elements of two devices that have no contact with each other. Only the link data DL allows the anonymous linking of the image data DIC and the anonymization data DA.

[0122] In a fifth step 205, the second device 102 calculates mesh data DM of the digital twin JN by modifying mesh data DMI of an initial digital twin JNI according to morphological information extracted from the captured image data DIC, and texture data DT of the digital twin JN from texture data extracted from the captured image data DIC.

[0123] The initial digital twin JNI is a digital representation of an asexual living being, human or animal. This digital representation comprises the DMI mesh data representative of a three-dimensional mesh structure represented in the three-dimensional geometric space by a set of vertices of polygonal shapes forming a structure, each polygonal shape sharing at least one side with another polygonal shape.

[0124] According to an exemplary embodiment of the fifth step 205, the DMI data are modified by modifying the spatial positions of the vertices defined in the three-dimensional geometric space.

[0125] The three-dimensional mesh structure of the initial twin JNI is then deformed (stretched, pressed, etc.) so that the modified three-dimensional mesh structure of the initial twin JNI resembles in the three-dimensional geometric space the geometry of the part or set of parts of the living being 10 or the whole of the living being 10.

[0126] The DM mesh data of the digital twin JN is then equal to the modified DMI data.

[0127] The DM data of the digital twin JN are obtained by an iterative process of deformation of the DMI data of a digital twin JNI which incorporates all the knowledge of the morphology of a living being such as for example all the morphology of a Caucasian, Afro, Asian human being and / or of an animal being, that is to say the position, a structure and a deformation of the eyes, mouth and folds of the skin, an underlying muscular system and a series of controllers allowing the deformations.

[0128] An iterative deformation method can be based on a neural network (Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications, arXiv preprint arXiv: 1704.04861, 2017) and / or on a deep learning method (Ayush Tewari, Michael Zollôfer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Theobalt Christian. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In The IEEE International Conference on Computer Vision (ICCV), 2017.) and / or on a reinforcement learning method and / or using generative adversarial networks (GAN from the English “Generative Adversarial Network”).

[0129] In the case where a digital twin JN is obtained to represent a face of a human being and where a deep learning method is used for the reconstruction of this face, a parametric modeling of the face called "3DMM" can be used. This is based on a statistical model of human faces which makes it possible to represent any face in a weighted sum of 3D mesh structure, 3D mesh structure which supposes the prior creation of an adapted 3D topology. Only the frontal part of the face - the only deformable one - is taken into account (we speak of "monkey mask" in English). The geometric aspect is enriched by the representation of the deformations corresponding to the expressions of the face. This is accompanied by a new set of parameters resulting in a number of the order of 150 for the identity of the face and 100 for the expressions. Luan Tran et al.(Nonlinear 3d face morphable model, in Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7346-7355, 2018) proposes a nonlinear modeling that marginally extends the representation capacity of the system. Pablo Garrido et al. (Reconstruction of Personalized 3D Face Rigs from Monocular Video. {ACM} Trans. Graph., presented at SIGGRAPH 2016), 35(3):28:1-28:15, 2016) introduces an analysis-by-synthesis (image) technique to iteratively estimate these parameters. This requires, on the one hand, a differentiable formulation of the rendering operation and, on the other hand, the most complete physical modeling possible of the scene including the face. This includes the interaction with ambient lighting, pose and camera characteristics. To achieve the best results, this optimization uses image data captured of the target person from multiple angles to resolve self-occlusions.It may also be useful for the person to make expressions to expose a maximum of deformation modes. The estimation of these hundreds of parameters can be carried out by regression of these parameters using a learned deep learning method. on a large set of face images (Ayush Tewari, Michael Zollôfer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Theobalt Christian. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In The IEEE International Conference on Computer Vision (ICCV), 2017), and (Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Large-scale ce-lebfaces attributes (celeb) dataset. Retrieved August,15:2018, 2018).

[0130] Thus, during this iterative process, a first three-dimensional geometric structure defined by the DNI data is positioned in the three-dimensional geometric space in rotation, oriented along the 3 XYZ axes. This first 3D mesh structure is then compared to 3D mesh structures stored in a database, each stored 3D mesh structure representing at least one part of a living being (its face). The iterative process also compares each 2D image calculated from the first 3D mesh structure to 2D images calculated from a 3D mesh structure stored in the database. The method of comparing these elements seeks to bring the 2D images calculated from the first 3D mesh structure closer to 2D images calculated from 3D mesh structures stored in the database.The method thus iteratively improves the distances between the vertices of the first mesh structure and the vertices of a 3D mesh structure and by microdisplacements of the vertices of the first 3D mesh structure in the three-dimensional geometric space so as to optimize the resulting first 3D mesh structure. This spatial operation of deformation of the first 3D mesh structure therefore proceeds by successive iterations until a resemblance is obtained between the first 3D mesh structure and a 3D mesh structure stored in the database geometrically representing at least one part of a living being. The three-dimensional geometric structure thus obtained corresponds in resemblance to the captured DIC image data.

[0131] According to a variant, a comparison method can also be used to correct possible differences between a capture by fixed camera platform of a mobile and / or fixed device and in controlled ambient light with a capture by a telephone.

[0132] According to an exemplary embodiment, the texture data DT is texture data extracted from an area defined on a part or a set of parts of the living being or the whole of the living being from the captured image data DIC.

[0133] According to an exemplary embodiment, the texture data of the area are obtained by assembling the captured DIC image data, isolated, distributed and assembled to reconstruct a single texture.

[0134] For example, the assembly process involves deforming the texture of each of the images obtained from the DIC captured image data by modifying the color and contrast of the pixels of the texture images, corresponding to the texture of the images obtained from the DIC captured image data, and correcting the boundaries between the texture images to generate a unique texture.

[0135] This unique texture is then projected onto the surface of the 3D mesh structure corresponding to the DM data (projection known as UV mapping, the letters U and V designating the axes of the 2D texture). This then allows a reprojection of said assembled texture onto said 3D mesh structure.

[0136] For example, when a part of the living being 10 is a face of a person, the texture data can be obtained from a surface of an area defined between the forehead, the ears and the chin (“monkey mask” in English).

[0137] According to a variant of the method, the data DM and DT of the digital twin JN are stored in a memory of the second device 102 or accessible by the second device 102.

[0138] According to an exemplary embodiment of the fifth step 205, this memory is distinct from the databases 103 and 104 to maintain the anonymization of the data.

[0139] In a sixth step 206, the second device 102 calculates the second data D2 representative of a view according to the point of view of the digital twin JN carrying said at least one accessory from the mesh data DM and the texture data DT of the digital twin JN and the accessory data DAC.

[0140] According to an exemplary embodiment of the sixth step 206, the second device 102 comprises a rendering engine which calculates the second data D2 representative of an image of the digital twin JN carrying said at least one accessory according to the point of view.

[0141] According to an exemplary embodiment, the rendering engine is a real-time rendering engine.

[0142] According to an exemplary embodiment, the rendering engine is implemented on a GPU computing server.

[0143] For example, the real-time rendering engine is a 3D simulation engine.

[0144] In a seventh step 207, the second device 102 transmits the second data D2 to the third device 107.

[0145] According to an exemplary embodiment of the seventh step 207, the second data D2 are broadcast continuously (in English “streaming”) which avoids a complete download of these second data D2 before starting to render them on a screen for example.

[0146] According to an exemplary embodiment, the second data D2 are for example transmitted via a wireless connection when the request has been transmitted via a wireless connection.

[0147] According to an exemplary embodiment, the second data D2 are transmitted via a wired connection, for example when the request has been transmitted via a wired connection.

[0148] According to a variant of the seventh step 207, the second data D2 are transmitted with the data representative of the NFT to prove the authenticity of the second data D2 and, for example, the ownership of the first device 101 which issued the request.

[0149] In an eighth step 208, the third device 107 calculates a rendering of the view of the digital twin JN from the second data D2.

[0150] According to an exemplary embodiment of the eighth step 208, the second data D2 are displayed on a display screen of the third device 107.

[0151] According to a variant of the eighth step 208, the second data D2 are displayed with the certificate of authenticity corresponding to the NFT.

[0152] Such a method allows a person to virtually try on accessories and decide whether or not to purchase these accessories.

[0153] Through an HMI implemented on the third device 107, a person can access a sales site and select accessories. They can also choose to virtually try on these accessories. LTHM can then be provided so that this person adds the selected accessories to a basket, which can be called a virtual fitting room. The method described above is then implemented so that a digital twin of this person is calculated and the person can view a view of their digital twin then carrying accessories that this person would have selected. Thus, the person can interact with their digital twin from this HMI which displays the images of this digital twin carrying these accessories according to points of view that can be selected by this person. The person can also choose the conditions of the scene (lighting, content of the scene) in which their digital twin evolves.The person can also choose the lighting conditions of the scene in which their digital twin evolves or even change the scene in which the digital twin is presented.

[0154] Figures 3 to 8 schematically illustrate a system for virtual fitting of an accessory by a digitally represented living being, according to a particular and non-limiting exemplary embodiment of the present invention.

[0155] According to this example, the first device 101 is an image data capture system comprising a plurality of image capture means. Typically, this system is enclosed in a capture booth and the different image capture means are spatially distributed around a person wishing to virtually try accessories accessible from an HMI of the third device 103 which may be for example a telephone.

[0156] Let us assume that the method described in relation to [Fig.2], is implemented in the context of a site for the sale of accessories comprising a Human-Machine Interface (HMI) comprising areas of a touch screen associated with interactive applications (in English “widgets”).

[0157] [Fig.3] schematically illustrates an example of this HMI which is displayed on the screen of the telephone 107 according to a particular and non-limiting example of the present invention.

[0158] Suppose that a person wishes to virtually try on an accessory displayed in an area 1071 of a page of a sales site displayed on a touch screen of the telephone 107.

[0159] This person must begin by creating their digital twin JN. To do this, they sit in the capture booth at the center of the image data capture system (first device 101). An image capture method is then launched to capture image data DIC of a part of the person, for example their head (first step 201). Once these image data DIC have been captured, the image data capture system (first device 101) transmits to the second device 102, first data DI representative of a request to obtain second data D2 representative of a view of a three-dimensional digital twin JN 1 of the person's head from a point of view, for example from the front (second step 202).

[0160] The second device 102, which may for example be a server accessible via the Internet, receives the request and encodes the captured image data DIC and the anonymization data DA of the received request in at least one new block of the block chain 106 (fourth step 204).

[0161] The second device 102 then calculates mesh data DM according to morphological information extracted from the captured image data DIC, and texture data DT of the digital twin obtained from texture data extracted from the captured image data DIC (fifth step 205). The digital twin is advantageously stored in a memory of the second device 102 or accessible by the second device 102.

[0162] The person can then navigate to the sales site page to select at least one accessory for sale on this site.

[0163] For example, this person selects the accessory, for example earrings, represented in the area 1071 by pressing this touch-sensitive area of ​​the screen of the telephone 107. This accessory is added to a virtual fitting room 1072. The person may then want to view a view of his digital twin JN wearing the accessory(ies) which are present in the virtual fitting room by pressing, for example, a touch-sensitive area 1072 of the screen of the telephone 107.

[0164] Accessory data DAC representative of a three-dimensional representation of said at least one selected accessory are then transmitted by the telephone 107 to the second device 102 (third step 203). The second device 102 then calculates second data D2 representative of a view according to a point of view of the digital twin carrying said at least one accessory from the mesh data DM and the texture data DT of the digital twin JN and the accessory data DAC (sixth step 206). These second data D2 are then transmitted by the second device 102 to the telephone 107 (seventh step 207). The telephone 107 calculates a rendering of the view of the digital twin JN carrying said at least one accessory according to the point of view from the second data (eighth step 208). The telephone 107 then displays this view in an area 1073 ([Fig.4]). The [Fig.5] schematically illustrates an example of a front view of the bust of the person wearing the selected accessory, here a pair of earrings.

[0165] The person can then interact with the digital twin JN carrying the accessory. For example, the telephone 107 can allow this person to change the point of view of his digital twin JN and / or delete an accessory.

[0166] [Fig.6] illustrates the case of a side view of the digital twin JN wearing the selected accessory (pair of earrings) and figures 7 and 8 represent the digital twin JN without the accessory seen from the front and from the side.

[0167] The present invention is not limited to these few examples of views of a digital twin JN because the person can choose points of view all around the digital twin JN due to its three-dimensional representation evolving in a scene which is also three-dimensional.

[0168] The present invention is also not limited to a particular part of a living being or to a particular accessory.

[0169] The person can also choose the scene conditions (lighting, scene content) in which their digital twin JN evolves. The person can also choose the lighting conditions of the scene in which their digital twin evolves or change the scene in which the digital twin is presented. These interactions can be implemented by touch actions of the person on the dedicated areas of the screen of the telephone 107. At each change of interaction, a new request carrying data representative of the requested changes to the view currently displayed in the area 1073, is sent by the telephone 107 to the second device 102 which calculates new data D2 (sixth step 206) and which transmits this second data D2 (seventh step 207) to the telephone 107.The telephone 107 then calculates a new rendering of the view of the digital twin JN possibly carrying at least one accessory according to a . point of view from the new second data (eighth step 208). The telephone 107 then displays this new view in an area 1073 ([Fig.5]).

[0170] The method is also advantageous because the digital twin and the accessories are three-dimensional digital data which are defined in the metaverse. The digital twin carrying the accessories can thus be an element of this metaverse and thus be placed in a particular universe which can be defined in relation to a sales site and / or the accessories worn by this digital twin and / or according to the preferences of this person.

[0171] The blockchain 106 makes it possible to trace all of the operations or transactions relating to the management of data relating to the virtual fitting system of an accessory by a living being represented digitally in the “cloud”.

[0172] The third device 103 may change during the process described in relation to [Fig. 2]. Indeed, for example, a person may at any time during the process put on a pair of virtual reality or augmented reality or mixed reality glasses to immerse themselves in the scene in which their digital twin JN is viewed and continue to interact with it using control means, possibly haptic, worn for example on their hands or on a part of their body.

[0173] Among the interactions with his digital twin, the person can also record a video of a temporal evolution of his digital twin in a scene.

[0174] Of course, the present invention is not limited to the exemplary embodiments described above but extends to a method of communicating data from a virtual fitting system for at least one accessory by a digitally represented living being which would include secondary steps without thereby departing from the scope of the present invention. The same would apply to a system configured for the implementation of such a method.

Claims

Claims

1. Method for communicating data from a virtual fitting system of at least one accessory by a digitally represented living being, said method comprising the following steps: - capture (201), by a first device (101), of image data (DIC) representative of at least part of the living being from different points of view; - transmission (202), by the first device (101) and to a second device (102) implementing at least in part a blockchain (106), of first data (D1) representative of a calculation and reconstruction request to obtain second data (D2) representative of a view of a three-dimensional digital twin (JN) carrying said at least one accessory according to a point of view, said digital twin (JN) comprising mesh data (DM) representative of a three-dimensional mesh structure represented in a three-dimensional geometric space by a set of vertices of polygonal shapes forming a structure, each polygonal shape sharing at least one side with another polygonal shape, the three-dimensional digital twin further comprising texture data (DT) representing a texture of surfaces of the three-dimensional mesh structure,the first data (Dl) comprising the captured image data (DIC) and anonymization data (DA) making it possible to create a link between the captured image data (DIC) and an owner of the captured image data while maintaining the anonymity of said owner;, - transmission (203) to the second device (102), of accessory data (DAC) representative of a three-dimensional representation of said at least one accessory in the three-dimensional geometric space; - encoding (204), by the second device (102), the captured image data (DIC) and the anonymization data (DA) of the request in the blockchain; - calculation (205), by the second device (102), of the mesh data (DM) by modification of mesh data (DMI) of an initial digital twin (JNI) according to morphological information extracted from the captured image data (DIC), and texture data (DT) of the digital twin obtained from extracted texture data captured image data (DIC); - calculation (206), by the second device, of the second data (D2) representative of a view according to the point of view of the digital twin carrying said at least one accessory from the mesh data (DM) and the texture data (DT) of the digital twin (JN) and the accessory data (DAC); - transmission (207) of the second data (D2) by the second device (102) to a third device (107); and - calculation (208), by the third device (107), of a rendering of the view of the digital twin (JN) carrying said at least one accessory according to the point of view from the second data (D2).

2. The method of claim 1, wherein the captured image data (DIC) corresponding to a captured image must satisfy at least one quality criterion to be retained.

3. Method according to claim 1 or 2, for which the first data (Dl) are authenticated by the blockchain before being transmitted.

4. Method according to one of the preceding claims, for which the captured image data (DIC), the anonymization data (DA) and a link data (DL) allowing a connection between the anonymization data (DA) and the captured image data (DIC) are stored in separate memories and / or registers.

5. Method according to one of the preceding claims, for which the accessory data (DAC) are transmitted by the second or the third device.

6. Method according to one of the preceding claims, for which the request also comprises view parameterization data (DPV) representative of a point of view of the digital twin (JN) carrying said at least one accessory and / or parameterization of a view context of the digital twin (JN) carrying said at least one accessory.

7. Method according to one of the preceding claims, for which the captured image data (DIC) comprises information representative of image contents and / or descriptive information of geometry of said at least one part of the living being in the three-dimensional geometric space.

8. Method according to one of the preceding claims, for which the captured image data corresponds to one image or several images.

9. Method according to one of the preceding claims, in which the Mesh data (MD) and texture data (TD) of the digital twin are calculated by CPU and / or GPU and / or quantum computing.

10. System for virtual fitting of at least one accessory by a digitally represented living being comprising: - a first device (101) comprising communication means, at least one means for capturing image data and at least one processor provided for executing instructions for implementing at least one step of the method according to one of claims 1 to 9; - a second device (102) comprising communication means, means for implementing at least part of a blockchain and at least one processor provided for executing instructions for implementing at least one step of the method according to one of claims 1 to 9; - a third device (103) comprising communication means, a screen and at least one processor provided for executing instructions for implementing at least one step of the method according to one of claims 1 to 9;

11. The system of claim 10, wherein the first device and the third device are the same device.

12. System according to claim 11, for which the first device (101) and the third device (103) are: - the same telephone; or - the same pair of glasses equipped with at least one camera and said screen; or - the same pair of ocular contact lenses equipped with at least one camera and said screen then adaptable to a user's view.

13. Computer program comprising instructions for implementing the method according to any one of claims 1 to 9, when these instructions are executed by a processor.

Images