Creating a digital copy of a real instance from at least one image

Abstract

Creation of a digital copy of a real instance from at least one image The invention relates to a method for generating a digital copy of a real instance, the method comprising the following steps: - obtaining (300) at least one image representative of an environment; - detecting (301) at least one given real instance in said at least one received image; - obtaining (302-304) at least one piece of information relating to the given real instance; - generating (305) a digital copy of said given real instance, the digital copy comprising said at least one piece of information relating to the given real instance. Figure for abstract: [Fig 3]

Description

Title of the invention: Creation of a digital copy of a real instance from at least one image Technical field

[0001] The invention relates to the field of virtual representation of a real instance, object or person, also called twin or digital copy.

[0002] Its purpose is in particular to facilitate the creation of a copy or digital twin for a given real instance. State of the art

[0003] A digital copy or twin, or "digital twin" in English, is a virtual representation of a real instance, for example an object or product, a process or a person.

[0004] In order for the digital twin to model the real instance to which it corresponds, one or more sensors internal to the real instance, or external but capable of obtaining descriptive data of the real instance, can collect data which are sent to a device or server which stores the digital twin, in order to update the digital twin with the collected data.

[0005] In the case of a product or object, such data includes, for example, operating data of the product or object, temperature, weather conditions to which the product or object is subjected, dimensions of the object, state of use of the object, etc.

[0006] The digital twin can be used for design, simulation, monitoring, optimization and / or maintenance of this product or object. Actions can then be taken on the real instance, in order to improve its performance or its lifespan for example.

[0007] In the context of product modeling, there are different types of digital twins that can model a particular component of a product, an interaction between two components of a product, or the entire product as a system of components interacting with each other, when the product comprises several components.

[0008] Thus, the real instance corresponding to the digital twin can be a system of objects.

[0009] In order to create a new digital copy, for a real instance not yet virtually duplicated, it is currently required for an operator to manually define the real instance by a user interface of a user terminal for storage of the digital copy thus defined in the user terminal or in a remote server. It is furthermore necessary for the operator to determine the identity of each actual instance for which a digital copy is created.

[0010] It is thus possible to create digital copies for the real instances composing a simple environment comprising few real instances. However, such a creation requires significant human and technical resources: it mobilizes a human operator and requires a user interface to access an application dedicated to the digital twin creation service. Getting to grips with such an application, if it is not very intuitive, requires a long adaptation period for the human operator.

[0011] However, there is no solution for simply generating a digital copy of a real instance, requiring little or no human resources, particularly in an environment comprising a large number of real instances for which respective digital twins are to be created.

[0012] The invention offers a solution which does not have the drawbacks of the state of the art. Statement of the invention

[0013] To this end, according to a functional aspect, the invention relates to a method for generating a digital copy of a real instance, the method comprising the following steps: - obtaining at least one representative image of an environment; - detection of at least one given real instance in said at least one received image; - obtaining at least one piece of information relating to the given real instance; - generation of a digital copy of said given real instance, the digital copy comprising said at least one piece of information relating to the given real instance.

[0014] Thus, the invention allows the automated creation of a digital copy of a given real instance, from an image acquired from an environment comprising the given real instance. The digital copy may comprise one or more pieces of information relating to the given real instance: it is thus not only facilitated the creation of the digital copy, but also its enrichment with information relating to the given real instance. However, the digital copy can then be enriched with data acquired by sensors other than the image acquisition device having acquired the at least one image.

[0015] According to embodiments, the information relating to the given real instance may comprise location information of the given real instance and / or type information representative of a type of the given real instance.

[0016] Location information advantageously makes it possible to model the environment, and in particular the interactions between the different real instances making up an environment. Type information makes it easier to obtain information complementary to the type information.

[0017] In addition, obtaining the location information and / or the type information may comprise an image analysis of the at least one received image.

[0018] It is thus possible to obtain such information without requiring access to external resources.

[0019] In addition, obtaining the location information and / or the type information may comprise an analysis of the at least one image received by a convolutional neural network.

[0020] A convolutional neural network is particularly suitable for image segmentation, and / or for the classification of the given real instance into a type among a set of predetermined types for which the convolutional neural network has been previously trained.

[0021] According to embodiments, the at least one information may comprise at least one visible characteristic of the given real instance, comprising: - a size, color, make and / or model of the given actual instance; and / or - a text written on the given real instance; and / or - an interaction of the given real instance with at least one other real instance of the environment.

[0022] Such visible characteristics make it possible to automatically enrich the digital copy.

[0023] Additionally, the visible characteristic may be identified by image analysis of a portion of the at least one image comprising the given actual instance.

[0024] It is thus possible to obtain such information without requiring access to external resources.

[0025] According to embodiments, said at least one piece of information may comprise at least one internal and / or functional characteristic of the given real instance, obtained from a resource storing data distinct from the at least one image obtained.

[0026] Thus, it is made possible to enrich the digital copy in an automated manner from information acquired from an external resource. In certain embodiments, the digital copy can thus comprise both visible characteristics, resulting from an analysis of the at least one acquired image, but also the non-visible characteristics (internal and / or functional) obtained from the external resource. It is thus made possible to automate the generation of a digital copy with a complete description.

[0027] Additionally, the at least one internal and / or functional characteristic can be obtained from said resource via a generative artificial intelligence interface programming interface.

[0028] The use of a generative AI (Artificial Intelligence) API (Application Programming Interface) makes it possible to obtain complete internal and / or functional characteristics, determined from very large databases.

[0029] In addition or as a variant, obtaining said at least one internal and / or functional characteristic may comprise: - the generation of at least one instruction based on data representative of the given real instance, the data representative of the given real instance being obtained from said at least one image; - obtaining at least one response, following transmission of the at least one generated instruction, the response comprising said at least one technical and / or functional characteristic.

[0030] It is thus made possible to obtain targeted internal and / or functional characteristics, by the use of at least one instruction, or prompt. It is furthermore possible to provide a series of instructions making it possible to refine the characteristics thus obtained.

[0031] In addition, the data representative of the given real instance can be: - a part of the at least one image comprising the given real instance; or - information relating to the given real instance, resulting from an image analysis applied to a part of the at least one image comprising the given real instance.

[0032] It is thus made possible to generate an instruction or prompt, from information obtained automatically in the preceding steps of the method according to the invention. The invention can in particular provide an image analysis, enriched by the search for internal and / or functional information obtained from an external resource, by describing the given real instance from the information determined previously during the image analysis.

[0033] Further in addition, the instruction can be generated from a given visible characteristic resulting from the image analysis applied to the part of the at least one image comprising the given real instance, the given visible characteristic being the text written on the given real instance.

[0034] The text written on an object can allow precise identification of the object, and obtaining precise internal / functional information for the identified object. The extraction of a written text can also be based on known and robust character recognition solutions.

[0035] Additionally or alternatively, the generated instruction may further indicate a given format, the given format being a predetermined format associated with the digital copy, the response may include said at least one technical and / or functional characteristic according to the given format and the digital copy may be generated according to the given format. Thus, the generation of the digital copy is simplified in that it does not require formatting of at least some of the information relating to the given real instance.

[0036] According to embodiments, the method may further comprise: - detection of at least one other real instance in said at least one image obtained in addition to the given real instance; - obtaining at least one piece of information relating to the other real instance; - creation of a digital copy of said other given real instance, the digital copy comprising said at least one piece of information relating to the other real instance.

[0037] It is thus made possible to create digital copies of all the real instances detected in a scene, in an automated manner.

[0038] According to embodiments, the method may comprise obtaining at least one new image, acquired after said at least one image, the method further comprising: - detection of at least one new real instance given in said at least one new image obtained; - obtaining at least one new piece of information relating to the given new real instance; - if it is determined that the new given real instance and the given real instance are the same real instance, updating the digital copy of the given real instance based on the new information obtained.

[0039] It is thus possible to automatically update the digital copy of a real instance, by acquiring successive images representative of an environment comprising the real instance.

[0040] In the event of absence of detection of the given real instance in the at least one new image, the method may comprise the deletion of the digital copy of the given real instance. It is thus possible to automatically delete the digital copy of a real instance, when it leaves the environment.

[0041] According to a hardware aspect, the invention relates to a device for generating a digital copy of a real instance, the device comprising an interface configured to obtain at least one image representative of an environment, and a processor configured to: - detecting at least one given real instance in said at least one received image; - obtaining at least one piece of information relating to the given real instance; - generating a digital copy of said given real instance, the digital copy comprising said at least one piece of information relating to the given real instance.

[0042] According to another material aspect, the invention also relates to a computer program capable of being implemented on a device, the program comprising code instructions which, when the program is executed by a processor, carry out the steps of the defined method.

[0043] Such programs may use any programming language. They may be downloaded from a communications network and / or recorded on a computer-readable medium.

[0044] According to another material aspect, the invention relates to a data medium on which at least one series of program code instructions has been stored for the execution of the method defined above. Brief description of the drawings

[0045] The invention will be better understood on reading the following description, given by way of example and with reference to the appended drawings in which:

[0046] [Fig.l] illustrates a system for generating a digital copy of a real instance from at least one image representative of an environment, according to embodiments of the invention;

[0047] [Fig.2] illustrates an image acquired by a copy generation system digital of a real instance, according to embodiments of the invention;

[0048] [Fig.3] illustrates the steps of a method for generating and updating a copy digital of a real instance from an image of an environment, according to embodiments of the invention;

[0049] [Fig.4] illustrates a device for generating a digital copy of an instance real, according to embodiments of the invention. Description of the embodiments

[0050] [Fig.l] illustrates a system 100 for generating a digital copy of a real instance in an environment 120.

[0051] The environment 120 may comprise a set of at least one real instance. By "real instance" is meant any object or product, any component of such an object or product, any system of objects or products, any process, any person or any animal or plant. In the example of [Fig.l], the environment 120 comprises, by way of illustration, a first real instance 110, which is a fire extinguisher, and a second real instance 112, which is a chair.

[0052] The fire extinguisher 110 may be considered as composed of a set of individual components forming the fire extinguisher, each of these components being an actual instance. Alternatively, and as considered in the remainder of the description, the fire extinguisher 110 is considered as a single first real instance. The same applies to chair 112 which is considered in the following as a single second real instance.

[0053] At least some of the real instances may include a written text, in particular for the real instances of object or component type. This is the case in the example shown in [Fig.l], in which a text 111 is written on the extinguisher 110. The text 111 may for example be instructions for using the extinguisher 110.

[0054] The system 100 according to the invention comprises an image acquisition device 101 capable of obtaining at least one image representative of the environment 120. The image acquisition device 101 is capable of obtaining an image, a series of images or a video stream comprising video images or video frames.

[0055] For the purpose of simplifying the description of the invention, it is considered in the following that the image acquisition device 101 is a camera.

[0056] In the example of [Fig.l], the first real instance 110 and the second real instance 112 are both located in a field of view of the camera 101. Such a field of view depends on intrinsic characteristics of the camera 101, in particular its focal length, and on the position of the camera 101 relative to the environment 120. The field of view may be fixed, and comprise the first real instance 110 and the second real instance 112. Alternatively, the camera 101 is mobile and the field of view may vary, so as to acquire images of different fields of view corresponding to respective positions / orientations of the camera 101. For example, a first image may comprise the first real instance 110 and a second image, acquired before or after the first image, may comprise the second real instance 112.

[0057] In the following, it is considered, by way of illustration, that the camera is capable of acquiring an image 200 representative of the first real instance 110 and of the second real instance 112, as represented in [Fig.2], described in the following.

[0058] The system 100 further comprises a digital copy generation device 102 capable of communicating with the camera 101. As described below, the device 102 is capable of generating a digital copy of at least one of the real instances, from at least one image representative of the environment 120, acquired by the camera 101. An update of the digital copy thus generated can also be implemented by the device 102.

[0059] The device 102 can store the digital copy thus generated or updated, locally in a memory of the device 102 or in a memory in communication with the device 102, or in a remote storage device 104, which the device 102 can access via a wide area network 103, of the IP type for “Internet Protocol” for example. Alternatively, the network 103 is a local network. No res No power is attached to the storage device 104 which may be a storage server 104 dedicated to a digital twin service. The server dedicated to the service can thus process the descriptive data of the real instances corresponding to the stored digital copies, for the purposes of design, simulation, monitoring, optimization and / or maintenance of the real instances.

[0060] The storage device 104 can in particular be shared between several devices 102. The storage device 104 can thus store digital copies of real instances located in separate locations, or even in separate sites.

[0061] In the example of [Fig.l], the device 102 is connected to a single camera 101. The device 102 can communicate with a plurality of cameras 101, for example located on the same site, in order to generate and update digital copies of some or all of the real instances located in the respective fields of vision of the cameras 101.

[0062] The system 100 may further comprise an external resource 105 accessible via the network 103, which may be an external database. The database 105 may in particular be queried via a software interface, such as an application programming interface, API, which may be a local API 106, or at the edge, executed in the device 102 or in another device forming part of the local network of the device 102, which may be a remote application programming interface 107 (of the “cloud” type), accessible by the device 102 via the network 103, or which may be an API hosted by the device 102 itself. The database 105 can store detailed information about a set of objects or people, and querying the database 105 thus makes it possible to obtain information relating to the actual instances detected in the image 200, as described with reference to [Fig.3] describing the method according to the invention.

[0063] According to the invention, several external databases 105 can be queried by different respective APIs, which can be local or remote.

[0064] [Fig.2] shows an image 200 acquired by the camera 101 and to which the device 102 has applied a first image analysis in order to identify the first real instance 110 and the second real instance 112 in first and second parts 210 and 212 of the image 200.

[0065] [Fig.3] shows the steps of a method for creating a digital copy of a real instance, according to embodiments of the invention.

[0066] The method can be implemented in the device 102 described previously with reference to [Fig.l].

[0067] In a step 300, the device 102 receives at least one image from the camera 101, the image being representative of an environment, such as the environment 120 described previously. As described previously, the device 102 can receive a image or series of images, such as a series of video images, at step 300.

[0068] In a step 301, the device 102 can detect at least one real instance in said at least one image received in step 300. Such detection can comprise the determination of a part of the image, for example a rectangle, also called a "bounding box" in English, comprising the real instance. In the example shown in [Fig.2], step 301 thus comprises the determination of the first and second parts 210 and 212 respectively comprising the first real instance 110 and the second real instance 112. When several real instances are detected in step 301, the following steps of the method are implemented for at least one of the detected real instances, for example for the first real instance 110 which is a fire extinguisher in the example considered previously. Step 301 can thus comprise the determination of the first part 210 of the image 200 comprising the first real instance 110.In addition, step 301 may comprise, from the first determined part 210, the determination of location information of the first real instance in the environment 120. Step 301 is based on a first analysis of the image 200 received in step 300, or of the image stream, when several images are received in step 300. No restriction is attached to the first image analysis, which may be based on a known image analysis algorithm, capable of detecting instances of an image, for example by segmentation of the image. By way of example, the first image analysis may for example be implemented by submitting the image as input to a first convolutional neural network, or first CNN (“Convolutional Neural Network” in English), capable of detecting real instances in the image received as input and of segmenting the image according to the detected real instances.Such a convolutional neural network may be derived from machine learning, in particular from deep learning when the convolutional neural network is a deep neural network. Other technical solutions may be used to implement the first image analysis, according to the invention.

[0069] The first image analysis may be based on one of the following algorithms: You Only Look Once (or YOLO in English, which is a real-time instance detection algorithm based on a CNN), Faster R-CNN, SSD (“Single Shot Detector” in English), Mask R-CNN, RetinaNet, Inception, OpenCV, or any other image analysis algorithm capable of detecting a real instance in a part of the image. Alternatively, the first image analysis may be based on an algorithm such as, for example, Segment Anything Model from Meta™ which makes it possible to finely cut out an instance in an image: the first part 210 thus determined during the detection of the extinguisher 110 has in this case a shape close to the extinguisher, and is not a rectangle as shown in [Fig.2].

[0070] The first image analysis may be implemented by the device 102 (software stored in the device 102 and implementing the first image analysis), or by another device in the local network of the device 102, or may be implemented by a Cloud-type software service, which the device 102 accesses via the network 103.

[0071] At a step 302, the device 102 may identify type information of the first real instance detected at step 301. Such type information identifies a type, a family or a class of the real instance. No restriction is attached to the level of generality corresponding to the identified type / family / class. For example, in the case where the first real instance is a fire extinguisher, the type information may identify the type “fire extinguisher”, or a more general type such as “object”, “actionable object”, “safety device”, or any other general designation within which the instance “fire extinguisher” is included. The precision associated with the type depends on the second image analysis implemented at step 302.The aforementioned examples of algorithms are also suitable for implementing the second image analysis, namely the following algorithms: You Only Look Once, Faster R-CNN, SSD, Mask R-CNN, RetinaNet, Inception, OpenCV, or any other image analysis algorithm suitable for detecting an instance in an image or in a part of the image.

[0072] The second image analysis may be implemented by the device 102 (software stored in the device 102 and implementing the second image analysis), or by another device in the local network of the device 102, or may be implemented by a Cloud-type software service, which the device 102 accesses via the network 103.

[0073] According to embodiments, steps 301 and 302 can be implemented jointly by an image analysis, carried out by a single software solution. For example, the aforementioned algorithms YOLO, Faster R-CNN, SSD, Mask R-CNN, RetinaNet, Inception, OpenCV make it possible to carry out steps 301 and 302 jointly.

[0074] According to a first embodiment, following step 302, the method proceeds directly to a step 305 of generating a digital copy of the first real instance, or of updating the first real instance, described later. Alternatively, following step 302, the method proceeds to a step 303 or a step 304, which are described in the following.

[0075] In a step 303, the device 102 can obtain at least one visible characteristic of the first real instance 110, by a third image analysis applied to the part of the image comprising the given real instance. By "visible characteristic" is meant a characteristic which can be deduced or estimated from the visual aspect of the given real instance, and therefore by an image analysis.

[0076] The at least one visible feature is any one of the following features, or any combination of the following features: - a size (including a set of dimensions), color, make and / or model of the given actual instance; and / or - a text inscribed on the given real instance, such as text 111 inscribed on the first real instance 110; and / or - an interaction of the given real instance with at least one other real instance detected in the at least one image.

[0077] The third image analysis can be implemented by software, stored by the device 102 or by another device of the local network of the device 102, or implemented in the form of a cloud service, accessible remotely via the network 103, optionally via one of the APIs 106 or 107. The software solution implementing the third image analysis can be for example one of the following solutions, listed in a non-limiting manner: - Google Cloud Vision AI™, which is capable of recognizing objects, scenes and people in images. In addition, such a solution can be used to extract technical, non-visible information from recognized objects (in accordance with step 304 described below); - Amazon Rekognition™, which is capable of recognizing objects. In addition, such a solution can be used to extract technical, non-visible information from recognized objects (in accordance with step 304 described below); - Microsoft Azure Computer Vision™, which is able to analyze the content of an image; - IBM Watson Visual Recognition™, which is capable of implementing object detection and classification of objects detected in an image; - Clarifai™, which is a platform hosting artificial intelligence tools to recognize objects and classify images; - TesseractTM, which is a text extraction service from an image; - Open Source Computer Vision Library, which is a library, from which the third image analysis can be trained; - Mobile Vision API, which is an API that provides a computer vision function.

[0078] Note that the third image analysis may include the application of at least two separate software solutions, for example, one to obtain the size, color, brand and / or model characteristics of the given real instance, and the other to extract text written on the given real instance. The extraction of text written on the given real instance may in particular be implemented by an optical character recognition tool, also called OCR, for "Optical Character Recognition" in English.

[0079] Following step 303, the method may proceed directly to step 305 described later, or may proceed to step 304 described below.

[0080] At a step 304, the device 102 can obtain at least one piece of information complementary to the visible information obtained at step 303, and / or at least one internal and / or functional characteristic, from an external resource, such as the database 105 described previously.

[0081] The term “complementary information of the visible information” means external and visible information of the given real instance, but which cannot be obtained by analysis of the at least one image received in step 300, or which makes it possible to increase the precision of a visible characteristic obtained by analysis of the at least one image received in step 300. The complementary information may in particular comprise the precise dimensions of the given real instance, which may be obtained after identification of the model during step 303, and by querying the database 105, which may store in correspondence such complementary information in association with data representative of the given real instance (for example the model of the given real instance).

[0082] An internal and / or functional characteristic is understood to mean a characteristic that is not visible to the given real instance, and therefore not deducible by simple visual observation of the given real instance in an image. The internal and / or functional characteristics may include: - any chemical composition of a product stored in the given real instance; and / or - an identifier of an API for controlling the given real instance, when the given real instance is a connected object; and / or - a mode of operation of the given real instance; and / or - a default setting of the given real instance; and / or - any other internal and / or functional characteristic, which depends on the actual instance considered.

[0083] Such internal and / or functional characteristics may for example be detailed in a technical notice associated with the given real instance, which may be stored in the database 105 in association with a model identifier. Thus, the device 102 may query the database 105 based on the model of the given real instance obtained in step 303, or, alternatively, based on the type of object determined in step 302.

[0084] More generally, the device 102 can query the database 105 from any data representative of the given real instance, or any combination of data representative of the given real instance. The data representative of the given real instance can be: - the part of the image comprising the given real instance; and / or - any information, or combination of information, relating to the given real instance obtained in step 301, step 302 and / or step 303.

[0085] According to advantageous embodiments, step 304 may comprise querying a generative artificial intelligence API, capable of returning a response based on an instruction provided as input, also called a “prompt”. The generative artificial intelligence API may be an artificial intelligence conversational agent, such as ChatGPT™. Alternatively, the generative artificial intelligence API may be Llama2™, Claude™, Mistral AI, or any other known solution.

[0086] Such a generative artificial intelligence API may be an edge API 106, a cloud-based remote API 107, or an embedded API in the device 102.

[0087] Thus, according to the invention, at least one instruction is generated in step 304 by the device 102, from at least one data item representative of the given real instance. The at least one instruction is submitted to the generative AI API, and at least one response is received in return, the at least one response comprising at least one internal and / or functional characteristic of the given real instance and / or the at least one aforementioned complementary information.

[0088] For example, all or some of the information relating to the given real instance obtained in steps 301 to 303 may be included in a single instruction, and a response comprising at least one internal and / or functional information of the given real instance, and / or at least one complementary information of the visible characteristics, is received in return. Alternatively, the device 102 may be configured to generate a series of instructions, each instruction comprising at least one data item representative of the object. A series of responses to the instructions is received in return, each response in the series comprising at least one internal and / or functional information, and / or at least one complementary information of the visible characteristics, distinct from those contained in the other responses.

[0089] According to embodiments of the invention, the at least one generated instruction may further indicate a response format, and the at least one response received when querying the external resource conforms to the response format indicated in the at least one instruction. Such a possibility is offered by certain generative AI APIs. The indicated response format may advantageously correspond to a predetermined format of the digital copy to be generated. For example, the predetermined format may be a JSON format.

[0090] In step 305, the device 102 determines whether a digital copy has been previously created for the given real instance or not. Such a determination may consist of checking in the storage device 104, or in an internal memory of the device 102, if at least some of the information relating to the given real instance, matches relative information included in a previously generated digital copy. For example, if the type information and location information, indicating for example a site or a room, of the given real instance, matches similar information stored in a previously generated digital copy, the device 102 can determine that the previously generated digital copy is the digital copy of the given real instance. No restriction is attached to the information, or information, relating to the given real instance which is / are used to check whether a digital copy has already been generated or not for the given real instance. If the device 102 determines that no digital copy has been previously created for the given real instance, the device 102 generates, in step 305, a digital copy of the real instance, the digital copy comprising at least one piece of information relating to the given real instance, the at least one piece of information relating to the given real instance being one or any combination of the following information: - the location information determined in step 301; - the type information determined in step 302; - the at least one visible characteristic determined in step 303; - at least one piece of information complementary to the visible characteristic determined in step 304; and / or - at least one internal and / or functional characteristic determined in step 304, optionally according to the predetermined format.

[0091] The generated digital copy may be stored in an internal memory of the device 102 or may be stored in the digital copy storage device 104.

[0092] If the device 102 determines in step 305 that a digital copy already exists for the given real instance, the device 102 may update the digital copy based on the at least one characteristic relating to the given real instance. Such an update may comprise: - detecting a difference between a value indicated by the information relating to the given real instance, and determined in one of steps 301 to 304, and a stored value of the digital copy for this same information. Such a stored value may come from a previous iteration of the method implemented for the same given real instance, either during a previous update of the digital copy associated with the given real instance, or during the generation of the digital copy associated with the given real instance;

[0093] - modifying / updating the value stored in the digital copy with the value determined in one of steps 301 to 304. The modified stored value can thus be updated in a local memory of the device 102 and / or transmitted to the device storage 104 for updating the digital copy associated with the given real instance.

[0094] Alternatively or additionally, at least one piece of information relating to the given real instance determined in one of steps 301 to 304 is not included in the digital copy previously generated for the given real instance, in which case the update comprises the addition of the information not yet included in the digital copy.

[0095] The invention thus allows an automatic update of some of the information stored in the digital copy. The steps of the method can therefore be iterated, by being applied to images, or series of images, acquired at different times by the same camera 101.

[0096] For example, during a previous iteration of the method applied to the extinguisher 110, the digital copy may indicate a “not used” use state. During a current iteration, the third image analysis of step 303 may determine a “used” use state, for example because an element of the activation mechanism of the extinguisher 110 is in a position indicating that the extinguisher has been activated and used. In this case, during step 305 of the current iteration, the device 102 detects a difference between the use state stored in the digital copy from the previous iteration, which is an “not used” state, and a use state determined in step 303 of the current iteration.

[0097] According to the invention, the update may apply to other information relating to the given real instance than its state of use. For example, an update of the digital copy may be carried out for any information indicating a change of location and / or a change of shape, and / or for any other change.

[0098] Note that in the event of absence of identification of a real instance corresponding to a previously generated digital copy (during a previous iteration of the steps of the method) in the image obtained in step 300, the device 102 can delete the previously generated digital copy, from its internal memory or from the storage device 104.

[0099] After generating and storing the digital copy for the given real instance, or updating the digital copy of the given real instance, steps 302 to 304 may be implemented for at least one other real instance detected in the at least one image received in step 300. Alternatively, steps 302 to 304 are applied in parallel to the given real instance and to the at least one other real instance.

[0100] Multiple digital copy generations or updates may be implemented in parallel, or sequentially one after the other.

[0101] As described previously, any of the combinations of steps 302 to 304 may be implemented prior to generating or updating the digital copy in step 305 according to the invention.

[0102] In a first embodiment, the device 102 is able to implement steps 300 and 301, step 302 of identifying type information, then to implement step 305 after step 302 (without implementing steps 303 and 304). In this case, the digital copy of the given real instance generated or updated in step 305 may comprise the type information, and optionally the location information determined in step 301.

[0103] In a second embodiment, the device 102 is able to implement steps 300 and 301, step 302 and step 303, and then to implement step 305 after step 303 (without implementing step 304). In this case, the digital copy of the given real instance generated or updated in step 305 may comprise: - the type information determined in step 302; - the location information determined in step 301; and / or - the at least one visible characteristic determined in step 303.

[0104] In a third embodiment, the device is able to implement steps 300 and 301, step 302, then step 304, then step 305 after step 304. In this case, the digital copy of the given real instance generated or updated in step 305 may comprise:

[0105] - the type information determined in step 302; - the location information determined in step 301; and / or - at least one internal and / or functional characteristic determined in step 304; - at least one complementary characteristic of a visible characteristic, determined in step 304.

[0106] In the third embodiment, the type information determined in step 302 can be used to generate the at least one instruction from which the database 105 is queried to obtain the at least one internal and / or functional characteristic, or the at least one complementary characteristic, determined in step 304.

[0107] In a fourth embodiment, the device 102 is capable of implementing all of steps 300 to 304 before implementing step 305. In this case, the digital copy of the given real instance generated or updated in step 305 may comprise:

[0108] - the type information determined in step 302; - the location information determined in step 301; - the at least one visible characteristic determined in step 303; - at least one internal and / or functional characteristic determined in step 304; - at least one piece of complementary information of the at least one visible characteristic, determined in step 304.

[0109] It should be noted that the at least one visible characteristic determined in step 303 and / or the type information determined in step 302, may not be integrated into the generated or updated digital copy, but may be used by the device 102 as data representative of the given real instance to generate the at least one instruction during step 304, and thus to obtain the at least one internal and / or functional characteristic, and / or the at least one complementary information, during step 304.

[0110] In particular, when a text written on the given real instance is extracted in step 303, the device 102 can use the text to generate the at least one instruction, or prompt, in step 304, without including the extracted text in the digital copy in step 305.

[0111] For all of the embodiments described above, each piece of information relating to the given real instance may be determined in association with a confidence score. The confidence score is representative of the degree of certainty with which the information relating to the given real instance is determined. Each confidence score determined in association with a piece of information relating to the given real instance may be integrated into the generated or updated digital copy of the given real instance, if the information relating to the given real instance is integrated therein. The confidence score may be expressed as a percentage.

[0112] In the following, two detailed examples of generating a digital copy are given for illustrative purposes only.

[0113] In a first example, the given real instance is the first real instance 110 of [Fig.l], which is a fire extinguisher.

[0114] Creating a digital copy of a fire extinguisher is useful in the context of fire safety training, management and location of safety equipment, or fire simulation for training. Manual generation according to the prior art of a digital copy of a fire extinguisher may require skills in 3D modeling, software development and data integration. In particular, it may be desired to create a digital copy modeling the fire extinguisher 110, and reflecting its characteristics, dimensions, condition and location.

[0115] Examples of information relating to a fire extinguisher that may be incorporated into a digital copy are given below: - physical properties (internal characteristics): weight, capacity, type of liquid or gas it contains, pressure; - condition of the extinguisher (internal / functional characteristic): condition among a set of predetermined conditions (new, used), date of last inspection; - location (location information): descriptive information of an area of ​​a building in which the fire extinguisher is located; - interactions with the digital copy (functional characteristics): it is thus made it possible to create an interactive digital copy, to allow a user to grasp, manipulate, place, activate the mechanism of the digital copy of the extinguisher; - maintenance data (internal / functional characteristics): data from a last inspection of the extinguisher, and date of the next planned inspection; - integration with other systems: if the extinguisher is used in a broad context such as fire safety management of a building in which it is located. In this case, the extinguisher object is a component of a broader system of which a digital copy can be created, requiring the definition of digital copies for the other components of the management system; - three-dimensional modeling from visible and internal characteristics of the extinguisher, such as its dimensions, shape, design and components; - visualization from visible and internal characteristics of the extinguisher: the digital copy can be used with visualization software allowing to display and interact with the digital copy, on computer, smartphone, tablet or virtual reality headset for example.

[0116] Thus, a digital copy template can be predefined from any combination of the fire extinguisher information given above, and the analyses and tools implemented in the device 102 are provided to obtain the fire extinguisher information provided in the predefined template.

[0117] In the first example, an image 200 comprising the fire extinguisher 110 is received at step 300. At a step 301, the fire extinguisher is detected in the image 200, in a first portion 210 of the image 200.

[0118] The device 102 implements step 301 and determines information of the type “packaged goods” in association with a confidence score of 82%. In this first example, step 301 does not make it possible to determine a more precise type (“fire extinguisher”, or “fire safety device”, for example), than the type “packaged goods”. The level of certainty associated with the type information is however high.

[0119] In the first example, the device 102 implements step 302, based on OCR software, and extracts the text 111 written on the extinguisher 110, from the first part 210 of the image 200.

[0120] In the first example, the device 102 implements step 303, by generating an instruction comprising the text 111 extracted in the previous step, and by submitting the instruction to a generative AI API, in order to obtain a response comprising the following internal and / or functional characteristics, and complementary information of visible characteristics: - the type of extinguisher (additional information to the visible characteristics): ex- ABC type powder tinker; - a fire extinguisher brand (additional information on visible characteristics); - a volume of 2kg (internal characteristic); - a type of ABC powder (internal characteristic); - a use in which the extinguisher is suitable for use on class A fires, solid fuels, B, flammable liquids, and C, gases (functional characteristics); - instructions for using the 110 fire extinguisher (functional characteristics).

[0121] An additional instruction may be generated by the device 102 to request the exact dimensions of the fire extinguisher (complementary characteristics of visible characteristics), associated with the brand and type of fire extinguisher identified in the response received to the previous instruction. The response to the additional instruction thus includes the dimensions of the fire extinguisher 110, which allows the production of a three-dimensional representation of the fire extinguisher 110 from its generated digital copy.

[0122] As described previously, the instruction and the additional instruction may indicate a response format corresponding to the predetermined format for creating the digital copy, such as the JSON format.

[0123] The digital copy of the fire extinguisher 110 can thus be generated in step 305 from the information relating to the fire extinguisher 110 obtained during step 304. It is in fact not necessary to add the information of the “packaged goods” type determined in step 302, since more precise type information was obtained in step 304, nor to include the text 111 extracted in step 303, which was used to generate the instruction.

[0124] As previously indicated, the digital copy may further comprise a usage state of the fire extinguisher 110. When generating the digital copy described herein, the usage state determined may be an “unused” state. The digital copy of the fire extinguisher 110 may be updated during a subsequent iteration of the method, if a state other than the “unused” state is detected, for example a “used” state.

[0125] In a second example, the given real instance is the second real instance 112 of [Fig.l], which is a chair.

[0126] In the second example, the device 102 can detect the chair 112 in the second part 212 of the image 200 at step 301, by applying the first image analysis to the image 200.

[0127] In step 302, the "chair" type information may be determined by applying the second image analysis. The "chair" type information may be determined in association with a high confidence score, for example 94%.

[0128] According to the second example, the device 102 can implement step 304 after step 302 (without performing step 303), by generating an instruction for a generative AI API, the instruction comprising the second part 212 of the image 200 as data representative of the chair 112. Indeed, certain generative AI APIs are able to receive an image as input, and to return a response to the instruction, which may comprise the following internal / functional characteristics and complementary characteristics of visible characteristics: - the dimensions of the chair 112 and / or of each of the components of the chair, such as the legs, the seat, the frame, etc. (characteristics complementary to visible characteristics); - a material of the chair and / or a material for each component of the chair (additional information of visible information and internal information); - a model and / or brand of the chair (complementary characteristic of visible characteristics); - a type of chair design (complementary characteristic of visible features); - a type of tissue (internal characteristic).

[0129] Each of the above information can be associated with a confidence score indicated in the response.

[0130] As for the fire extinguisher 110, the digital copy associated with the chair 112 may also be updated during subsequent iterations of the steps of the method according to the invention. For example, when generating the digital copy, an occupancy state of “occupied” may be determined at step 303 if a person is sitting on the chair in the image 200. At a subsequent iteration of the method, upon obtaining a new image or series of images at step 300, the device 102 may determine an occupancy state different from the occupied state, for example an “empty” state, if no one is sitting on the chair in the new image or series of images of the subsequent iteration. In the event of no detection of the chair in the new image or series of images of the subsequent iteration, the device 102 may delete the digital copy associated with the chair, in its internal memory and / or from the storage device 104.

[0131] The above examples are given for illustrative purposes. The invention can be applied to the detection of real instances in any environment 120, not only inside a building. The environment 120 can for example be: - a public park, the real instances being trees; - a public space, the actual instances being elements of urban furniture; - a network infrastructure, with the actual instances being elements of the network infrastructure, etc.

[0132] [Fig.4] shows the structure of the device 102 for creating a digital copy, according to embodiments of the invention.

[0133] The device 102 comprises a processor 401 configured to communicate unidirectionally or bidirectionally, via one or more buses or via a direct wired connection, with a memory 402 such as a memory of the “Random Access Memory” type, RAM, or a memory of the “Read Only Memory” type, ROM, or any other type of memory (Flash, EEPROM, etc.). Alternatively, the memory 402 comprises several memories of the aforementioned types.

[0134] The memory 402 comprises at least one non-volatile memory in which the data used and / or resulting from the implementation of the steps of the method described with reference to [Fig.3] are stored, temporarily or permanently.

[0135] In particular, the memory 402 may store, according to certain embodiments, the algorithm(s) / software implementing the first image analysis, the second image analysis and / or the third image analysis previously described. The memory 402 may also store instruction formats corresponding to a digital copy template, for the implementation of the step 304 described previously.

[0136] Alternatively, the device 102 comprises separate modules for implementing the steps 301 to 304 described above. The device 102 can thus comprise any combination of the following modules: - a first module configured for the implementation of step 301; - a second module configured for the implementation of step 302; - a third module configured for the implementation of step 303; - a fourth module configured for the implementation of step 304.

[0137] The processor 401 is capable of executing instructions, stored in the memory 402, for the implementation of the steps 300 to 305 described with reference to [Fig.3].

[0138] The device 102 comprises a first interface 403 capable of communicating with the camera 101, in particular for obtaining at least one image during the step 300 described previously.

[0139] The device 102 may further comprise a second interface 404 capable of accessing the wide area network 103 and / or the local area network of the device 102, to communicate remotely with entities accessing the network, in particular with the storage device 104 of digital copies for transmission of a generated digital copy or for updating or deleting a digital copy stored in the storage device 104, but also with the APIs 106 and 107.

Claims

Claims

1. Method for generating a digital copy of a real instance (110; 112), the method comprising the following steps: - obtaining (300) at least one image (200) representative of an environment (120); - detecting (301) at least one given real instance in said at least one received image; - obtaining (302-304) at least one piece of information relating to the given real instance; - generating (305) a digital copy of said given real instance, the digital copy comprising said at least one piece of information relating to the given real instance.

2. Method according to claim 1, wherein said at least one information relating to the given real instance comprises location information of the given real instance and / or type information representative of a type of the given real instance.

3. A method according to claim 2, wherein obtaining the location information and / or the type information comprises an image analysis of the at least one received image (200).

4. Method according to one of the preceding claims, wherein the at least one information comprises at least one visible characteristic of the given real instance (110; 112), comprising: - a size, a color, a brand and / or a model of the given real instance; and / or - a text written (111) on the given real instance; and / or - an interaction of the given real instance with at least one other real instance of the environment.

5. A method according to claim 4, wherein the visible feature is identified by image analysis of a portion (210; 212) of the at least one image (200) comprising the given actual instance.

6. Method according to one of the preceding claims, wherein said at least one information comprises at least one internal and / or functional characteristic of the given real instance (110; 112), obtained from a resource (105) storing data distinct from the at least one image (200) obtained.

7. Method according to claim 6, wherein the at least one internal and / or functional characteristic is obtained from said resource (105) via a generative artificial intelligence interface programming interface (106; 107).

8. Method according to claim 6 or 7, wherein obtaining (304) said at least one internal and / or functional characteristic comprises: - generating at least one instruction based on data representative of the given real instance (110; 112), the data representative of the given real instance being obtained from said at least one image; - obtaining at least one response, following a transmission of the at least one generated instruction, the response comprising said at least one technical and / or functional characteristic.

9. Method according to claim 8, in which the data representative of the given real instance is: - a part (210; 212) of the at least one image (200) comprising the given real instance (110; 112); or - information relating to the given real instance, resulting from an image analysis applied to a part of the at least one image comprising the given real instance.

10. A method according to claim 9 and claim 5, wherein the instruction is generated from a given visible characteristic resulting from the image analysis applied to the part (210; 212) of the at least one image (200) comprising the given real instance (110; 112), the given visible characteristic being the text (111) written on the given real instance.

11. Method according to one of claims 8 to 10, wherein the generated instruction further indicates a given format, the given format being a predetermined format associated with the digital copy, wherein the response comprises said at least one technical and / or functional characteristic according to the given format and wherein the digital copy is generated (305) according to the given format.

12. Method according to one of the preceding claims, further comprising: - detecting (301) at least one other real instance in said at least one image obtained in addition to the given real instance; - obtaining (302-304) at least one information relating to the other real instance; - generating (305) a digital copy of said other real instance given, the digital copy comprising said at least one piece of information relating to the other real instance.

13. Method according to one of the preceding claims, comprising obtaining at least one new image, acquired after said at least one image, the method further comprising: - detecting (301) at least one new given real instance in said at least one new obtained image; - obtaining (302-304) at least one new information relating to the new given real instance; - in the event of determining that the new given real instance and the given real instance are the same real instance, updating (305) the digital copy of the given real instance according to the new information obtained.

14. Device (102) for generating a digital copy of a real instance, the device comprising an interface (403) configured to obtain at least one image (200) representative of an environment (120), and a processor (401) configured to: - detect at least one given real instance (110; 112) in said at least one received image; - obtain at least one piece of information relating to the given real instance; - generate a digital copy of said given real instance, the digital copy comprising said at least one piece of information relating to the given real instance.

15. A computer program implementable in a device as defined in claim 14, the program comprising code instructions which, when executed by a processor (401), performs the steps of the method defined in one of claims 1 to 13.

Images