Computerized scenario creation system for external systems

The computer creation system automates scenario creation for external systems by using an algorithmic device and intelligent module to process raw data into interactive 3D instructions or digital twins, addressing complexity and cost issues, and enhancing user interaction.

FR3154205A1Pending Publication Date: 2025-04-18TWINWORKS SAS
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
FR2023011163
Authority / Receiving Office
FR · FR
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-10-16
Publication Date
2025-04-18

AI Technical Summary

Technical Problem

Existing systems for creating scenarios for external systems, such as industrial machines or factories, are time-consuming, complex, and expensive due to the need for manual intervention, dedicated software, and non-flexible solutions, with difficulties in interfacing with sensors and APIs.

Method used

A computer creation system utilizing an algorithmic device and intelligent module to automate scenario creation from raw data, incorporating a digital learning model to process and format data into interactive 3D instructions or digital twins, reducing manual effort and enhancing flexibility.

Benefits of technology

The system significantly reduces creation time by up to 90%, improves accuracy and efficiency, and facilitates intuitive interaction with external systems through immersive 3D guides, while maintaining flexibility and cost-effectiveness.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

COMPUTER SYSTEM FOR SCENARIO CREATION FOR EXTERNAL SYSTEM The invention relates to a computer system for creating scenarios for interaction with an external system based on raw data (14a, 14b, 14c, 14d) relating to at least one component of this external system, comprising an algorithmic device (16) including a module (16a) for receiving said raw data (14a, 14b, 14c, 14d), user input data, a data structure and a desired scenario framework, and a module (16b) for creating a textual query (20), the computer system also including an intelligent module (22) including a numerical learning model (24) configured to receive said textual query (20) as input data and to generate processed data representative of the desired scenario as output data,and in that the algorithmic device (16) is further configured to receive the processed data and format it for user interaction according to the created scenario. Figure for the abstract: Figure 1,
Need to check novelty before this filing date? Find Prior Art

Description

Title of the invention: COMPUTER SCENARIO CREATION SYSTEM FOR EXTERNAL SYSTEMS Technical field of the invention

[0001] The invention relates to a computer system for creating scenarios for an external system. In particular, the invention relates to a computer system for creating scenarios that allows the creation of different scenarios from raw data representing one or more components of the external system, for example, assembly, repair, etc. instructions for the external system, or the labeling and categorization of raw data relating to the external system, or the creation of a digital twin of one or more components of the external system or even of the external system as a whole. Technological background

[0002] The invention falls within the field of digital transformation and advanced human-machine interaction. In particular, the invention is part of the Industry 4.0 concept, which refers to a desire for increased automation of industrial processes and the extensive use of machine learning methods.

[0003] In particular, the invention seeks to automate the creation of scenarios linked to an external system such as an object, an industrial machine or even an entire factory, these scenarios being able to be in particular the creation of instructions linked to a 3D model of the external system or the creation of a digital twin for an interaction with a user.

[0004] The creation of interactive 3D instructions is a prime example of a challenge posed by Industry 4.0. Augmented reality devices, in particular, can assist operators in implementing maintenance or repair procedures for the systems they are required to work on. While the interfaces for following these interactive instructions are user-friendly, their implementation is nonetheless time-consuming and requires the use of complex, dedicated software.

[0005] The creation of digital twins also presents disadvantages in terms of development time and software complexity. Furthermore, dedicated solutions are generally inflexible and very expensive due to the difficulty of developing modules specific to each type of external system. Another difficulty with digital twins is the implementation of the various interfaces with sensors, which generally rely on APIs or communication buses whose configuration is complex to to be implemented by a user who is not a specialist in the API or the communication bus.

[0006] The inventors therefore sought to provide a solution to overcome these drawbacks, and in particular to automate the creation of scenarios. Objectives of the invention

[0007] The invention aims to provide a scenario creation system enabling the automation of the creation of scenarios dedicated to an external system, from raw data relating to at least one component of this external system.

[0008] The invention aims in particular to provide, in at least one embodiment, a scenario creation system reducing the number of manual interventions for scenario creation.

[0009] The invention aims in particular to provide, in at least one embodiment, a scenario creation system configured to produce interactive instruction lists on a 3D model of the external system or of one or more of its components.

[0010] The invention aims in particular to provide, in at least one embodiment, a scenario creation system configured to simplify the generation of a digital twin of the external system or of one or more of its components. Description of the invention

[0011] To this end, the invention relates to a computer system for creating scenarios for a user's interaction with an external system based on raw data relating to at least one component of that external system, characterized in that it comprises a computer device, referred to as an algorithmic device, comprising at least one processor and: - a receiving module that receives raw data, user input data, a predetermined data structure, and a framework of at least one desired scenario from one or more databases. - a module for creating a text query adapted to the desired scenario and the external system, based on a pre-configured query text, the predetermined data structure, the structured dataset, and the scenario framework,

[0012] in that the computer creation system further comprises an intelligent module including at least one numerical learning model trained from a generic and / or specific dataset and configured to receive said textual query as input data and to generate as output data processed data representative of the desired scenario in which the raw data conform to the predetermined data structure according to the scenario framework desired,

[0013] and in that the algorithmic device is further configured to receive the processed data and to format said processed data to provide data intended for user interaction with the external system according to the scenario created.

[0014] A scenario creation computer system according to the invention thus makes it possible to obtain, from raw data, a scenario relating to an external system that can be formatted by the algorithmic device for interaction with a user. The formatting of the textual query by the algorithmic device allows for simplified, personalized, and robust processing of the raw data to obtain output data that can be easily used by the algorithmic device to implement this user interaction solution linked to the scenario. The textual query is generally known as a prompt. The textual query can include other data in addition to text, such as image data, 3D model data, etc., when the training model is a multimodal model capable of handling this type of data.

[0015] The creation system therefore makes it possible to autonomously and intuitively transform heterogeneous data, whether textual, visual (2D images), 3D models, etc., into a scenario, for example a sequence of interactive and animated 3D instructions, or an interface for the integration, editing and dynamic adaptation of digital twins.

[0016] The creation system is based on a bifunctional architecture which is based on the algorithmic device in bidirectional exchange with the intelligent module for optimized processing of the raw input data of the system and the output data resulting from the processing by the machine learning model.

[0017] The intelligent module is notably responsible for the interpretation, categorization, labeling and conversion of raw data into an intelligible file that can be processed by the algorithmic device.

[0018] The algorithmic device is notably responsible for the recognition, naming, classification and description of the different components of the external system in the 3D models in order to create the link between these elements and the raw data which concern it, in particular the textual data.

[0019] In particular, the predetermined data structure and scenario framework provide the intelligent module with a skeleton to which the raw data is conformed after processing by the intelligent module. The data structure and scenario framework are, for example, implemented as a data structure such as that used in an object-oriented programming language, allowing the categorization of the components of the external system, their associated parameters, and the possible interactions during scenario implementation.

[0020] Using the system to create interactive instructions provides the user with a more immersive and efficient learning and application experience. Integrating a machine learning model could result in time savings of up to 90% on content creation. In particular, a raw list of instructions written in natural language is segmented and classified into different distinct steps, and options relating to the components of the external system can be selected, for example, a display color, a component state (open, closed, on, off, etc.).

[0021] The intelligence module can also, from the scenario framework, select display options such as visual aids, arrows targeting a component, animations, representations of the component's state, and associated parameters.

[0022] The output data of the intelligent module conforms to the provided data structure and scenario framework and is therefore easily interpreted by the algorithmic device. The output data is, for example, embedded in a text file in JSON or XML format. This output data is then processed by the algorithmic device using a set of intelligent tools and scripts, such as algorithms for preventing part collisions, selecting view orientations, choosing the screwing direction, etc.

[0023] For the generation of digital twins, the creation system makes it possible to automate the process of creating and adjusting digital twins, by integrating pre-programmed modules, allowing in particular communication with the APIs of the different components, in a consistent and real-time manner, while allowing the user to interact in natural language with the system.

[0024] The invention also notably enables a significant reduction in instruction creation time, and improves the accuracy and efficiency of the instruction generation process by integrating ergonomic standards into the creation process. Another advantage is the unification of the creation and interpretation of an interactive guide (for example, red will be used to indicate a part that should not be touched, and green for parts with which interaction will occur during a specific step). The invention also offers greater flexibility in scenario creation, thanks to its ability to handle a variety of data types, as well as more robust and diverse data processing.By enabling the transformation of traditional manuals into interactive 3D guides, the creation system facilitates a more intuitive understanding and learning of procedures, thus improving the user experience and the efficiency of product use.

[0025] The creation system also makes it possible to provide a user interface that It combines speech recognition and speech synthesis to create an interactive dialogue between the user and the system, while also incorporating visual aids into the scenario. This interactivity gives the impression of having a conversation with an intelligent assistant, allowing for more efficient resolution of the problems encountered during the operation.

[0026] Throughout the text, a module may, for example, consist of a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components, or, for example, a computer program, a set of computer programs, a library of a computer program or a function of a computer program executed by a computing device such as a computer, a set of computing devices, an electronic component or a set of electronic components.

[0027] Advantageously and according to the invention, the raw data comprises one or more of the following elements: - one or more two-dimensional images of at least one component of the external system, annotated or not, - one or more three-dimensional models of at least one component of the external system, annotated or not, - textual data relating to the external system, - output data from a previous processing by the intelligent module, - pre-programmed modules for generating a digital twin of the external system, pre-programmed modules including modules for modeling the behavior of at least one component of the external system and / or modules for modeling interfacing with at least one component of the external system.

[0028] According to this aspect of the invention, the raw data includes at a minimum text for obtaining the textual query, and preferably a combination of several raw data formats.

[0029] Raw data may also include output data already obtained through the intelligent module of the external system. This output data then constitutes pre-processing of the data, for example, through categorization and labeling, which simplifies raw data for further processing. This labeling and categorization can also reduce the size of the raw data when the text query for the machine learning model is limited in size, or reduce the cost if a machine learning model is implemented on an external server that charges for processing based on data size. The algorithmic device extracts this information from the existing data, for example, the Given existing names in the 3D model, the intelligent module can process the raw data using machine vision recognition and then send this classified, categorized, and ordered information to the algorithmic module for storage and reuse, either directly or after processing by the intelligent module. The algorithmic device's function in this case is to prepare the raw data (for example, in terms of the amount of text so that it is "processable" by the intelligent module) and send it to the intelligent module for classification, categorization, and so on.

[0030] The pre-programmed modules for generating a digital twin are of different types: - The modeling modules include technical characteristics of at least one component of the system to allow its simulation in the context of the scenario, for example to simulate an output based on input parameters, to simulate the query of sensor values, etc. Interface modeling modules simplify user interaction with the digital twin by providing communication bus or API (Application Programming Interface) information used for communication with the device. The intelligent module takes these interface modeling modules into account to provide a simplified interface, for example, by allowing the user to use natural language adapted to the interface used by the digital twin, based on the settings provided by the interface modeling module.

[0031] These pre-programmed modules can advantageously be provided to the system in the form of an extension module (plugin in English) which can be retrieved and added to the computer creation system according to the user's needs or the scenarios to be created.

[0032] Advantageously and according to the invention, the scenario framework is chosen from one or more of the following scenario frameworks: - a scenario framework of instructions, configured to obtain a list of instructions associated with actions relating to one or more components of the external system, for implementation by a human user, - a scenario framework for labeling and categorizing raw data according to the predetermined data structure to obtain labeled and categorized data suitable for use in the text query, - a digital twin scenario framework, configured to obtain a digital twin of at least one component of the external system, for a simulation of at least one component of the external system and for the supply

[0033]

[0034]

[0035]

[0036]

[0037] of an interface for sending instructions or requests to said simulation of said component. According to this aspect of the invention, the different scenario frameworks allow the result to be adapted to the user's needs. The computer system is capable of adapting to the new frameworks proposed by the user to create the appropriate scenario. According to other variants of the invention, the scenario can for example constitute a route of movement and action to be carried out in the external system which can be a city, a building (for example a factory), etc. In particular, examples of applications include: - Product assembly: The invention can be used to create interactive 3D instructions for product assembly, thereby improving the customer experience and potentially reducing product returns. - Maintenance and repair: The 3D instructions generated by the system can help technicians perform maintenance and repair operations more efficiently and with fewer errors. - Vocational training: The invention can be used in industrial environments to train workers in the assembly of products or the use of complex machines. - Education: Interactive 3D instructions could be used in an educational context to help students understand complex concepts. - Consumer guides: The invention could be used as an alternative to user manuals, given its more expressive nature compared to textual and illustrated instructions. - Museum guided tours: 3D instructions could be used to create interactive guides for museum visitors, thus enhancing the visitor experience. Advantageously and according to the invention, the intelligent module is integrated into an external server accessible via the Internet and in that the creation system includes a communication network intended to transmit the textual request to the intelligent module and to receive as output data the processed data representative of the scenario. According to this aspect of the invention, at least one machine learning model of the intelligent module is hosted on one or more external servers, which can be queried via the internet using a dedicated API. An example of such an implementation is the machine learning models from OpenAI, such as GPT-2, GPT-3, or GPT-4, via its ChatGPT service.

[0038] Advantageously and according to the invention, the intelligent module is integrated into a private server internal to the computer creation system and the machine learning model pre-trained by a dataset is then tuned for scenario creation from textual queries adapted to a plurality of scenarios and a plurality of external systems.

[0039] According to this aspect of the invention, at least one machine learning model can be managed on an internal server and preferably fine-tuned to the needs of the scenario creation computer system. Internal integration of the learning model allows for complete control of the exchanged data and limits the risk of confidentiality issues. An example of such a learning model that can be integrated on an internal server is, for example, the LLaMa machine learning model from Meta AI.

[0040] Advantageously and according to the invention, the machine learning model is a large-scale machine learning model.

[0041] A large-scale machine learning model, referred to in English as "Large Model", refers to a machine learning model comprising a large number of parameters (generally at least one million, in practice several billion or even trillions of parameters), and generally trained with a large dataset, in particular several million, billion, trillion or more words or tokens.

[0042] Advantageously and according to the invention, the large-scale machine learning model is a machine learning model from one or a combination of the following: - a large language model, - a large multimodal model, - a large foundation model, - a model for image processing, in particular pattern recognition in an image and / or image segmentation.

[0043] According to this aspect of the invention, these different types of models can be used alone or in combination to compose the intelligent module. For example, a model intended for image processing can be used to perform a first pass, and the output data of this model is transmitted to a large multimodal model along with the other raw data.

[0044] The large language model, more commonly known as LLM, allows for the processing of textual data. Examples of such models include Google's BERT, OpenAI's GPT-2, GPT-3, and GPT-4, Meta AI's LLaMa and LLaMa 2, Alpaca (derived from LLaMa), etc.

[0045] The large-size multimodal model is better known as LMM for Large Multimodal Model (LMM) refers to a model that can process data of any type, particularly textual, visual, etc. LMMs can, for example, be derived from LLMs, particularly by preprocessing image data to obtain textual data, or be created specifically to be multimodal.

[0046] Large Foundation Models (LFMs) are more commonly known as Large Foundation Models. These models use data of all types, such as multimodal models, and are also by nature very general but specifically designed to be fine-tuned to suit specific situations.

[0047] The invention also relates to a user system for creating scenarios for user interaction with an external system, characterized in that it comprises a computer system for creating the scenario of the invention and a user interface device comprising: - Information input methods configured to allow the user to enter input data, select the scenario and raw data used, preprocess the raw data, and interact with the data intended for user interaction with the external system according to the created scenario, - means of displaying information configured for displaying information relating to the scenario creation system and / or displaying a visual representation of the data intended for user interaction with the external system according to the scenario created.

[0048] A user system according to the invention therefore makes it possible to provide the user with a user interface allowing the user to interact with the computer system for creating the scenario, upstream for the creation of the scenario and downstream for the interaction with the external system according to the scenario created.

[0049] For example, when the scenario involves interactive instructions, the user can act upstream to adjust the desired scenario to their needs, and then downstream implement a process following the interactive instructions created.

[0050] As another example, when the scenario concerns a digital twin, the user can select upstream the components of the external system that will be integrated into the digital twin and select the desired interactions, and downstream interact with the digital twin, preferably in natural language directly from the user system.

[0051] The display means may combine several components, such as a screen, an augmented reality device, a virtual reality device, a mixed reality device. The scenario may be accompanied, for example, by a 3D model of one or more Components of the external system, or the entire external system, can be displayed on the display devices. The displayed data can be managed by software integrated into and dedicated to the computer system, or by external software that receives data from the computer system in a file format readable by that external software.

[0052] The invention also relates to a computer system for creating scenarios and a user system characterized in combination by all or part of the characteristics mentioned above or below. List of figures

[0053] Other objects, features and advantages of the invention will become apparent from the following description, given by way of non-limiting example only, and which refers to the accompanying figures in which: • [Fig.1] is a schematic view representing a computer creation system according to an embodiment of the invention and the various data processing operations carried out by this creation system.

[0054] Detailed description of an embodiment of the invention

[0055] In the figures, the scales and proportions are not strictly respected for the purposes of illustration and clarity.

[0056] Identical, similar or analogous elements are designated by the same references in all figures.

[0057] Fig. 1 schematically represents a computer creation system according to an embodiment of the invention and the various data processing operations carried out by this creation system.

[0058] The computer system 10 enables the creation of scenarios for user interaction with an external system 12 based on raw data 14a, 14b, 14c, 14d relating to at least one component of that external system. The system 10 includes, in particular, a computer device, referred to as an algorithmic device 16, comprising at least one processor. The raw data are, for example, one or more two-dimensional images 14a, one or more three-dimensional models 14b, textual data 14c, and any other type of data 14d, in particular pre-configured modules for creating digital twins.

[0059] The algorithmic device 16 further comprises a receiving module 16a from one or more databases 18 of said raw data, user input data, a predetermined data structure, and a frame of at least one desired scenario. The database 18 may in particular include the data 14a, 14b, 14c, 14d, even though they are schematically represented here outside the database.

[0060] The algorithmic device 16 also includes a module 16b for creating a Query 20 textual adapted to the desired scenario and the external system, from a pre-configured query text, the predetermined data structure, the structured dataset and the scenario framework.

[0061] The computer creation system 10 further comprises an intelligent module 22 including at least one numerical learning model 24, preferably a large numerical learning model, trained on a generic and / or specific dataset and configured to receive as input data the textual query 20 and to generate as output data processed data 26 representative of the desired scenario in which the raw data conforms to the predetermined data structure according to the desired scenario framework. If necessary, the intelligent module 22 transmits the data exchange format 28 to the algorithmic device 16 so that the creation of the textual query 20 conforms to the received format. The intelligent module 22 may be hosted on an internal private server or managed by an external server accessible from the Internet.

[0062] A user system, not shown, may include a computer system as described and allow interaction with a user and the display of raw and processed data.

[0063] The text query is, in particular, a prompt-type query comprising one or more messages and giving the intelligent module instructions for processing the raw data. A simplified example of a prompt in French for generating, in the manner of an object-oriented program, a scenario (class "Scenario") comprising a list of steps (class "Steps") associated with components (class "Components") is, for example, the following: - A first message intended to define the classes of the objects to be processed, that is to say, the data structure: "New message {" the text query must be transcribed into a JSON format which takes the form of a "Scenario" object with the following structure: class: class Scenario { string Product-Name; string ScenarioTitle; List<steps> Steps;} class Steps { string StepTitle; string TextDescription; List <composant>Components;} class Component { string ComponentName; bool ValueSetting; string _material; string _color; string VisualIndication; string _VisualIndicationPosition; string _VisualIndicationOrientation;}"}. - A message containing the instructions to be processed: New message {"The instructions are as follows:" +ScenarioQuery.Instructions} - A message indicating the components of the external system: New message {"The components linked to the steps must be selected and highlighted" +ScenarioQuery.ComponentName} - A message indicating the colors to be chosen for the highlighted components: New message {"If components are selected, the highlighting is done by default and a color is chosen, green for components to be used or moved, blue to attract attention and red for safety information"} - A message regarding the visual cues that can be integrated: New message {"The visual cue choices are as follows"+ScenarioQuery.VisualCallbacks+ ", they will be attached to the relevant component (example: PressButton to button type components"} - A message regarding color choices: New message {"The color choices are as follows"+ScenarioQuery.ColorList+ ", which are the colors of the highlighted rooms"} - A message allowing the integration of a user-supplied request: New message {RequestScenario.UserPrompt}

[0064] In the query, the parts between quotes are strings of characters which are advantageously concatenated with raw data from the Scenario or ScenarioQuery classes which are integrated into the data structure.

[0065] The text query may advantageously be written in French, English, or another language if the learning model is more effective in that language, particularly if the learning model has been trained with a larger volume of words in that language. The text query may also include image data, 3D model data, etc., in addition to the text.

[0066] Blocks 30, 32, and 34 schematically represent the selection of the scenario to be created, which takes place in the intelligent module 22. In step 30, the desired scenario is selected. An example of a first scenario 32 could be a scenario of interactive instructions to be provided to the user. An example of a second scenario 34 could be a scenario of data categorization and labeling. Other examples of scenarios include, for example, the creation of digital twins.

[0067] The algorithmic device 16 is further configured to receive the processed data 26 and to format said processed data 26 to provide data 36 intended for user interaction with the external system according to the created scenario. In particular, the data is accessible to a user 40 via a user system 38 comprising interaction means and display means.

[0068] When the scenario involves the creation of a digital twin or interaction with a digital twin, the raw data includes pre-programmed modules of various types dedicated to this creation or interaction. In particular, the modules may be, for example: - Modeling modules for the behavior of at least one component of the external system, in particular for sensors, actuators, communication interfaces and other elements of the product architecture - Modeling modules for interfacing with at least one external system component; that is, modules that can automatically integrate with the product's APIs for easy and dynamic data access. Specifically, these modules have interfaces that can be linked to various types of sensors (temperature, pressure, etc.), actuators (motors, cylinders, etc.), and other APIs to enable a wide variety of configurations. These modules also allow integration with the product's APIs for easy and dynamic data access.

[0069] Raw data may also include pre-built simulation environments that can be attached to real-time sensors for data visualization in various potential forms (3D model, 2D, graph, curves, spreadsheet, etc.).

[0070] A possible embodiment for the implementation of a digital twin scenario is for example based on a graphical language or visual scripting or visual programming implementation, where the pre-programmed modules are represented by nodes, whether they are nodes representing components of the external system, nodes representing sensors or nodes representing interfaces.

[0071] The nodes are, for example: - nodes of a data layer, including sensors, for example temperature, pressure, etc. sensors. - Nodes of a communication layer, configured for defining real-time communication protocols and communication security protocols, - nodes of a model layer, including 2D models, 3D models, semantic models, etc. - nodes of an analysis layer, configured for data analytics, machine learning, diagnostics, and other types of analysis, - nodes of an action layer, configured for control adjustments, maintenance plans, or other actions.

[0072] The intelligent module then acts as a controller in the orchestration of the nodes, linking them appropriately according to the scenario and user needs. The intelligent module can also generate or control node parameters in real time. For example, if a user wants to monitor the temperature, the An intelligent module can generate an instance of a temperature sensor node and connect it to the 3D model to display this data. In one embodiment of the invention, the graphical language implementation is visually accessible to the user, who can modify it within their user system interface.

[0073] The intelligent module can also generate an output assistant that processes user requests in natural language and data in various forms (text, image, 3D models, etc.) in a procedure for integrating a component into an existing digital twin or creating a new digital twin.

[0074] The user can interact with the system to add, remove or adjust modules, and the intelligent module acts accordingly to rearrange and adjust the digital twin and produce suggestions to the user.

[0075] The algorithmic device includes everything that is programmed and pre-programmed as tools created in classical code, while the intelligent module is responsible for organizing data and raw user queries to simplify communication and completing user intentions to integrate an external system within the digital twin of the target entity, thus creating a complete and flexible architecture.< / composant>

Claims

Claims

1. Computer system for creating a scenario for the interaction of a user with an external system from raw data (14a, 14b, 14c, 14d) relating to at least one component of this external system, characterized in that it comprises a computer device called an algorithmic device (16) comprising at least one processor and: - a module (16a) for receiving from one or more databases (18) said raw data (14a, 14b, 14c, 14d), input data from a user, a predetermined data structure and a frame of at least one desired scenario, - a module (16b) for creating a text query (20) adapted to said desired scenario and to the external system, from a preconfigured query text, the predetermined data structure, the structured data set and the scenario frame, in that the creation computer system further comprises an intelligent module (22) comprising at least one digital learning model (24) trained from a generic and / or specific data set and configured to receive as input data said textual query (20) and to generate as output data processed data representative of the desired scenario in which the raw data (14a, 14b, 14c, 14d) are conformed to the predetermined data structure according to the desired scenario frame, and in that the algorithmic device (16) is further configured to receive the processed data and format said processed data to provide data intended for the interaction of the user with the external system according to the created scenario.

2. A computer system for creating scenarios according to claim 1, characterized in that the raw data (14a, 14b, 14c, 14d) comprises one or more of the following elements: - one or more two-dimensional images of at least one component of the external system, annotated or not, - one or more three-dimensional models of at least one component of the external system, annotated or not, - textual data relating to the external system, - output data from a previous processing by the intelligent module, - pre-programmed modules for generating a digital twin of the external system, the pre-programmed modules comprising modules for modeling the behavior of at least one component of the external system and / or modules for modeling interfacing with at least one component of the external system.

3. Scenario creation computer system according to one of claims 1 or 2, characterized in that the scenario frame is chosen from one or more of the following scenario frames: - an instruction scenario frame, configured to obtain a list of instructions associated with actions relating to one or more components of the external system, for implementation by a human user, - a raw data labeling and categorization scenario frame according to the predetermined data structure for obtaining labeled and categorized data suitable for use in the text query, - a digital twin scenario frame, configured to obtain a digital twin of at least one component of the external system,for a simulation of at least one component of the external system and for providing an interface for sending instructions or requests to said simulation of said component.,

4. Computer system for creating scenarios according to one of claims 1 to 3, characterized in that the intelligent module (22) is integrated into an external server accessible via the Internet and in that the creation system comprises a communication network intended to transmit the textual request (20) to the intelligent module (22) and to receive as output data the processed data representative of the scenario.

5. Computer system for creating scenarios according to one of the claims- indications 1 to 3, characterized in that the intelligent module (22) is integrated into a private server internal to the creation computer system and the machine learning model (24) pre-trained by a data set is then adjusted for the creation of scenarios from textual queries adapted to a plurality of scenarios and a plurality of external systems.

6. Computer system for creating scenarios according to one of claims 1 to 5, characterized in that the machine learning model (24) is a large machine learning model.

7. Computer system for creating scenarios according to claim 6, characterized in that the large-scale machine learning model (24) is a machine learning model from one or a combination of the elements of the following list: - a large-scale language model, - a large-scale multimodal model, - a large-scale foundation model, - a model intended for image processing, in particular recognition of shapes in an image and / or image segmentation.

8. User system for creating scenarios for user interaction with an external system, characterized in that it comprises a computer system (10) for creating scenarios according to one of claims 1 to 7 and a user interface device comprising: - information input means configured to allow the user to enter input data, select the scenario and the raw data used, preprocess the raw data, and interact with the data intended for the user's interaction with the external system according to the created scenario, - information display means configured to display information relating to the scenario creation system and / or display a visual representation of the data intended for the user's interaction with the external system according to the created scenario.

Citation Information

Patent Citations

  • Augmented reality virtual personal assistant for external representation

    US20140310595A1

  • Systems and methods for integrating and using augmented reality technologies

    US20220139049A1