Method and system for generating a digital assistant for assisting an operator in a manufacturing process

Abstract

The present invention relates to a method for generating a digital assistant for assisting an operator in a manufacturing process, as well as to the corresponding system for generating a digital assistant. According to the invention, the method, through the dynamic decomposition of industrial documents commonly available in manufacturing companies (such as bills of materials and routing sheets), and the acquisition of images via camera, automatically generates a digital assistant.

Description

[0001] “METHOD AND SYSTEM FOR GENERATING A DIGITAL ASSISTANT FOR ASSISTING AN OPERATOR IN A MANUFACTURING PROCESS”

[0002] FIELD OF THE INVENTION

[0003] The present invention relates to a method for generating a digital assistant for assisting an operator in a manufacturing process, as well as to the corresponding system for generating a digital assistant.

[0004] BACKGROUND OF THE INVENTION

[0005] Currently, in industry, the adoption of advanced digital technologies is increasingly widespread in order to optimize the efficiency and sustainability of manufacturing processes, with growing attention toward human-centered technological solutions. Indeed, in current production lines, which are characterized by agility and connectivity, operators are subject to increased cognitive load and to the constant need to update their skills through reskilling and continuous training. At the same time, manufacturing systems are increasingly required to customize products, resulting in a demand for flexibility which, among other things, takes the form of requiring operators to modify the tasks they perform in the manufacturing of goods.

[0006] In this context, so-called "digital assistants" are spreading, that is, entities, either software or hardware, capable of perceiving the surrounding environment through sensors or other input information provided to them, processing the collected information, and offering functionalities capable of suggesting "intelligent" actions that simplify complex operations, improving operational effectiveness and efficiency. These assistants use artificial intelligence algorithms to learn from experience and adapt to changes in the complex environment in which they operate.

[0007] However, known digital assistants are not without drawbacks, among which is the fact that, in particular those currently used in the manufacturing sector, are based on rule-based architectures, which are rigid and highly specific and customized, thus limiting flexibility and adaptability in dynamic contexts. Moreover, there is a certain resistance to the integration of more human-like features, which are perceived as less controllable variables, but which limits interactions with the operator to short and purely functional exchanges, reduces their effectiveness, and worsens the usability of the tools.

[0008] Other drawbacks include the poor robustness of known digital assistants in handling language variability, sensitivity to interference caused by environmental noise, and the tendency to generate incomplete or incorrect responses, a phenomenon known ashallucinations.

[0009] Furthermore, at present, there are no solutions capable of automatically generating digital assistants dedicated to various and different manufacturing processes.

[0010] Therefore, current digital assistants present significant limitations in terms of flexibility, scalability, and ultimately excessive costs for their setup.

[0011] SUMMARY OF THE INVENTION

[0012] In view of the above, the object of the present invention is to provide a method for generating a digital assistant (hereinafter also DIA- "Digital Intelligent Assistant") for assisting an operator in a manufacturing process, which makes it possible to overcome the limitations of the known art by automatically, effectively and efficiently generating a digital assistant for manufacturing processes, even of different types.

[0013] Within this objective, it is a purpose of the present invention to provide a method for generating a digital assistant that allows to obtain digital assistants capable of enhancing the capabilities of operators, reducing their cognitive load and facilitating task execution, thus transforming their role from mere executors to decision-makers.

[0014] Another purpose of the present invention is to obtain digital assistants that are always up-to-date and capable of improving the training of operators on production processes requiring high flexibility, improving, for example, the onboarding phases of new operators, the reskilling of existing operators, as well as the quality and speed of the assistance provided to operators during complex production processes.

[0015] Another purpose of the present invention is to obtain digital assistants capable of using artificial intelligence algorithms both to identify and characterize the context in real time, adapting to changes in the complex environment in which they operate, and to acquire the knowledge base of manufacturing processes, and to implement a human-machine interaction layer of dialogic type.

[0016] Another purpose of the present invention is to provide a method for generating a digital assistant that allows to obtain a highly reliable and accurate digital assistant, and thus not subject to the so-called "hallucinations."

[0017] A further purpose of the invention is to provide a method for generating a digital assistant that allows to obtain, in a flexible and versatile manner, but also scalable and replicable, digital assistants with various, different and adaptable characteristics, and therefore applicable to any manufacturing process.

[0018] The above-mentioned object, as well as the aforementioned purposes and others whichwill better appear below, are achieved by a method for generating a digital assistant for assisting an operator in a manufacturing process according to claim 1, as well as by a system for generating a digital assistant according to claim 8 and by a computer program according to claim 9.

[0019] Other features are set out in the dependent claims.

[0020] BRIEF DESCRIPTION OF THE FIGURES

[0021] Further features and advantages will become more apparent from the illustrative but non-limiting description of a preferred embodiment of the present invention, illustrated with the aid of Figure 1, which shows a block diagram of the method for generating a digital assistant for assisting an operator in a manufacturing process, according to the invention.

[0022] DETAILED DESCRIPTION OF THE INVENTION

[0023] With particular reference to the figure, the method for generating a digital assistant for assisting an operator in a manufacturing process, implemented on a computer, comprises the steps of:

[0024] A) Acquiring information contained in one or more descriptive documents 10 associated with a manufacturing process and in one or more images of said manufacturing process, detected through at least one image acquisition device ii;

[0025] B) Segmenting said information to extract and divide it into at least the following categories: image category 110, table category 120, text category 130, and metadata category 140;

[0026] C) Generating, for each of said categories 110, 120, 130, 140 at least one textual description 150, 160, 170, 180 descriptive of said information divided into said categories 110, 120, 130, 140;

[0027] D) Reassembling said textual descriptions 150, 160, 170, 180 into a comprehensive textual document 190.

[0028] The method according to the invention further comprises the following steps:

[0029] El) Generating a process coordinator agent 320 for coordinating a plurality of process phase agents 620, comprising the sub-steps of:

[0030] o Providing a template, or model, 310 of a process coordinator agent containing instructions related to the tasks of said process coordinator agent 320;

[0031] o Generating a process coordinator agent 320 based on said template 310;E2) Generating a plurality of process phase agents 620, comprising the sub-steps of:

[0032] o Providing a template, or model, 610 of a process phase agent containing instructions related to the tasks of said process phase agent 620;

[0033] o Reworking said comprehensive textual document 190 to generate a plurality of textual sub-documents, each associated with a respective phase of said manufacturing process;

[0034] o Generating, based on said textual sub-documents and on said template 610, a process phase agent 620 for each phase of said manufacturing process;

[0035] E3) Generating system rules 720, comprising the sub-steps of:

[0036] o Providing a template, or model, 710 of system rules;

[0037] o Generating based on said template 710, system rules 720 defining at least the relationships between said process coordinator agent 320 and said plurality of process phase agents 620;

[0038] The method according to the invention further comprises step F) of generating a digital assistant 900 for assisting an operator 12 in a manufacturing process obtained by integrating said process coordinator agent 320, said plurality of process phase agents 620 and said system rules 720, wherein said system rules 720 also define the interaction relationships between the operator 12 and the digital assistant 900. The digital assistant 900 is configured to receive vocal and / or textual requests from the operator 12 and to receive one or more images from said at least one image acquisition device 11 to provide assistance to the operator 12.

[0039] Preferably, step B) of segmenting the information to extract and divide it into categories, step C) of generating, for each of the categories, at least one corresponding textual description of the information divided into the categories, and step D) of reassembling the textual descriptions into a comprehensive textual document are performed by means of artificial intelligence algorithms. More preferably, at least step C) and / or step D) are performed by an artificial intelligence algorithm based on a Large Language Model (LLM).

[0040] Preferably, a template, or model, may include one or more prompts.

[0041] Preferably, the descriptive documents 10 associated with the manufacturing process comprise one or more of the following documents typically used in industry: bill of materials, routing sheets or work cycles, job operation sheets.

[0042] Generally, the descriptive documents 10 may also comprise one or more of the following documents: technical drawings, technical specifications, quality control sheets,assembly instructions, maintenance manuals, safety data sheets, equipment list, logistics documents.

[0043] Preferably, the image acquisition device 11 comprises a camera.

[0044] More preferably, the image acquisition device 11 comprises a distributed camera system, i.e., a system including a plurality of cameras distributed in the area where the manufacturing process takes place.

[0045] Preferably, the images captured by the image acquisition device 11 may be one or more image sequences or one or more video sequences.

[0046] Preferably, the image acquisition device 11 is configured to capture at least one realtime image of a work scene associated with the manufacturing process, in particular of a specific work scene where one or more operators perform a specific phase of the manufacturing process. Such an image thus provides, once processed, useful information for contextualizing the specific operation performed in the overall manufacturing process.

[0047] For example, the image acquisition devices 11 may be installed on machinery that operates during the manufacturing process, to directly monitor specific operations of the manufacturing process, or in correspondence with the workspace to provide real-time support to the operator performing the manufacturing process.

[0048] For example, a plurality of image acquisition devices 11 may be present, each configured to capture at least one image of a work scene in which one or more operators perform a specific phase of the overall manufacturing process.

[0049] The image acquisition devices 11 may be positioned so as not to capture the operator’s face, in accordance with current privacy requirements.

[0050] The image acquisition devices 11 may be active for the entire duration of the manufacturing process, or activated on request by the operator performing the manufacturing process.

[0051] As explained further below, the images and / or image sequences captured by the image acquisition device 11 are processed by the segmentation and description module 100 to automatically generate a textual description of the images 150, using, for example, an artificial intelligence algorithm based on LLM models. The segmentation and description module 100 is therefore configured to provide a detailed description 150 of what is present in the image or in the image sequence captured by the image acquisition device 11.

[0052] Preferably, steps El), E2), and E3) are carried out in parallel with each other.Preferably, step E2) of generating a plurality of process phase agents 620 further comprises the sub-step of processing one or more of said textual sub-documents to generate one or more optimized word sequences ("optimal chunks") describing a respective phase of said manufacturing process.

[0053] In step E2), the comprehensive textual document 190 is divided into sub-documents, which are further fragmented into "optimal chunks." Each "optimal chunk" represents a sequence of words describing a relevant part of a production phase. The chunk size is adaptive and dynamically determined by the artificial intelligence algorithm based on the complexity of the text and the identification of semantic tags associated with the manufacturing context of the process of interest.

[0054] Advantageously, this fragmentation into "optimal chunks" reduces the risk of information loss, improves the processing of subsequent phases, and enables the construction of more accurate vector databases for each agent.

[0055] The present invention also relates to a system 1 for generating a digital assistant 900 for assisting an operator 12 in a manufacturing process, comprising a processor configured to execute the method described above.

[0056] Preferably, the system 1 comprises one or more of the following:

[0057] - an information acquisition module 15 configured to perform step A);

[0058] - a segmentation and description module 100 configured to perform steps B) and C); - a reassembly module 200, configured to perform step D);

[0059] - a module 300 for generating a process coordinator agent, configured to perform step El);

[0060] - a module 600 for generating a plurality of process phase agents, configured to perform step E2);

[0061] - a module 700 for generating system rules, configured to perform step E3);

[0062] - an integration module 800, configured to perform step F) and to generate the digital assistant 900.

[0063] Preferably, the system 1 comprises all of the above modules.

[0064] Preferably, the system 1 also comprises a module 400 configured to reprocess the comprehensive textual document 190 and to generate the plurality of textual sub-documents, each associated with a respective phase of the manufacturing process, reporting all the necessary details, such as quantities, work orders, components.Preferably, the system 1 also comprises a module 500 configured to process one or more of said textual sub-documents to generate one or more word sequences describing a respective phase of said manufacturing process.

[0065] Preferably, the word sequences have a number of words dynamically adjusted based on the complexity of the information contained in the analyzed textual sub-document and the identification of semantic tags associated with the manufacturing context of the process of interest.

[0066] Preferably, the segmentation and description module 100 is configured to extract images, tables, and text from documents in PDF format, associating each element with the corresponding metadata, such as the position in the original file.

[0067] The present invention also relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method described above.

[0068] The operation of the system 1 for generating a digital assistant 900 is clear from what has been described above.

[0069] In any case, the following is a description of an example of operation of the system 1 for generating a digital assistant 900 for assisting an operator 12 in a manufacturing process, as well as the description of the operation of the digital assistant 900 thus generated.

[0070] The system 1 acquires, through module 15, the detailed description of a production manufacturing process, where such detailed description consists of descriptive documents 10 and images 11.

[0071] The information thus received is segmented by module 100. In this step, using Artificial Intelligence (Al), the various types of data present in the provided document are extracted and divided, such as images 110, tables 120, texts 130, and metadata 140. For each type of data, using Al models, particularly a Large Language Model (LLM), a textual description of each segmented element 150, 160, 170, 180 is generated. This description includes references to the relevant manufacturing process phase, the position in the document 10, and other relevant information.

[0072] Through module 200, again using Al, the previously segmented documents 10 are reassembled into a new document 190 containing only textual information, optimized for subsequent steps.

[0073] Starting from this new document 190, the three steps El), E2), and E3) are carried out in parallel.In step El) of generating the process coordinator agent 320, using a template, or model, 310 provided as input to module 300, the process coordinator agent 320 is generated. This template 310 contains a list of elements required for the creation and operation of the process coordinator agent 320. Once the digital assistant 900 is generated, the process coordinator agent 320 knows all the high-level information of the manufacturing process and can identify the phase in which an operator 12 is located based on the question posed by the operator 12 to the digital assistant 900 and possibly on a visual input, for example provided by the image acquisition device 11. The process coordinator agent 320 is essentially the central agent of the digital assistant 900 since it coordinates the process phase agents 620.

[0074] In step E2) of generating the process phase agents 620, the document 190 reprocessed by the Al of module 200 is further reprocessed by the Al and divided into sub-documents, each containing the detailed explanation of a single phase of the manufacturing process. This is done via module 400.

[0075] For each manufacturing phase, all necessary details (quantities, work orders, components, etc.) are provided. Each process phase agent 620 developed will be an expert in a specific phase of the manufacturing process and will know all its specifications.

[0076] The description of each phase is further divided into smaller "optimal chunks" to facilitate processing, via module 500. A chunk is a word sequence determined by the model’s ability to process input and learn language structures and context. In this phase, system 1 determines the optimal size of the chunks to avoid information loss during transfer to the embedding model for creating vector databases. These vectors will form the database used by the LLM to create a number of phase agents equal to the number of phases. Advantageously, the chunks are created with variable sizes selected by the LLM itself.

[0077] These process phase agents 620 are generated based on a template, or model, 610 provided as input to module 600.

[0078] In step E3) of generating the system rules 700, using the reprocessed document 190, the system rules are defined based on an initial template, or model, 710. These rules define how the digital assistant 900 processes and manages the input received from the operator 12 and how the process coordinator agent 320 and the process phase agents 620 interact with each other.

[0079] These three elements (process coordinator agent 320, process phase agents 620, and system rules 720) are then automatically integrated by module 800 to create the digital assistant 900 expert in the specific manufacturing process.The digital assistant 900 is a multimodal system, capable of receiving both textual or voice inputs (questions from the operator 12) and visual inputs (images captured by camera 11). The operator 12 can interact with it either vocally or textually. Thanks to the process coordinator agent 320, the digital assistant 900 is able to direct the operator’s 12 request to the most appropriate process phase agent 620, which will provide the necessary answer to the operator 12.

[0080] It has been found in practice that the method and system for generating a digital assistant for assisting an operator in a manufacturing process, according to the present invention, fulfil the task and the intended purposes, as they allow the automatic, effective, and efficient generation of a digital assistant that is specific each time to a given manufacturing process.

[0081] Another advantage of the system according to the invention is that, starting from the description of any process, the system is capable of autonomously constructing all the elements of the architecture of a digital assistant (DIA), thus enabling the creation of a tool specific to the process in question. It is therefore a system of Al that generates other Al systems.

[0082] A further advantage is that the system is capable of acquiring inputs of various kinds, including metadata and structured and unstructured data (such as images, videos, audio, text).

[0083] Furthermore, thanks to its architecture based on a coordinator agent and specialized agents, the digital assistant can respond accurately and precisely to operators' questions concerning the production process, reducing the risk of hallucinations or incomplete information. Each phase agent is in fact dedicated to specific phases (tasks) of the manufacturing process.

[0084] Another advantage of the method and system according to the invention is that it requires industrial documents already available in manufacturing processes, which therefore do not need to be created ad hoc, as they are reprocessed by the system itself.

[0085] Another advantage is that the implementation of specialized agents for each process phase, coordinated by a central agent, solves the known problems of Al agents generating incorrect or hallucinated responses. Advantageously, the coordinator agent queries each specialized agent only on questions and specifications relevant to their area of expertise, significantly reducing the risk of hallucinations.

[0086] In the manufacturing context, each production process can vary significantly, requiring flexible solutions that can adapt to different conditions and specifications. The invention addresses this problem by proposing a versatile solution applicable to any manufacturing process. Unlike current solutions, which are rigid and specific to individual processes, thesystem according to the invention offers a flexible structure able to adapt to a wide range of production scenarios, improving the usability and effectiveness of the generated digital agent.

[0087] Moreover, the breakdown of a manufacturing process into fundamental phases and the assignment to dedicated agents for each of these phases, under the management of an automatic coordinator agent, enables the digital agent to be flexible and able to adapt even to any changes in production processes in a dynamic way. In fact, it is always possible to generate or update, with the method according to the invention, a digital assistant whenever there are changes to the manufacturing process.

[0088] The flexibility, scalability, and replicability required by the market today are met by the system according to the invention, which is in fact not a digital assistant per se, but a "generator of digital assistants" that are always up-to-date and updatable.

[0089] In essence, digital assistants (DI As) obtained through the method according to the invention facilitate rapid data analysis, support the decision-making process, and contribute to cost reduction and decreased downtime, while at the same time promoting knowledge transfer among operators. In particular, DIAs are useful in training operators engaged in complex tasks, offering them support during production processes. Moreover, DIAs simplify tasks that could cause cognitive overload, such as those related to complex assembly, characterized by a high variability of components and a high risk of error, by providing real-time assistance to operators and thus reducing their cognitive load.

[0090] The method and system thus conceived are susceptible to numerous modifications and variants, all falling within the scope of the inventive concept.

[0091] Furthermore, all details may be replaced by other technically equivalent elements.

Claims

CLAIMS1. A computer-implemented method for generating a digital assistant for assisting an operator in a manufacturing process, comprising the steps of:A) Acquiring information contained in one or more descriptive documents (10) associated with a manufacturing process and in one or more images of said manufacturing process, detected through at least one image acquisition device (ii);B) Segmenting said information to extract and divide it into at least the following categories: image category (110), table category (120), text category (130), and metadata category (140);C) Generating, for each of said categories (110, 120, 130, 140), at least one textual description (150, 160, 170, 180) descriptive of said information divided into said categories (110, 120, 130, 140);D) Reassembling said textual descriptions (150, 160, 170, 180) into a comprehensive textual document (190);said method further comprising the following steps:El) Generating a process coordinator agent (320) for coordinating a plurality of process phase agents (620), comprising the sub-steps of:o Providing a template (310) of a process coordinator agent containing instructions related to the tasks of said process coordinator agent (320); o Generating a process coordinator agent (320) based on said template (310);E2) Generating a plurality of process phase agents (620), comprising the substeps of:o Providing a template (610) of a process phase agent containing instructions related to the tasks of said process phase agent (620); o Reworking said comprehensive textual document (190) to generate a plurality of textual sub-documents, each associated with a respective phase of said manufacturing process;o Generating, based on said textual sub-documents and on said template (610), a process phase agent (620) for each phase of said manufacturing process;E3) Generating system rules (720), comprising the sub-steps of:o Providing a template (710) of system rules;o Generating, based on said template (710), system rules (720) defining at least the relationships between said process coordinator agent (320) and said plurality of process phase agents (620);said method further comprising step F) of generating a digital assistant (900) for assisting an operator (12) in a manufacturing process obtained by integrating said process coordinator agent (320), said plurality of process phase agents (620), and said system rules (720), where said system rules (720) also define the interaction relationships between said operator (12) and said digital assistant (900),said digital assistant (900) being configured to receive voice and / or textual requests from said operator (12) and to receive one or more images from said at least one image acquisition device (11) to provide assistance to said operator (12).

2. A method according to the preceding claim, wherein said step B), said step C), and said step D) are carried out through artificial intelligence algorithms.

3. A method according to one or more of the preceding claims, wherein said at least one image acquisition device (11) comprises a camera and / or a distributed camera system.

4. A method according to one or more of the preceding claims, wherein said at least one image acquisition device (11) is configured to capture at least one real-time image of a work scene associated with said manufacturing process.

5. A method according to one or more of the preceding claims, wherein said at least one image acquisition device (11) comprises a plurality of image acquisition devices (11), each configured to capture at least one image of a work scene in which one or more operators perform a specific process phase of said manufacturing process.

6. A method according to one or more of the preceding claims, wherein said step E2) of generating a plurality of process phase agents (620) further comprises the sub-step of processing one or more of said textual sub-documents to generate one or more word sequences descriptive of a respective phase of said manufacturing process.

7. A method according to claim 6, wherein said one or more word sequences have a number of words dynamically adjusted based on the complexity of the information contained in said textual sub-document and / or based on the presence of semantic tags associated with the context of said manufacturing process.

8. A method according to one or more of the preceding claims, wherein said process coordinator agent (320) is configured to identify a current phase of the manufacturingprocess in which the operator (12) is involved on the basis of both a question posed by the operator (12) to the digital assistant (900) and at least one image captured by said image acquisition device (11), and to select, in accordance with the system rules (720), the corresponding process phase agent (620) to be queried.

9. A system (1) for generating a digital assistant (900) for assisting an operator (12) in a manufacturing process comprising a processor configured to execute the method according to one or more of claims 1 to 8.

10. A computer program comprising instructions which, when executed by a computer, cause said computer to execute the method according to one or more of claims 1 to 8.

Images