Method and system for testing and retraining an AGV using synthetic obstacles

Abstract

A method and system for retraining an Automated Guided Vehicle (AGV) (104) in a factory environment (100) is disclosed. The method involves receiving a visual feed (120) from at least one sensor (106) configured to capture visual data of the factory environment (100). A plurality of synthetic obstacles (114) is inserted into the received visual feed (120) to generate a modified visual feed (116), simulating real-world conditions by embedding digital representations of real-world obstacles. The modified visual feed (116) is analyzed using a reinforcement learning-based navigation model (108) to generate a navigational decision based on the detection of the synthetic obstacles (114). A reinforcement learning procedure is then initiated to retrain the navigation model (108) based on the analysis of the generated navigational decision.

Description

[0001] Siemens Aktiengesellschaft

[0002] 1 METHOD AND SYSTEM FOR TESTING AND RETRAINING AN AGV USING

[0003] SYNTHETIC OBSTACLES

[0004] The present invention relates to the field of automated guided vehicles (AGVs). More specifically, it pertains to methods and systems for retraining AGVs in factory environments using synthetic obstacles and reinforcement learning techniques.

[0005] Automated Guided Vehicles (AGVs) have become essential components in modern factory floors, providing efficient and reliable material handling solutions. These AGVs, equipped with advanced sensors and Al-based navigation systems, autonomously optimize logistics within manufacturing and assembly environments. However, despite their technological advancements, AGVs face significant challenges in adapting to dynamic and evolving factory conditions.

[0006] Traditional AGVs use basic sensors like infrared, ultrasonic, and simple visual cameras for obstacle detection. Basic algorithms are employed to navigate around detected obstacles. These systems often lack the sophistication to handle complex and dynamic environments, resulting in failures to recognize and respond to new or unexpected obstacles, leading to operational inefficiencies and potential collisions.

[0007] Modern AGVs use advanced vision systems, including cameras and LIDAR, combined with deep learning models like YOLO (You Only Look Once) for object detection and reinforcement learning models for navigation. However, the performance of these models heavily relies on the quality and comprehensiveness of training data. Collecting and labeling extensive datasets that encompass all possible obstacles and scenarios is challenging and time-consuming. Current training environments often miss out on testing AGVs in various real-world scenarios, leading to AGVs being unprepared for new obstacles or changes in the factory layout. F actories are dynamic environments where layouts and conditions frequently change, and AGVs trained on previous layouts may struggle to adapt to new configurations, resulting in collisions or operational delays.

[0008] Some solutions simulate entire factory environments for AGV training and testing. These virtual environments replicate factory conditions and allow AGVs to be tested without physical risks. However, simulated environments may not perfectly replicate the nuances of real-world conditions, leading to discrepancies between simulated and actual factory environments. Creating and maintaining high-fidelity simulations are resource -intensive processes, requiring significant Siemens Aktiengesellschaft

[0009] 2 computational power and expertise.

[0010] Given the limitations of current AGV systems and testing methods, there is a need for a robust solution that can dynamically test and improve the resilience of AGVs in real-time operational environments. The primary challenges include incomplete and inadequate training data, existing training datasets often lack the diversity and comprehensiveness required to prepare AGVs for all possible obstacles and scenarios. There is a lack of real-world scenario testing, as current methods do not provide adequate testing in real-world conditions, leading to AGVs being unprepared for new or dynamically changing obstacles. Frequent changes in factory layouts necessitate continuous adaptation by AGVs, which current training and testing methods fail to address adequately. Collisions and halts due to unrecognized obstacles result in operational downtime, efficiency losses, and potential safety hazards.

[0011] In light of above, there exists a need for an efficient method and system for retraining of an automated guided vehicle. An object of the present invention is achieved by a computer implemented method and system for retraining an Automated Guided Vehicle (AGV) in a factory environment.

[0012] The object of the present invention is achieved by a computer-implemented method for retraining the Automated Guided Vehicle (AGV) in the factory environment. Retraining the AGV is necessary to ensure that the AGV is enabled to adapt to dynamic and evolving conditions within the factory environment. Examples of the dynamic and evolving conditions comprises presence of new obstacles or one or more changes in a factory layout of the factory environment. Incomplete training data, lack of testing in various scenarios, and alterations in the factory's physical arrangement can all contribute a difficulty in navigating and avoiding collisions within the factory environment. By retraining, the AGV is enabled to handle unforeseen situations efficiently, thereby minimizing operational disruptions and financial losses. Moreover, retraining significantly enhances a safety of both the AGV and factory personnel.

[0013] A factory environment refers to an industrial setting where manufacturing or assembly processes occur. Examples of the factory environment include automotive assembly plants, electronics manufacturing facilities, and food processing units. The factory environment typically consists of various machines, workstations, storage areas, and pathways for personnel and equipment movement. Automated Guided Vehicles operate within this environment to Siemens Aktiengesellschaft

[0014] 3 transport materials, components, or finished products between different locations on a factory floor within the factory environment.

[0015] The Automated Guided Vehicle (AGV) is a mobile robot used in industrial applications for material handling and transportation tasks. Examples of AGVs include forklift AGVs that transport pallets, tugger AGVs that pull carts, and unit load AGVs that carry single loads. The AGV is configured to navigate autonomously using one or more sensors, and Abbased navigation models to avoid obstacles and follow predetermined paths.

[0016] The method of retraining the AGV is implemented in an automation module. The automation module comprises computer-readable code that can be executed by a processing unit to implement the method, ensuring that the AGV is retrained properly. The automation module is configured to receive a visual feed from sensors mounted on the AGV, insert synthetic obstacles into the visual feed to simulate real-world conditions, and analyze a modified visual feed using a reinforcement learning-based navigation model. The automation module is further configured to generate a plurality of navigational decisions and initiates a reinforcement learning procedure to retrain the navigation model based on one or more performance metrics of the AGV. The retrained model is then deployed in an operational environment of the AGV, enhancing resilience of the AGV to new obstacles and changes within the factory environment.

[0017] The method comprises receiving a visual feed from at least one sensor. The at least one sensor is configured to capture visual data of the factory environment. The captured visual data refers to the information captured by the at least one sensor. The visual data represent a surrounding of the AGV. The visual data comprises images with depth view, images without depth view, and point clouds generated by sensors like LIDAR. The captured visual data is crucial for the AGV to understand and navigate environment accurately.

[0018] An example of the at least one sensor is at least one of a LIDAR sensor, a depth sensing camera. A LIDAR sensor, which stands for Light Detection and Ranging, is configured to emit pulses of LASER measures a time taken by the LASER to return after hitting an object. The measured time is used to create detailed 3D maps of the factory environment. The depth sensing camera is configured to capture images that include depth information, allowing the AGV to perceive a distance to various objects in a field of view of the AGV. In one example, the at least one sensor is mounted on the AGV. In one example, the at least one sensor is configured to capture a field-of-view from a perspective of the AGV. The at Siemens Aktiengesellschaft

[0019] 4 least one sensor enables the AGV to continuously monitor the surroundings and detect any obstacles or changes in the factory environment.

[0020] The received visual feed comprises a continuous stream of visual data captured by the at least one sensor. The visual feed comprises a plurality of visual frames. Each visual frame of the plurality of visual frames represents a snapshot of the surrounding of the AGV at a specific time. The plurality of visual frames can be represented as data objects in various formats, such as JPEG images for standard pictures, PNG images for depth maps, or PCD files for point clouds generated by LIDAR sensors.

[0021] The method further comprises receiving one or more user- defined criteria associated with a plurality of synthetic obstacles which should be used to train the AGV. The plurality of synthetic obstacles comprises one or more digital representations of real- world obstacles. In one example, the plurality of synthetic obstacles comprises three-dimensional models which are configured to move. In one example, a shape and a position of at least one of the plurality of synthetic obstacles changes with time.

[0022] The plurality of synthetic obstacles can be implemented in different formats depending on a type of visual feed received by the automation module. In a first scenario in which the visual feed is a video, the plurality of synthetic obstacles are embedded into one or more video frames of the visual feed, thereby giving the AGV an illusion of encountering real obstacles in the factory environment. For example, digital representations of boxes, human workers, or machinery can be inserted into the video feed, and these synthetic obstacles can move or change positions to simulate dynamic factory conditions. In a second scenario in which the visual feed is a point cloud or a depth map, the plurality of synthetic obstacles are represented as three-dimensional data points or depth information. For instance, LIDAR- generated point clouds might have additional clusters of points representing pallets or carts, while depth maps could include simulated depth variations corresponding to objects like forklifts or conveyor belts.

[0023] The plurality of synthetic obstacles are used for testing and retraining the AGV. Using the plurality of synthetic obstacles for training rather than using real- world obstacles offers several advantages. The plurality of synthetic obstacles allow for controlled and repeatable testing conditions, ensuring that the AGV encounters a wide range of scenarios that might not be easily replicated in a real- world environment. Use of the plurality of synthetic obstacles also eliminates the risk of damaging the AGV or factory equipment during training since the Siemens Aktiengesellschaft

[0024] 5 plurality of synthetic obstacles are virtual. Additionally, the plurality of synthetic obstacles can be quickly modified to test various configurations and behaviors, making the training process more efficient and comprehensive.

[0025] In one example, the received one or more user-defined criteria specify one or more characteristics and behaviors of at least one of the plurality of synthetic obstacles. The one or more user-defined criteria are parameters set by a user to generate the plurality of synthetic obstacles according to specific training needs. In one example, the one or more user- defined criteria are received through a user interface or configuration file where a user inputs one or more desired parameters. The types of the one or more user-defined criteria associated with each synthetic obstacle include a specific type of obstacle to be simulated using the synthetic obstacle, a location within the factory environment where the synthetic obstacle is to appear, a timing and duration for which the synthetic obstacle is to be present in the visual feed, a movement pattern of the synthetic obstacle, and an interaction behavior of the synthetic obstacle, including changes in shape, size, or color over time.

[0026] For example, in a context of the factory environment, the one or more user- defined criteria may define a criterion to simulate a specific type of obstacle such as a moving cart. Further, the one or more user-defined criteria may specify the location within the factory where the synthetic obstacle should appear, such as a high-traffic intersection. Furthermore, the one or more user- defined criteria may also set the timing and duration for which the synthetic cart is present in the visual feed, such as during peak operation hours. Additionally, the one or more user-defined criteria may define a movement pattern for the synthetic cart, such as moving back and forth between two points. Interaction behavior criteria may include changes in the cart's size or color to simulate different types of loads or conditions.

[0027] Each of the plurality of synthetic obstacles is associated with at least one user- defined criterion of the one or more user- defined criteria. For example, a synthetic obstacle representing a worker in a different colored uniform could be associated with the one or more user-defined criteria specifying a uniform color, a worker's location, and a duration the worker remains in a pathway of the AGV. By incorporating the one or more user- defined criteria, the method ensures that the AGV is exposed to realistic and relevant training scenarios, enhancing its ability to navigate and perform tasks efficiently in the ever-changing factory environment. Siemens Aktiengesellschaft

[0028] 6

[0029] The method further comprises utilizing a Large Language Model (LLM) to generate each of the plurahty of synthetic obstacles based on the one or more user- defined criteria. The Large Language Model (LLM) is an advanced Artificial Intelligence model trained on vast amounts of data to understand and generate complex outputs, including graphical objects. The LLM generates the plurality of synthetic obstacles by interpreting the one or more user- defined criteria and creating detailed graphical representations and configurations of the obstacles accordingly.

[0030] Examples of LLMs capable of generating graphical objects include DALL-E, which is renowned for its ability to generate detailed and contextually accurate images from textual descriptions. DALL-E can analyze user- defined criteria, such as the type, size, movement patterns, and positions of obstacles, and produce highly detailed graphical objects that can be directly used in the AGV's visual feed. Another example is GauGAN, which excels in converting text-based scene descriptions into realistic images, enabling the creation of synthetic obstacles that closely mimic real-world conditions. Additionally, models like DGato, which is designed to handle multiple types of data and tasks, can be leveraged for their versatility in generating complex graphical objects based on textual inputs.

[0031] The LLM is configured to generate highly detailed and accurate graphical representations of synthetic obstacles. For instance, given a user-defined criterion to simulate a moving cart that changes color from red to blue every 10 seconds, the LLM is configured to generate a sequence of images that visually depict an appearance and behavior of the cart over time. Similarly, the LLM is enabled to be fine-tuned to understand specific factory-related terminology and generate appropriate graphical obstacles that align with the factory environment. By leveraging these advanced LLMs, the method ensures that the synthetic obstacles are not only realistic but also varied and contextually accurate, providing a comprehensive training environment that enhances an ability of the AGV to navigate and perform tasks efficiently in dynamic factory settings.

[0032] The LLM is configured to generate the plurahty of synthetic objects after receiving one or more prompts. The prompts fed into the LLM are generated based on the one or more user-defined criteria. For instance, if the user defines a criterion for simulating a moving cart, the prompt comprises details such as the type of cart, a location of the cart within the factory environment, a movement pattern of the cart, and any changes in an appearance of the cart over time. The LLM processes the prompts to generate a detailed graphical representation and Siemens Aktiengesellschaft

[0033] 7 configuration of the synthetic cart, which is then used to create the synthetic obstacle associated with the cart, in the visual feed.

[0034] For example, when the user specifies a moving cart, the prompt might be: "Generate a moving cart that travels between two points in the factory, changing its color from red to blue every 10 seconds." If the user wants the synthetic obstacle to appear in a specific location within the factory, such as a high-traffic intersection, the prompt might be: "Create a synthetic worker standing at the high-traffic intersection near the assembly line, wearing a yellow uniform." When specifying the timing and duration for which the synthetic obstacle is to be present in the visual feed, the prompt might be: "Simulate a forklift appearing in the visual feed for 30 seconds during peak operation hours, moving from the storage area to the loading dock." If the user defines a movement pattern of the synthetic obstacle, such as a zigzag pattern, the prompt might be: "Generate a synthetic pallet that moves in a zigzag pattern across the factory floor, avoiding obstacles and changing direction every 5 meters."

[0035] In one example, the prompts are generated by use of one or more prompt templates. For example, a prompt template for a moving cart might be: "Generate a [type of obstacle] that [movement pattern] between [start point] and [end point], changing its [property] from [initial value] to [final value] every [time interval]." If the user specifies a moving cart, the filled-in prompt would be: "Generate a moving cart that travels between two points in the factory, changing its color from red to blue every 10 seconds."

[0036] Similarly, for a synthetic worker, the prompt template might be: "Create a synthetic [type of worker] standing at [specific location] near [landmark], wearing a [uniform color] uniform." If the user wants the synthetic obstacle to appear in a specific location within the factory, such as a high-traffic intersection, the filled-in prompt would be: "Create a synthetic worker standing at the high- traffic intersection near the assembly line, wearing a yellow uniform."

[0037] When specifying the timing and duration for which the synthetic obstacle is to be present in the visual feed, the prompt template might be: "Simulate a [type of obstacle] appearing in the visual feed for [duration] during [time period], moving from [start location] to [end location]." The filled-in prompt would be: "Simulate a forklift appearing in the visual feed for 30 seconds during peak operation hours, moving from the storage area to the loading dock."

[0038] For defining a movement pattern of the synthetic obstacle, the prompt template Siemens Aktiengesellschaft

[0039] 8 might be: "Generate a synthetic [type of obstacle] that moves in a [movement pattern] across [location], avoiding obstacles and changing direction every [distance]." The filled-in prompt would be: "Generate a synthetic pallet that moves in a zigzag pattern across the factory floor, avoiding obstacles and changing direction every 5 meters."

[0040] The method further comprises determining, from the received visual feed, a set of visual frames for the synthetic obstacle. A visual frame is a single image or snapshot captured by the sensors mounted on the AGV, representing the AGV's surroundings at a specific moment in time. Examples of visual frames include individual video frames in a continuous video feed or singular depth images in a depth map.

[0041] The set of visual frames are determined based on an analysis of the one or more user-defined criteria associated with the synthetic obstacle. For example, if the user-defined criteria specify that a synthetic obstacle should appear at a high- traffic intersection within the factory environment, the set of visual frames will include all frames captured by the AGV's sensors when the AGV is approaching and navigating through the high-traffic intersection. If the criteria specify a timing and duration, such as a synthetic cart appearing for 30 seconds during peak operation hours, the set of visual frames will include all frames captured within that specific time window.

[0042] To determine the set of visual frames, the method involves analyzing the one or more user- defined criteria using a technical solution such as a machine learning algorithm or a rule-based engine. The machine learning algorithm or the rulebased engine processes the one or more user-defined criteria to identify the relevant frames in the visual feed. For example, when analyzing criteria based on location, the system may use image recognition techniques to identify frames where the AGV is in the specified location. For timing-based criteria, the automation module is configured to use timestamps to filter the set of visual frames within a specified duration. For criteria based on the movement pattern of the synthetic obstacle, the automation module is configured to track the movement of the AGV and select frames that align with the movement pattern.

[0043] By determining the set of visual frames based on an analysis of the one or more user- defined criteria, the method ensures that the plurality of synthetic obstacles are accurately and contextually embedded into the visual feed. Thus, the automation module enhances a training process by exposing the AGV to realistic Siemens Aktiengesellschaft

[0044] 9 and relevant scenarios, thereby improving an ability of the AGV to navigate and perform tasks efficiently in the dynamic factory environment.

[0045] The method further comprises generating a series of configurations and a series of positions of the synthetic obstacle by application of the LLM on the one or more user-defined criteria. Each of the series of configurations and the series of positions is indicative of a configuration or position of the synthetic obstacle at a time instance corresponding to each frame of the set of visual frames.

[0046] The series of configurations refers to the different forms or states that the synthetic obstacle can take across various frames. For example, if the synthetic obstacle is a moving cart, the series of configurations may include the cart in different orientations, such as facing forward, turning left, or turning right. The series of positions refers to the specific locations of the synthetic obstacle within the visual frame at each time instance. For the moving cart, the series of positions may indicate the cart's location at different points along its path, such as at the start, midpoint, and end of its journey. The term "series" is used because the configurations and positions change over time, creating a sequence that aligns with the progression of visual frames. This sequence allows the synthetic obstacle to appear dynamic and realistic, simulating real-world conditions more accurately.

[0047] Generating the series of configurations and positions offers several advantages. One advantage is the ability to create more complex and varied scenarios for training the AGV. By having a series of configurations, the synthetic obstacle can exhibit different behaviors and states, challenging the AGV to adapt to a wide range of situations. Another advantage is an enhanced realism of a training environment. A moving cart that changes its orientation and position over time provides a more lifelike simulation, improving an ability of the AGV to handle real-world obstacles.

[0048] For example, consider a synthetic worker that needs to appear in the visual feed. The series of configurations comprises the worker standing, bending, and walking, while the series of positions is indicative of a location of the workers at different points along a designated pathway in the factory environment. By embedding these configurations and positions into the visual frames, the AGV encounters a dynamic and realistic obstacle, enhancing a training experience of the AGV.

[0049] The method involves the application of the LLM to generate the series of Siemens Aktiengesellschaft

[0050] 10 configurations and positions for each synthetic obstacle based on the one or more user- defined criteria. The LLM processes the one or more user- defined criteria and produces detailed descriptions of states of the synthetic obstacle's states and locations, which are then embedded into the visual frames.

[0051] The method further comprises embedding each synthetic obstacle into an appropriate visual frame of the set of visual frames, based on the generated series of configurations and the series of positions of the synthetic obstacle. In the case where the visual feed is a video feed, the embedding process involves inserting the synthetic obstacle into each frame of the video at a specified position and configuration. The insertion is performed frame by frame, ensuring that the synthetic obstacle appears consistently and realistically int eh video feed. For example, if the synthetic obstacle is a moving cart, the cart is rendered into each frame at the correct location and orientation according to the series of positions and configurations.

[0052] In the case where the visual feed is a 3D depth map, the embedding process involves integrating the synthetic obstacle into the 3D space represented by the depth map. The integration requires adjusting the depth values to reflect the presence of the synthetic obstacle at the specified positions. For instance, if the synthetic obstacle is a worker, the depth values at the worker's location are modified to match a dimension and shape of the worker, ensuring that the depth map accurately represents the presence of the synthetic environment in the 3D environment.

[0053] Examples of algorithms used to embed the synthetic obstacle include image compositing techniques such as alpha blending and depth-based compositing. Alpha blending involves combining the synthetic obstacle with the visual frame using transparency values to create a seamless integration. Depth-based compositing uses the depth information from the 3D depth map to accurately place the synthetic obstacle within the 3D space, ensuring that it interacts correctly with other objects in the environment. Other algorithms such as 3D rendering and ray tracing can also be used to achieve realistic embedding of synthetic obstacles in both 2D and 3D visual feeds.

[0054] The synthetic obstacle is embedded based on the generated series of configurations and the series of positions of the synthetic obstacle. For example, if the user-defined criteria specify a moving cart that changes its orientation and position over time, the embedding process ensures that the cart appears in each visual frame at the correct location and orientation. The embedding process Siemens Aktiengesellschaft

[0055] 11 involves rendering the cart into the visual frame using a specified configuration, such as facing forward or turning left, and placing it at the appropriate position within the visual frame. This approach creates a dynamic and reahstic obstacle that challenges the AGV to adapt to various scenarios.

[0056] By embedding the synthetic obstacle into the visual frame based on the generated series of configurations and the series of positions, the method ensures that the AGV encounters reahstic and varied training scenarios. This comprehensive training environment enhances the AGV's ability to navigate and perform tasks efficiently in the dynamic factory environment, ultimately improving its resilience and adaptability. In other words, the method further comprises inserting the plurality of synthetic obstacles into the received visual feed to generate a modified visual feed comprising the plurality of synthetic obstacles. The plurality of synthetic obstacles are inserted into the visual feed to simulate real-world condition in the factory environment.

[0057] The method further comprises detecting the plurality of synthetic obstacles by application of an object detection algorithm on the modified visual feed. An object detection algorithm is a machine learning model designed to identify and locate objects within an image or a sequence of images. The algorithm is trained using a large dataset of annotated images, where each image contains labeled instances of various objects along with their bounding boxes, which define the objects' positions within the images.

[0058] For example, an object detection algorithm such as YOLO (You Only Look Once) can be trained using a dataset that includes thousands of images of factory environments, each annotated with labels for objects such as workers, carts, pallets, and machinery. The training process involves feeding these annotated images into the algorithm, which learns to recognize the patterns and features associated with each type of object. The object detection algorithm adjusts internal parameters to minimize a difference between predictions and actual labels in the training data.

[0059] The object detection algorithm provides output in a plurality of bounding boxes around detected synthetic obstacles, along with confidence scores that indicate likelihood that the identified obstacles match specified labels. For instance, after processing a visual frame, the algorithm might output bounding boxes around two detected synthetic obstacles, labeling one as a worker with a confidence score of 0.95 and the other as a cart with a confidence score of 0.90. Siemens Aktiengesellschaft

[0060] 12

[0061] In the context of the method, the object detection algorithm processes the modified visual feed, which includes the plurality of synthetic obstacles. The algorithm scans each frame of the modified visual feed, identifying and locating one or more synthetic obstacles based on patterns and features. For example, when the object detection algorithm processes a frame that includes a synthetic worker, it outputs a bounding box around the worker, along with a high confidence score indicating an accuracy of detection.

[0062] The method further comprises analyzing the detected plurality of obstacles using a reinforcement learning-based navigation model to generate a navigation decision. The reinforcement learning-based navigation model is an advanced Al model that learns to make decisions by interacting with the factory environment and receiving feedback in form of rewards or penalties. The navigation model uses feedback to improve navigation process over time, aiming to maximize cumulative rewards.

[0063] Examples of the reinforcement learning-based navigation model comprises a Deep Q-Networks (DQN). The Deep Q-Networks (DQN) can be trained to navigate the AGV within the factory environment. The navigation model receives input the modified visual feed with the detected plurality of synthetic obstacles, and outputs navigational commands to control a movement of the AGV. One or more actions of the AGV, such as moving forward, turning left, or stopping, are evaluated based on an effectiveness of the one or more actions in causing the AGV to avoid obstacles and reaching a target location. The navigation model is configured to continuously learn to adjusting strategies to improve performance of the navigation model.

[0064] The navigation model is used to control and navigate the AGV in the factory environment. For example, if the object detection algorithm detects a synthetic obstacle such as a moving cart, the navigation model processes information associated with the detected synthetic obstacle and determines a best course of action to avoid a collision with the detected synthetic obstacle. The navigation model is configured to consider a current position and speed of the AGV, the location of the cart, and a layout of the factory environment to make an informed decision.

[0065] Examples of the navigation model comprises algorithms such as Proximal Policy Optimization (PPO) or Actor-Critic methods. The navigation model is configured to evaluate multiple potential actions and select an action that maximizes expected rewards. For instance, the PPO algorithm is configured to balance Siemens Aktiengesellschaft

[0066] 13 exploration and exploitation, ensuring that the AGV not only follows known safe paths but also explores new routes to improve overall navigation efficiency.

[0067] The navigation decision is an output generated by the navigation model, specifying a next action of the AGV based on the detected plurality of synthetic obstacles. For example, if the navigation model detects a synthetic worker standing in a path of the AGV, the navigation decision might be to stop and wait until the path is clear. Alternatively, if the navigation model detects a moving cart approaching from the left, the navigation decision might be to turn right to avoid a collision.

[0068] The navigation decision is taken based on the detection of the plurality of synthetic obstacles. The navigation model analyzes a positions and a configurations of the detected plurality of synthetic obstacles, considering factors such as distance, speed, and trajectory. For example, if the navigation model detects multiple obstacles, such as a worker and a cart, the navigation model prioritizes actions that minimize a risk of collision with both objects. The navigation model evaluates potential actions, such as changing speed, altering direction, or stopping, and selects the action that maximizes safety and efficiency.

[0069] The method further comprises executing the generated navigational decision by controlling the movement of the AGV, based on the navigational decision. The AGV is equipped with a plurality of components that allow the automation module to control the movement of the AGV. The plurality of components include motor controllers, steering mechanisms, and braking systems. For example, the motor controllers adjust the speed of the AGV, the steering mechanisms change a direction of the AGV, and the braking systems enable the AGV to stop. The movement of the AGV comprises actions such as moving forward, turning left or right, accelerating, decelerating, or stopping entirely.

[0070] The method further comprises monitoring a response of the AGV to the navigational decision. The response is monitored using a plurality of apparatuses, including cameras and a Central Monitoring Station (CMS). In one example, cameras are mounted on the AGV or within the factory environment to capture visual data of the movement of the AGV. The captured visual data is analyzed to determine whether the AGV is following the navigational decisions accurately.

[0071] In another example, the Central Monitoring station (CMS) is used to monitor the response of the AGV. The CMS is an integrated system that collects and analyzes Siemens Aktiengesellschaft

[0072] 14 data from a plurlaity of sensors installed on the AGV. The plurahty of sensors includes accelerometers, gyroscopes, and GPS modules, which provide real-time information on a speed, orientation, and position of the AGV. For instance, the CMS is configured detect any deviations from an intended path by comparing an actual trajectory of the AGV with a planned route.

[0073] The response of the AGV is monitored based on a plurahty of real-world performance metrics. The plurahty of real-world performance metrics refers to quantifiable measures that evaluate an effectiveness and efficiency of operation of the AGV. Examples of the plurality of real-world performance metrics include a count of collision occurrences, a count of stop times, and a count of deviations from the intended path of the AGV. The count of collision occurrences tracks the number of times the AGV collides with real-world obstacles, the count of stop times records the frequency and duration of stops of the AGV, and the count of deviations measures how often and how far the AGV strays from the planned route.

[0074] The automation module determines the plurality of real-world performance metrics by analyzing data collected from the plurahty of sensors. For example, the automation module uses collision sensors to count the number of collisions and analyze video feed to identify stops and deviations. By processing this data, the automation module generates detailed performance reports that highlight areas for improvement.

[0075] Monitoring the response of the AGV offers several advantages. It provides valuable insights into the AGV's operational performance, helping identify any issues that may affect its efficiency. For instance, frequent stops or deviations might indicate obstacles that were not detected or navigational decisions that need refinement. Additionally, monitoring allows for continuous improvement of navigation model of the AGV by providing real-world feedback that can be used to retrain the navigation model.

[0076] By executing the generated navigational decision and monitoring the response of the AGV based on a plurahty of real-world performance metrics, the automation module ensures that the AGV operates effectively within the factory environment. Thus, reliability and efficiency of the the AGV is enhanced leading to smoother and more productive operations in dynamic and complex factory settings. Siemens Aktiengesellschaft

[0077] 15

[0078] The method further comprises generating a simulation instance of the factory environment. The generated simulation instance includes the plurality of synthetic obstacles and a digital twin of the AGV. The generated simulation instance is a virtual representation of the factory environment, created to replicate real-world conditions for testing and training purposes. The plurality of synthetic obstacles are mapped to the simulation instance based on the one or more user-defined criteria. For example, if the user-defined criteria specify a moving cart at a high-traffic intersection, the simulation instance will include the cart positioned and moving according to the user-defined criteria. The cart's movement pattern, timing, and location will be accurately represented within the virtual environment.

[0079] The digital twin is a precise virtual model of a physical object — in this case, the AGV. The digital twin of the AGV replicates physical characteristics, behavior, and operational parameters of the AGV. The digital twin is used to simulate how the AGV would interact with the plurality of synthetic obstacles and navigate the factory environment. For example, the digital twin is configured to simulate the AGV's response to a worker crossing its path, allowing the developers to study and improve the AGV's navigation processes without risking damage to the AGV.

[0080] In terms of implementation, the digital twin and the simulation instance are represented as structured data objects within a simulation framework. The digital twin of the AGV is implemented as a complex data structure, such as a class in an object-oriented programming language, encapsulating attributes like dimensions, sensor configurations, actuator specifications, and operational parameters. Each attribute is typically stored as a nested data structure; for instance, sensors might be represented as a dictionary with keys corresponding to sensor types and values detailing their properties. The simulation instance itself can be modeled as a composite data structure that integrates various elements of the factory environment, including a list of synthetic obstacles and the digital twin of the AGV. Each synthetic obstacle can be represented as an object with attributes such as type, initial position, movement pattern, and visual characteristics, stored in nested dictionaries or lists. The factory layout can be stored as a grid or graph data structure that defines spatial relationships and pathways within the environment. These structured data objects enable the simulation instance to dynamically update the state of the digital twin and synthetic obstacles based on real-time inputs, thereby providing an accurate and flexible testing environment for enhancing the AGV's resilience.

[0081] Simulating movement involves virtually repheating one or more actions of the Siemens Aktiengesellschaft

[0082] 16

[0083] AGV in response to the generated navigational decision within the simulation instance. For instance, if the navigational decision dictates that the AGV should turn right to avoid a synthetic obstacle, the simulation will show the digital twin of the AGV executing this turn within the virtual factory environment.

[0084] The method further comprises simulating the movement of the AGV in response to the generated navigational decision. The movement is simulated within the simulation instance. The method further comprises monitoring the simulated movement of the AGV in the simulation instance. The simulated movement is monitored based on a plurality of simulated performance metrics of the AGV.

[0085] The plurality of simulated performance metrics are quantifiable measures used to evaluate the AGV's performance within the simulation instance. Examples of plurality of simulated performance metrics include a number of collisions with synthetic obstacles, a number of times the AGV had to stop, and a number of deviations from the intended path. For instance, if the digital twin of the AGV collides with a synthetic cart three times during the simulation, this collision count is recorded as a performance metric. Similarly, if the AGV stops five times or deviates from its planned route twice, these instances are also recorded and analyzed.

[0086] Monitoring the simulated movement based on the plurality of simulated performance metrics allows for a thorough evaluation of the navigation model of the AGV and navigational strategies. Thus, the automation module is configured to identify potential improvements and adjustments needed before deploying the AGV in the actual factory environment. By using the digital twin and simulated performance metrics, the automation module ensures a comprehensive and risk- free evaluation of the AGV's capabilities, ultimately enhancing its real-world performance and reliability.

[0087] The method further comprises determining a resilience parameter of the navigation model based on an analysis of the plurality of real-world performance metrics and the plurality of simulated performance metrics. The resilience parameter is a quantifiable measure that reflects a robustness and adaptability of the navigation model in handling various scenarios and obstacles in the factory environment. For example, the resilience parameter can be represented as a numerical score, where higher values indicate better performance and greater adaptability. Siemens Aktiengesellschaft

[0088] 17

[0089] The method further comprises comparing the determined resilience parameters with a threshold.

[0090] The threshold is a predefined value that the resihence parameter is compared against to evaluate the navigation model's effectiveness. The threshold is determined by the automation module based on historical data, expert judgment, and one or more specific requirements of the factory environment. For instance, if the historical data shows that an AGV with a resilience parameter above 80 operates efficiently without frequent collisions or deviations, the threshold may be set at 80. In one example, the threshold is received from a user.

[0091] The advantages of using both the plurality of real-world performance metrics and the plurality of simulated performance metrics to determine the resihence parameter, instead of only the plurality of simulated performance metrics, are significant. By incorporating real-world performance metrics, the automation module captures actual operational conditions and challenges faced by the AGV. Thus, the resihence parameter accurately reflects the navigation model performance in both simulated and real- world environments. For example, real- world performance metrics might reveal issues such as unexpected obstacles or changes in the factory environment that were not present in the simulation, providing a more holistic evaluation.

[0092] When the resilience parameter is lesser than the threshold, it signifies that the navigation model is not performing optimally and may struggle to handle certain scenarios or obstacles effectively. This determination indicates that the model requires further refinement to improve its robustness and adaptability.

[0093] Initiating the reinforcement learning procedure only after determining that the resilience parameter is lesser than the threshold offers several advantages. This targeted approach ensures that computational resources are used efficiently, focusing on improving the navigation model only when necessary. For example, if the resilience parameter falls below the threshold, the reinforcement learning procedure can be initiated to retrain the model, incorporating new data and scenarios to enhance its performance. Thus, a more resilient and capable navigation model is generated, ultimately improving an ability of the AGV to navigate the factory environment safely and efficiently. In other words, the method further comprises initiating a reinforcement learning procedure to retrain the navigation model based on an analysis of the generated navigational decision. Siemens Aktiengesellschaft

[0094] 18

[0095] The method further comprises deploying the retrained navigation model in an operational environment of the AGV after completion of the reinforcement learning procedure. Retraining the navigation model through the reinforcement learning procedure offers several advantages. The retraining process allows the model to learn from new data, including recent real-world performance metrics and simulated performance metrics, thereby improving its ability to handle various scenarios and obstacles more effectively. For example, if the AGV encountered a previously unseen obstacle during its operations, the retrained model would incorporate information about the unseen obstacle, enabling the AGV to navigate similar obstacles more efficiently in future.

[0096] Deploying the retrained navigation model in the operational environment enhances the AGV's performance by leveraging the updated strategies and knowledge gained during the reinforcement learning procedure. The retrained model is better equipped to make accurate and timely navigational decisions, reducing the likelihood of collisions, stops, and deviations. For instance, if the retrained model has learned to recognize and avoid a new type of obstacle that frequently appears in the factory, the AGV will navigate around it more smoothly, improving overall operational efficiency.

[0097] The method further comprises displaying a resilience report based on the determined resilience parameter. The displayed resilience report is a comprehensive document that provides detailed insights into the performance and robustness of the navigation model. The resilience report is generated based on the determined resilience parameter, which quantifies how effectively the navigation model handles various scenarios and obstacles.

[0098] The resilience report contains a detailed analysis of the plurality of real-world performance metrics and the plurality of simulated performance metrics. The resilience report includes counts of collision occurrences, stop times, and deviations from the intended path of the AGV. For example, the resilience report may present data showing that the AGV experienced five collisions, ten stop times, and three deviations during a specific evaluation period. Additionally, the resilience report includes graphical representations such as charts and graphs to visualize the performance trends over time.

[0099] Moreover, the resilience report offers actionable insights and suggestions for optimizing a performance of the AGV. For example, the resilience report might recommend retraining the navigation model with additional data or adjusting certain parameters to improve obstacle detection accuracy. The resilience report Siemens Aktiengesellschaft

[0100] 19 also includes comparisons between a performance of the AGV in the real-world and simulated environments, providing a holistic view of its capabilities and areas for enhancement.

[0101] The present invention offers numerous advantages that significantly enhance an operational efficiency and safety of AGVs on the factory environment. A primary benefit is an ability to conduct real-time testing in an actual operational environment of the AGV, rather than relying solely on simulated environments. Thus, the AGV is exposed to realistic and dynamically changing conditions, allowing them to better adapt to unforeseen obstacles and changes in the factory layout. By introducing the plurality of synthetic obstacles directly into the visual feed captured by the at least one sensor of the AGV, the automation module can test the AGV's response to a wide range of scenarios without need for physical alterations to the factory environment. The present invention not only saves time and resources but also minimizes a risk of damage to the AGV and factory equipment during testing.

[0102] The present invention for continuous and comprehensive monitoring of the AGV's performance by collecting detailed response data to each synthetic obstacle. This data-driven approach enables the identification of specific weaknesses in the AGV's navigation and navigation models, facilitating targeted retraining and improvement. By utilizing the central monitoring station (CMS) to manage the retraining, the automation module ensures seamless integration and coordination, providing a robust framework for evaluating and enhancing a resilience of the AGV. Additionally, a flexibility to conduct resilience testing before planned changes in the factory environment, such as altering the color of workers' uniforms, ensures that AGV is adequately prepared for new conditions, thereby reducing operational downtime and financial losses. Overall, the automation module offers a proactive and efficient solution to improving the adaptability and robustness of AGVs, leading to safer and more reliable operations in dynamic factory environments.

[0103] The present invention incorporates several advanced technical features that enhance its efficacy and robustness. One key feature is the use of the sophisticated automation module that orchestrates entire testing process by dynamically inserting the plurality of synthetic obstacles into the visual feed and collecting detailed performance metrics. The automation module leverages high- performance image processing units and machine learning algorithms to generate realistic the plurality of synthetic obstacles that accurately mimic real- world conditions. Additionally, the automation module employs state-of-the-art Siemens Aktiengesellschaft

[0104] 20 reinforcement learning models to analyze navigation process of the AGV in response to the plurality of synthetic obstacles, providing precise feedback for further training. The visual feed is rerouted through the automation module using secure, lowlatency communication protocols, ensuring seamless integration and real-time processing.

[0105] Introduction of the plurality of synthetic obstacles into the factory environment allows for comprehensive testing of response mechanisms of the AGV in a controlled manner, without risking physical damage and harm to one or more factory personnel or industrial devices in the factory environment. Thus, the AGV is enabled to navigate around new and unexpected obstacles with greater precision, thereby reducing a likelihood of collisions that could result in injuries or damage to valuable assets. An ability of the automation module to simulate a wide range of scenarios, including potential safety hazards, allows for thorough validation of the AGV's navigation algorithms, ensuring that the vehicle can make safe and effective navigational choices in real-time. Enhanced monitoring tools, such as real-time error detection and decision latency tracking, further contribute to the overall safety by providing continuous oversight of the AGV's operations. By addressing these safety concerns, the invention not only improves operational efficiency but also fosters a safer work environment, promoting confidence in the deployment of AGVs within dynamic and complex factory settings.

[0106] In one example, the plurality of synthetic objects are inserted by the automation module into the visual feed in real-time, during an operation of the AGV in the factory environment. The automation module is thus enabled to perform dynamic testing of an ability of the AGV to navigate around new and unforeseen obstacles without disrupting the factory environment. The visual feed may be a live visual feed captured by the at least one sensor in real-time. By integrating synthetic obstacles into the live visual feed, the AGV is configured to perceive the inserted plurality of synthetic obstacles as part of a natural surroundings of the AGV, providing a more accurate assessment of its resilience and navigation capabilities in real-time.

[0107] In one example, the Central Monitoring Station (CMS) is configured to reroutes the visual feed from the AGV, and processes the visual feed to insert the plurality of synthetic obstacles, and then sends the modified visual feed back to the AGV. Thus, the AGV receives a continuous stream of the modified visual data which has the plurality of synthetic obstacles. The CMS also collects detailed logs of a response of the AGV, including actions taken, navigation times, and other critical Siemens Aktiengesellschaft

[0108] 21 performance metrics. Thus, the CMS incorporates a real-time feedback loop which allows for immediate evaluation and adjustment of the navigation algorithm of the AGV, enhancing its ability to adapt to changing conditions on the factory floor.

[0109] In one example, the automation module is enabled to conduct scenario-based resilience testing before implementing planned changes in the factory environment. For instance, if there is a plan to change the color of workers' uniforms, synthetic human figures with the new uniform color can be inserted into the visual feed of the AGV. This proactive testing ensures that the AGV can handle such changes without issues, thereby minimizing operational downtime and potential collisions. By simulating planned changes, the automation module is enabled to gauge a preparedness of the AGV as necessary, ensuring seamless adaptation to new conditions.

[0110] In one example, the CMS comprises a comprehensive library of synthetic objects, which can be used to create specific test cases for resilience testing. The synthetic objects are meticulously designed to mimic real-world obstacles, including their shapes, sizes, and movement patterns. Each test case is logged with a response of the AGV, including successful navigation or collision, which helps in identifying weaknesses in an algorithm of the AGV. This structured approach to resilience testing allows for targeted improvements and more effective retraining of the AGV, ensuring it can handle a wide range of scenarios.

[0111] In one example, the automation module is configured to execute batch processing of the plurality of visual frames to give the AGV a perception of encountering a real 3D object. By processing multiple frames, we create a dynamic and realistic representation of the synthetic obstacle, ensuring that the AGV perceives them as genuine parts of the factory environment.

[0112] Detailed Data Collection and Analysis for Retraining

[0113] In one example, the resilience report comprises logs of interactions of the AGV with the plurality of synthetic obstacles. Each interaction of the plurality of synthetic objects is considered as a test case of a plurality of test cases. Each test case is marked as failed if the AGV collides with the synthetic obstacle or temporarily halts an operation of the AGV. Thus, the resilience report comprises a robust dataset that can be used to retrain the navigation model of the AGV. By analyzing the resilience report, a user is enlightened about specific areas where a resilience of the AGV needs improvement and thereby enables the user to Siemens Aktiengesellschaft

[0114] 22 implement targeted training to address one or more weaknesses of the AGV.

[0115] The object of the present invention is also achieved by a central monitoring station for retraining an Automated Guided Vehicle (AGV) in a factory environment. The central monitoring station comprises one or more processors and a memory coupled to the processors. The memory comprises an automation module stored in the form of machine-readable instructions executable by the processors. The automation module is configured for performing the method as described above.

[0116] The object of the present invention is also achieved by a factory environment. The factory environment comprises a central monitoring station, and an Automated Guided Vehicle. The central monitoring station is configured to perform the above-described method steps.

[0117] The object of the present invention is also achieved by a computer-program product, having machine-readable instructions stored therein, that when executed by a processing unit is configured to perform the above-described method steps.

[0118] The object of the present invention is also achieved by an automated guided vehicle which comprises an automation module. The automation module is configured to perform the above-described method steps.

[0119] The above-mentioned and other features of the invention will now be addressed with reference to the accompanying drawings of the present invention. The illustrated embodiments are intended to illustrate, but not limit the invention.

[0120] FIG 1A is a block diagram of a factory environment capable of retraining an Automated Guided Vehicle (AGV), according to an embodiment of the present invention!

[0121] FIG. IB is an exemplary illustration of insertion of a plurality of synthetic obstacles in a visual feed of an Automated Guided Vehicle (AGV), according to an embodiment of the present invention!

[0122] FIG 2 is a block diagram of an central monitoring station, such as those shown in FIG. 1, in which an embodiment of the present invention can be implemented;

[0123] FIG 3 is a block diagram of an automation module, such as those shown in FIG 2, in which an embodiment of the present invention can be implemented! and Siemens Aktiengesellschaft

[0124] 23

[0125] FIG 4 is a process flowchart illustrating an exemplary method of retraining an Automated Guided Vehicle (AGV), according to an embodiment of the present invention.

[0126] Various embodiments are described with reference to the drawings, wherein like reference numerals are used to re-fer the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for the purpose of explanation, numerous specific details are set forth in order to provide thorough understanding of one or more embodiments. It may be evident that such embodiments may be practiced without these specific details.

[0127] FIG 1A is a block diagram of a factory environment 100 capable of retraining an Automated Guided Vehicle (AGV) 104, according to an embodiment of the present invention.

[0128] The factory environment 100 comprises a central monitoring station (CMS) 102 the AGV 104 and a real-world object 110. The central monitoring station 102 comprises an automation module 112 and a processing unit 202. The AGV 104 comprises at least one sensor 106 which is configured to capture visual data from a field of view 118 of the AGV 104. The AGV 104 further comprises a navigation model 108. The AGV 104 is configured to communicate with the CMS 102 via one or more wired or wireless communication protocols.

[0129] In one example, the CMS 102 may be incorporated as a software or a hardware module within the AGV 104. In such a case, the automation module will be implemented as a part of the AGV 104.

[0130] Referring to FIG IB, a visual feed 120 captured by the at least one sensor 106 is shown. The visual feed 120 comprises a set of visual frames 120A. Each visual frame of the set of visual frames 120A comprises the visual data captured by the at least one sensor at a specific instance of time. The automation module 112 is configured to dynamically insert the a plurality of synthetic obstacles 114 into the set of visual frames 120A to generate a modified set of visual frames 122. Each visual frame of the modified set of visual frames 122 comprises the plurality of synthetic obstacles in a different configuration and position. For example, the modified set of visual frames 122 has the plurality of synthetic obstacles 114 in a first configuration and position 114A in a first visual frame, a second configuration and position 114B in a second frame, and a third configuration and position 114A in a third visual frame. The automation module 112 inserts the Siemens Aktiengesellschaft

[0131] 24 plurality of synthetic obstacles 114 in the visual feed 120 to generate a modified visual feed 116.

[0132] Referring back to FIG. 1A, the modified visual feed 116 comprises an image 110A of the real-world obstacle 110. The factory environment 100 refers to an industrial setting where manufacturing or assembly processes occur. Examples of the factory environment 100 include automotive assembly plants, electronics manufacturing facilities, and food processing units. The factory environment 100 typically consists of various machines, workstations, storage areas, and pathways for personnel and equipment movement. The Automated Guided Vehicle (AGV) 104 operate within the factory environment 100 to transport materials, components, or finished products between different locations on a factory floor within the factory environment 100. The AGV 104 is a mobile robot used in industrial applications for material handling and transportation tasks. Examples of the AGV 104 include forklift AGVs that transport pallets, tugger AGVs that pull carts, and unit load AGVs that carry single loads. The AGV 104 is configured to navigate autonomously using the at least one sensor 106, and the navigation model 108 to avoid obstacles and follow predetermined paths.

[0133] The automation module 112 further causes the processing unit 202 to implement a method of retraining the AGV 104. The automation module 112 comprises computer-readable code that can be executed by the processing unit 202 to implement the method, ensuring that the AGV 104 is retrained properly. The automation module 112 is configured to receive the visual feed 120 from sensors 106 mounted on the AGV 104, insert the synthetic obstacles 114 into the visual feed 120 to simulate real-world conditions, and analyze the modified visual feed 116 using a reinforcement learning-based navigation model 108. The automation module 112 further causes the processing unit 202 to generate a plurality of navigational decisions and initiates a reinforcement learning procedure to retrain the navigation model 108 based on one or more performance metrics of the AGV 104. The retrained navigational model 108 is then deployed in an operational environment of the AGV 104, enhancing the resilience of the AGV 104 to new obstacles and changes within the factory environment 100.

[0134] The automation module 112 causes the processing unit 202 to receive the visual feed 120 from at least one sensor 106. The at least one sensor 106 is configured to capture visual data of the factory environment 100. The visual data represents the surrounding of the AGV 104. The visual data comprises images with depth view, images without depth view, and point clouds generated by sensors like Siemens Aktiengesellschaft

[0135] 25

[0136] LIDAR. The captured visual data is crucial for the AGV 104 to understand and navigate the factory environment 100 accurately.

[0137] An example of the at least sensor 106 is at least one of a LIDAR sensor or a depth-sensing camera. A LIDAR sensor, which stands for Light Detection and Ranging, is configured to emit pulses of LASER and measures the time taken by the LASER to return after hitting an object. The measured time is used to create detailed 3D maps of the factory environment 100. The depth-sensing camera is configured to capture images that include depth information, allowing the AGV 104 to perceive the distance to various objects in the field of view 118 of the AGV 104. In one example, the sensor 106 is mounted on the AGV 104. In one example, the at least one sensor 106 is configured to capture the field of view 118 from the perspective of the AGV 104. The at least one sensor 106 enables the AGV 104 to continuously monitor a surrounding of the AGV 104 and detect any obstacles or changes in the factory environment 100.

[0138] The received visual feed 120 comprises a continuous stream of visual data captured by the at least one sensor 106. The visual feed 120 comprises the set of visual frames 120A. Each visual frame of the set of visual frames 120A represents a snapshot of the surrounding of the AGV 104 at a specific time. The set of visual frames 120A can be represented as data objects in various formats, such as JPEG images for standard pictures, PNG images for depth maps, or PCD files for point clouds generated by LIDAR sensors.

[0139] The automation module 112 further causes the processing unit 202 to receive one or more user- defined criteria associated with the plurality of synthetic obstacles 114 which should be used to test / train the AGV 104. The plurality of synthetic obstacles 114 comprises one or more digital representations of real-world obstacles.

[0140] In one example, the plurality of synthetic obstacles 114 comprises three- dimensional models which are configured to move. In one example, a shape and a position of at least one of the plurality of synthetic obstacles 114 changes with time. The plurality of synthetic obstacles 114 can be implemented in different formats depending on the type of visual feed 120 received by the automation module 112. In a first scenario in which the visual feed 120 is a video, the plurality of synthetic obstacles 114 are embedded into one or more video frames of the visual feed 120, thereby giving the AGV 104 an illusion of encountering real obstacles in the factory environment 100. For example, digital representations of boxes, human workers, or machinery can be inserted into the Siemens Aktiengesellschaft

[0141] 26 video feed, and these synthetic obstacles 114 can move or change positions to simulate dynamic factory conditions. In a second scenario in which the visual feed 120 is a point cloud or a depth map, the plurality of synthetic obstacles 114 are represented as three-dimensional data points or depth information. For instance, LID AR-generated point clouds might have additional clusters of points representing pallets or carts, while depth maps could include simulated depth variations corresponding to objects like forklifts or conveyor belts.

[0142] The plurality of synthetic obstacles 114 are used for testing and retraining the AGV 104. Using the plurality of synthetic obstacles 114 for training rather than using real- world obstacles offers several advantages. The plurality of synthetic obstacles 114 allow for controlled and repeatable testing conditions, ensuring that the AGV 104 encounters a wide range of scenarios that might not be easily replicated in a real-world environment. Use of the plurality of synthetic obstacles 114 also eliminates the risk of damaging the AGV 104 or factory equipment during training since the plurality of synthetic obstacles 114 are virtual. Additionally, the plurality of synthetic obstacles 114 can be quickly modified to test various configurations and behaviors, making the training process more efficient and comprehensive.

[0143] In one example, the received one or more user-defined criteria specify one or more characteristics and behaviors of at least one of the plurality of synthetic obstacles 114. The one or more user-defined criteria are parameters set by a user to generate the plurality of synthetic obstacles 114 according to specific training needs. In one example, the one or more user-defined criteria are received through a user interface or configuration file where a user inputs one or more desired parameters.

[0144] The types of the one or more user-defined criteria associated with each synthetic obstacle 114 include a specific type of obstacle to be simulated using the synthetic obstacle 114, a location within the factory environment 100 where the synthetic obstacle 114 is to appear, a timing and duration for which the synthetic obstacle 114 is to be present in the visual feed 120, a movement pattern of the synthetic obstacle 114, and an interaction behavior of the synthetic obstacle 114, including changes in shape, size, or color over time.

[0145] For example, in a context of the factory environment 100, the one or more user- defined criteria may define a criterion to simulate a specific type of obstacle such as a moving cart. Further, the one or more user-defined criteria may specify the location within the factory environment 100 where the synthetic obstacle 114 Siemens Aktiengesellschaft

[0146] 27 should appear, such as a high-traffic intersection. Furthermore, the one or more user-defined criteria may also set the timing and duration for which the synthetic cart is present in the visual feed 120, such as during peak operation hours. Additionally, the one or more user-defined criteria may define a movement pattern for the synthetic cart, such as moving back and forth between two points. Interaction behavior criteria may include changes in the cart's size or color to simulate different types of loads or conditions.

[0147] The automation module 112 further causes the processing unit 202 to receive one or more user- defined criteria associated with the plurality of synthetic obstacles 114 which should be used to train the AGV 104. Each of the plurality of synthetic obstacles 114 is associated with at least one user-defined criterion of the one or more user-defined criteria. For example, a synthetic obstacle 114 representing a worker in a different colored uniform could be associated with the one or more user- defined criteria specifying a uniform color, a worker's location, and a duration the worker remains in a pathway of the AGV 104. By incorporating the one or more user-defined criteria, the automation module 112 ensures that the AGV 104 is exposed to realistic and relevant training scenarios, enhancing its ability to navigate and perform tasks efficiently in the ever-changing factory environment 100.

[0148] The automation module 112 further causes the processing unit 202 to utilize a Large Language Model (LLM) to generate each of the plurality of synthetic obstacles 114 based on the one or more user-defined criteria. The Large Language Model (LLM) is an advanced Artificial Intelligence model trained on vast amounts of data to understand and generate complex outputs, including graphical objects. The LLM generates the plurality of synthetic obstacles 114 by interpreting the one or more user-defined criteria and creating detailed graphical representations and configurations of the obstacles accordingly. Examples of LLMs capable of generating graphical objects include DALL-E, which is configured to generate detailed and contextually accurate images from textual descriptions. DALL-E is configured to analyze user-defined criteria, such as the type, size, movement patterns, and positions of obstacles, and produce highly detailed graphical objects that can be directly used in the AGV's 104 visual feed 120. Another example is GauGAN, which is configured to convert text-based scene descriptions into realistic images, enabling the creation of the plurality of synthetic obstacles 114 that closely mimic real-world conditions. Additionally, models like DGato, which is designed to handle multiple types of data and tasks, can be leveraged for their versatility in generating complex graphical objects based on textual inputs. Siemens Aktiengesellschaft

[0149] 28

[0150] The LLM is configured to generate graphical representations of the plurahty of synthetic obstacles 114. For instance, given a user-defined criterion to simulate a moving cart that changes color from red to blue every 10 seconds, the LLM is configured to generate a sequence of images that visually depict the appearance and behavior of the cart over time. Similarly, the LLM is enabled to be fine-tuned to understand specific factory-related terminology and generate appropriate graphical obstacles that align with the factory environment 100. By leveraging these advanced LLMs, the automation module 112 ensures that the plurality of synthetic obstacles 114 are not only realistic but also varied and contextually accurate, providing a comprehensive training environment that enhances the ability of the AGV 104 to navigate and perform tasks efficiently in dynamic factory settings.

[0151] The LLM is configured to generate the plurality of synthetic objects 114 after receiving one or more prompts. The prompts fed into the LLM are generated based on the one or more user-defined criteria. For instance, if a user defines a criterion for simulating a moving cart, the prompt comprises details such as the type of cart, the location of the cart within the factory environment 100, the movement pattern of the cart, and any changes in the appearance of the cart over time. The LLM processes the prompts to generate a detailed graphical representation and configuration of the synthetic cart, which is then used to create the plurality of synthetic obstacles 114 associated with the cart, in the visual feed 120.

[0152] For example, when the user specifies a moving cart, the prompt might be: "Generate a moving cart that travels between two points in the factory, changing its color from red to blue every 10 seconds." If the user wants the plurality of synthetic obstacles 114 to appear in a specific location within the factory environment 100, such as a high-traffic intersection, the prompt might be: "Create a synthetic worker standing at the high-traffic intersection near the assembly line, wearing a yellow uniform." When specifying the timing and duration for which the plurality of synthetic obstacles 114 is to be present in the visual feed 120, the prompt might be: "Simulate a forklift appearing in the visual feed 120 for 30 seconds during peak operation hours, moving from the storage area to the loading dock." If the user defines a movement pattern of the plurality of synthetic obstacles 114, such as a zigzag pattern, the prompt might be: "Generate a synthetic pallet that moves in a zigzag pattern across the factory floor, avoiding obstacles and changing direction every 5 meters." Siemens Aktiengesellschaft

[0153] 29

[0154] In one example, the prompts are generated by using one or more prompt templates. For example, a prompt template for a moving cart might be: "Generate a [type of obstacle] that [movement pattern] between [start point] and [end point], changing its [property] from [initial value] to [final value] every [time interval]." If the user specifies a moving cart, the filled-in prompt would be: "Generate a moving cart that travels between two points in the factory, changing its color from red to blue every 10 seconds." Similarly, for a synthetic worker, the prompt template might be: "Create a synthetic [type of worker] standing at [specific location] near [landmark], wearing a [uniform color] uniform." If the user wants the plurality of synthetic obstacle 114 to appear in a specific location within the factory environment 100, such as a high-traffic intersection, the filled- in prompt would be: "Create a synthetic worker standing at the high-traffic intersection near the assembly line, wearing a yellow uniform." When specifying the timing and duration for which the synthetic obstacle 114 is to be present in the visual feed 120, the prompt template might be: "Simulate a [type of obstacle] appearing in the visual feed 120 for [duration] during [time period], moving from [start location] to [end location]." The filled-in prompt would be: "Simulate a forklift appearing in the visual feed 120 for 30 seconds during peak operation hours, moving from the storage area to the loading dock."

[0155] For defining a movement pattern of the plurality of synthetic obstacles 114, the prompt template might be: "Generate a synthetic [type of obstacle] that moves in a [movement pattern] across [location], avoiding obstacles and changing direction every [distance]." The filled-in prompt would be: "Generate a synthetic pallet that moves in a zigzag pattern across the factory floor, avoiding obstacles and changing direction every 5 meters."

[0156] The automation module 112 further causes the processing unit 202 to determine, from the received visual feed 120, the set of visual frames 120A for the plurality of synthetic obstacles 114. Each visual frame of the set of visual frames is a single image or snapshot captured by the at least one sensors 106 mounted on the AGV 104, representing the surrounding of the AGV 104 at a specific moment in time. Examples of the visual frame include individual video frames in a continuous video feed or singular depth images in a depth map. The set of visual frames 120A is determined based on an analysis of the one or more user-defined criteria associated with the synthetic obstacle 114. For example, if the user- defined criteria specify that the plurality of synthetic obstacles 114 should appear at a high-traffic intersection within the factory environment 100, the set of visual frames 120A will include all frames captured by the AGV's 104 sensors 106 when the AGV 104 is approaching and navigating through the high-traffic intersection. Siemens Aktiengesellschaft

[0157] 30

[0158] If the criteria specify a timing and duration, such as a synthetic cart appearing for 30 seconds during peak operation hours, the set of visual frames 120A will include all frames captured within that specific time window.

[0159] To determine the set of visual frames 120A, the automation module 112 further causes the processing unit 202 to analyze the one or more user-defined criteria using a technical solution such as a machine learning algorithm or a rule-based engine. The machine learning algorithm or the rule-based engine processes the one or more user- defined criteria to identify the relevant frames in the visual feed 120. For example, when analyzing criteria based on location, the system may use image recognition techniques to identify frames where the AGV 104 is in the specified location. For timing-based criteria, the automation module 112 is configured to use timestamps to filter the set of visual frames 120A within a specified duration. For criteria based on the movement pattern of the synthetic obstacle 114, the automation module 112 is configured to track the movement of the AGV 104 and select frames that align with the movement pattern.

[0160] By determining the set of visual frames 120A based on an analysis of the one or more user-defined criteria, the automation module 112 ensures that the plurality of synthetic obstacles 114 are accurately and contextually embedded into the visual feed 120. Thus, the automation module 112 enhances the training process by exposing the AGV 104 to realistic and relevant scenarios, thereby improving the ability of the AGV 104 to navigate and perform tasks efficiently in the dynamic factory environment 100.

[0161] The automation module 112 further causes the processing unit 202 to generate a series of configurations and a series of positions of the synthetic obstacle 114 by applying the LLM on the one or more user-defined criteria. Each of the series of configurations and the series of positions is indicative of a configuration or position of the synthetic obstacle 114 at a time instance corresponding to each frame of the set of visual frames 120A. The series of configurations refers to the different forms or states that the synthetic obstacle 114 can take across various frames. For example, if the synthetic obstacle 114 is a moving cart, the series of configurations may include the cart in different orientations, such as facing forward, turning left, or turning right. The series of positions refers to the specific locations of the synthetic obstacle 114 within the visual frame at each time instance. For the moving cart, the series of positions may indicate the cart's location at different points along its path, such as at the start, midpoint, and end of its journey. The term "series" is used because the configurations and positions change over time, creating a sequence that aligns with the progression of visual Siemens Aktiengesellschaft

[0162] 31 frames. This sequence allows the synthetic obstacle 114 to appear dynamic and realistic, simulating real-world conditions more accurately.

[0163] Generating the series of configurations and positions offers several advantages. One advantage is the ability to create more complex and varied scenarios for training the AGV 104. By having a series of configurations, the synthetic obstacle 114 can exhibit different behaviors and states, challenging the AGV 104 to adapt to a wide range of situations. Another advantage is the enhanced realism of the training environment. A moving cart that changes its orientation and position over time provides a more lifelike simulation, improving the ability of the AGV 104 to handle real-world obstacles.

[0164] For example, consider a synthetic worker that needs to appear in the visual feed 120. The series of configurations comprises the synthetic worker standing, bending, and walking, while the series of positions is indicative of the location of the worker at different points along a designated pathway in the factory environment 100. By embedding these configurations and positions into the visual frames, the AGV 104 encounters a dynamic and realistic obstacle, enhancing the training experience of the AGV 104.

[0165] The automation module 112 further causes the processing unit 202 to apply the LLM to generate the series of configurations and positions for each synthetic obstacle 114 based on the one or more user-defined criteria. The LLM processes the one or more user- defined criteria and produces detailed descriptions of the synthetic obstacle's states and locations, which are then embedded into the visual frames.

[0166] The automation module 112 further causes the processing unit 202 to embed each synthetic obstacle of the plurality of synthetic obstacles 114 into an appropriate visual frame of the set of visual frames 120A, based on the generated series of configurations and the series of positions of the plurality of synthetic obstacles 114. In the case where the visual feed 120 is a video feed, the embedding process involves inserting the plurality of synthetic obstacle 114 into each frame of the video at a specified position and configuration. The insertion is performed frame by frame, ensuring that the plurality of synthetic obstacles 114 appears consistently and realistically in the video feed. For example, if the synthetic obstacle 114 is a moving cart, the cart is rendered into each frame at the correct location and orientation according to the series of positions and configurations. Siemens Aktiengesellschaft

[0167] 32

[0168] In the case where the visual feed 120 is a 3D depth map, the embedding process involves integrating the plurality of synthetic obstacles 114 into the 3D space represented by the depth map. The integration requires adjusting the depth values to reflect the presence of the plurality of synthetic obstacle 114 at the specified positions. For instance, if the synthetic obstacle 114 is a worker, depth values at the worker's location are modified to match the dimension and shape of the worker, ensuring that the depth map accurately represents the presence of the plurality of synthetic obstacles 114 in the 3D environment.

[0169] Examples of algorithms used to embed the plurality of synthetic obstacles 114 include image compositing techniques such as alpha blending and depth-based compositing. Alpha blending involves combining the synthetic obstacle 114 with the visual frame using transparency values to create a seamless integration. Depth-based compositing uses the depth information from the 3D depth map to accurately place the synthetic obstacle 114 within the 3D space, ensuring that it interacts correctly with other objects in the environment. Other algorithms such as 3D rendering and ray tracing can also be used to achieve realistic embedding of synthetic obstacles 114 in both 2D and 3D visual feeds.

[0170] By embedding the synthetic obstacle 114 into the visual frame 120A based on the generated series of configurations and the series of positions, the automation module 112 ensures that the AGV 104 encounters realistic and varied training scenarios. This comprehensive training environment enhances the AGV's 104 ability to navigate and perform tasks efficiently in the dynamic factory environment 100, ultimately improving its resilience and adaptability.

[0171] In other words, the automation module 112 further causes the processing unit 202 to insert the plurality of synthetic obstacles 114 into the received visual feed 120 to generate the modified visual feed 116 comprising the plurality of synthetic obstacles 114. The plurality of synthetic obstacles 114 are inserted into the visual feed 120 to simulate real-world conditions in the factory environment 100.

[0172] The automation module 112 further causes the processing unit 202 to detect the plurality of synthetic obstacles 114 by applying an object detection algorithm on the modified visual feed 116. The object detection algorithm is a machine learning model designed to identify and locate objects within an image or a sequence of images. The object detection algorithm is trained using a large dataset of annotated images, where each image contains labeled instances of various objects along with a plurality of bounding boxes. The plurality of bounding boxes define positions of the objects within the images. Siemens Aktiengesellschaft

[0173] 33

[0174] For example, an object detection algorithm such as YOLO (You Only Look Once) can be trained using a dataset that includes thousands of images of factory environments, each annotated with labels for objects such as workers, carts, pallets, and machinery. The training process involves feeding these annotated images into the algorithm, which learns to recognize the patterns and features associated with each type of object. The object detection algorithm adjusts internal parameters to minimize the difference between predictions and actual labels in the training data.

[0175] The object detection algorithm provides output in the plurality of bounding boxes around the detected plurality of synthetic obstacles 114, along with confidence scores that indicate the likelihood that the identified obstacles match specified labels. For instance, after processing a visual frame of the modified set of visual frames 122, the object detection algorithm is configured to output bounding boxes around the detected plurality of synthetic obstacles 114, labeling one as a worker with a confidence score of 0.95 and the other as a heart shaped object with a confidence score of 0.90.

[0176] In the context of the automation module 112, the object detection algorithm processes the modified visual feed 116, which includes the plurality of synthetic obstacles 114. The object detection algorithm scans each visual frame of the modified visual feed 116, identifying and locating the plurality of synthetic obstacles 114 based on patterns and features. For example, when the object detection algorithm processes a frame that includes a synthetic worker, it outputs a bounding box around the worker, along with a high confidence score indicating the accuracy of detection.

[0177] The automation module 112 further causes the processing unit 202 to analyze the detected plurality of obstacles 114 using a reinforcement learning-based navigation model 108 to generate a navigation decision. The reinforcement learning-based navigation model 108 is an advanced Al model that is configured to make navigational decisions by interacting with the factory environment 100 and receiving feedback in the form of rewards or penalties. The navigation model 108 uses feedback to improve a navigation process of the AGV 104 over time, aiming to maximize cumulative rewards.

[0178] Examples of the reinforcement learning-based navigation model 108 include Deep Q-Networks (DQN). The Deep Q-Networks (DQN) is trained to navigate the AGV 104 within the factory environment 100. The navigation model 108 receives Siemens Aktiengesellschaft

[0179] 34 input from the modified visual feed 116 with the detected plurality of synthetic obstacles 114 and outputs navigational commands to control the movement of the AGV 104. One or more actions of the AGV 104, such as moving forward, turning left, or stopping, are evaluated based on the effectiveness of the one or more actions in causing the AGV 104 to avoid obstacles and reach a target location. The navigation model 108 is configured to continuously learn and adjust strategies to improve performance.

[0180] The navigation model 108 is used to control and navigate the AGV 104 in the factory environment 100. For example, if the object detection algorithm detects a synthetic obstacle 114 such as a moving cart, the navigation model 108 processes information associated with the detected plurality of synthetic obstacle 114 and determines the best course of action to avoid a collision. The navigation model 108 is configured to consider the current position and speed of the AGV 104, the location of the cart, and the layout of the factory environment 100 to make an informed decision.

[0181] Examples of the navigation model 108 include algorithms such as Proximal Policy Optimization (PPO) or Actor-Critic methods. The navigation model 108 is configured to evaluate multiple potential actions and select an action that maximizes expected rewards. For instance, the PPO algorithm is configured to balance exploration and exploitation, ensuring that the AGV 104 not only follows known safe paths but also explores new routes to improve overall navigation efficiency.

[0182] The navigation decision is an output generated by the navigation model 108, specifying the next action of the AGV 104 based on the detected plurality of synthetic obstacles 114. For example, if the navigation model 108 detects a synthetic worker standing in the path of the AGV 104, the navigation decision might be to stop and wait until the path is clear. Alternatively, if the navigation model 108 detects a moving cart approaching from the left, the navigation decision might be to turn right to avoid a collision.

[0183] The navigation decision is taken based on the detection of the plurality of synthetic obstacles 114. The navigation model 108 analyzes the positions and configurations of the detected plurality of synthetic obstacles 114, considering factors such as distance, speed, and trajectory. For example, if the navigation model 108 detects multiple obstacles, such as a worker and a cart, the navigation model 108 prioritizes actions that minimize the risk of collision with both objects. The navigation model 108 evaluates potential actions, such as changing speed, Siemens Aktiengesellschaft

[0184] 35 altering direction, or stopping, and selects the action that maximizes safety and efficiency.

[0185] The automation module 112 further causes the processing unit 202 to execute the generated navigational decision by controlling the movement of the AGV 104, based on the navigational decision. The AGV 104 is equipped with a plurality of components that allow the automation module 112 to control the movement of the AGV 104. The plurality of components include motor controllers, steering mechanisms, and braking systems. For example, the motor controllers adjust the speed of the AGV 104, the steering mechanisms change the direction of the AGV 104, and the braking systems enable the AGV 104 to stop. The movement of the AGV 104 comprises actions such as moving forward, turning left or right, accelerating, decelerating, or stopping entirely.

[0186] The automation module 112 further causes the processing unit 202 to monitor the response of the AGV 104 to the navigational decision. The response is monitored using a plurality of apparatuses, including cameras and the Central Monitoring Station (CMS) 102. In one example, cameras are mounted on the AGV 104 or within the factory environment 100 to capture visual data of the movement of the AGV 104. The captured visual data is analyzed to determine whether the AGV 104 is following the navigational decisions accurately.

[0187] In another example, the Central Monitoring Station 102 is used to monitor the response of the AGV 104. The CMS 102 is an integrated system that collects and analyzes data from a plurality of sensors installed on the AGV 104. The plurality of sensors includes accelerometers, gyroscopes, and GPS modules, which provide real-time information on the speed, orientation, and position of the AGV 104. For instance, the CMS 102 is configured to detect any deviations from the intended path by comparing the actual trajectory of the AGV 104 with the planned route.

[0188] The response of the AGV 104 is monitored based on a plurality of real-world performance metrics. The plurality of real-world performance metrics refers to quantifiable measures that evaluate effectiveness and efficiency of the operation of the AGV 104. Examples of the plurality of real-world performance metrics include the count of collision occurrences, the count of stop times, and the count of deviations from the intended path of the AGV 104. The count of collision occurrences tracks the number of times the AGV 104 collides with real-world obstacles, the count of stop times records the frequency and duration of stops of the AGV 104, and the count of deviations measures how often and how far the AGV 104 strays from the planned route. Siemens Aktiengesellschaft

[0189] 36

[0190] The automation module 112 determines the plurality of real-world performance metrics by analyzing data collected from the plurality of sensors. For example, the automation module 112 uses collision sensors to count the number of collisions and analyze the video feed 120 to identify stops and deviations. By processing this data, the automation module 112 generates detailed performance reports that highlight areas for improvement.

[0191] Monitoring the response of the AGV 104 offers several advantages such as obtaining valuable insights into operational performance of the AGV 104, helping identify any issues that may affect an efficiency of the AGV 104. For instance, frequent stops or deviations might indicate obstacles that were not detected or navigational decisions that need refinement. Additionally, monitoring allows for continuous improvement of the navigation model 108 of the AGV 104 by providing real-world feedback that can be used to retrain the navigation model 108.

[0192] By executing the generated navigational decision and monitoring the response of the AGV 104 based on a plurality of real-world performance metrics, the automation module 112 ensures that the AGV 104 operates effectively within the factory environment 100. Thus, the reliability and efficiency of the AGV 104 are enhanced, leading to smoother and more productive operations in dynamic and complex factory settings.

[0193] The automation module 112 further causes the processing unit 202 to generate a simulation instance of the factory environment 100. The generated simulation instance includes the plurality of synthetic obstacles 114 and a digital twin of the AGV 104. The generated simulation instance is a virtual representation of the factory environment 100, created to replicate real-world conditions for testing and training purposes. The plurality of synthetic obstacles 114 are mapped to the simulation instance based on the one or more user-defined criteria. For example, if the user-defined criteria specify a moving cart at a high-traffic intersection, the simulation instance will include the cart positioned and moving according to the user-defined criteria. The cart's movement pattern, timing, and location will be accurately represented within the virtual environment.

[0194] The digital twin is a precise virtual model of a physical object — in this case, the AGV 104. The digital twin of the AGV 104 replicates the physical characteristics, behavior, and operational parameters of the AGV 104. The digital twin is used to simulate how the AGV 104 would interact with the plurality of synthetic Siemens Aktiengesellschaft

[0195] 37 obstacles 114 and navigate the factory environment 100. For example, the digital twin is configured to simulate the AGV's 104 response to a worker crossing its path, allowing the developers to study and improve the AGV's 104 navigation processes without risking damage to the AGV 104.

[0196] In terms of implementation, the digital twin and the simulation instance are represented as structured data objects within a simulation framework. The digital twin of the AGV 104 is implemented as a complex data structure, such as a class in an object-oriented programming language, encapsulating attributes like dimensions, sensor configurations, actuator specifications, and operational parameters. Each attribute is typically stored as a nested data structure; for instance, sensors might be represented as a dictionary with keys corresponding to sensor types and values detailing their properties. The simulation instance itself can be modeled as a composite data structure that integrates various elements of the factory environment 100, including a list of synthetic obstacles 114 and the digital twin of the AGV 104. Each synthetic obstacle 114 can be represented as an object with attributes such as type, initial position, movement pattern, and visual characteristics, stored in nested dictionaries or lists. The factory layout can be stored as a grid or graph data structure that defines spatial relationships and pathways within the environment. These structured data objects enable the simulation instance to dynamically update the state of the digital twin and synthetic obstacles 114 based on real-time inputs, thereby providing an accurate and flexible testing environment for enhancing the AGV's 104 resilience.

[0197] Simulating movement involves virtually replicating one or more actions of the AGV 104 in response to the generated navigational decision within the simulation instance. For instance, if the navigational decision dictates that the AGV 104 should turn right to avoid the plurality of synthetic obstacles 114, the simulation will show the digital twin of the AGV 104 executing this turn within the virtual factory environment 100.

[0198] The automation module 112 further causes the processing unit 202 to simulate the movement of the AGV 104 in response to the generated navigational decision. The movement is simulated within the simulation instance. The automation module 112 further causes the processing unit 202 to monitor the simulated movement of the AGV 104 in the simulation instance. The simulated movement is monitored based on a plurality of simulated performance metrics of the AGV 104. Siemens Aktiengesellschaft

[0199] 38

[0200] The plurality of simulated performance metrics are quantifiable measures used to evaluate a performance of the AGV 104 within the simulation instance. Examples of the plurality of simulated performance metrics include the number of collisions with the plurality of synthetic obstacles 114, the number of times the AGV 104 had to stop, and the number of deviations from the intended path. For instance, if the digital twin of the AGV 104 collides with a synthetic cart three times during the simulation, this collision count is recorded as a performance metric. Similarly, if the AGV 104 stops five times or deviates from its planned route twice, these instances are also recorded and analyzed.

[0201] Monitoring the simulated movement based on the plurality of simulated performance metrics allows for a thorough evaluation of the navigation model 108 of the AGV 104 and navigational strategies. Thus, the automation module 112 is configured to identify potential improvements and adjustments needed before deploying the AGV 104 in the actual factory environment 100. By using the digital twin and simulated performance metrics, the automation module 112 ensures a comprehensive and risk-free evaluation of the AGV's 104 capabilities, ultimately enhancing its real-world performance and reliability.

[0202] The automation module 112 further causes the processing unit 202 to determine a resilience parameter of the navigation model 108 based on an analysis of the plurality of real-world performance metrics and the plurality of simulated performance metrics. The resilience parameter is a quantifiable measure that reflects the robustness and adaptability of the navigation model 108 in handling various scenarios and obstacles in the factory environment 100. For example, the resilience parameter can be represented as a numerical score, where higher values indicate better performance and greater adaptability.

[0203] The automation module 112 further causes the processing unit 202 to compare the determined resilience parameter with a threshold. The threshold is a predefined value that the resilience parameter is compared against to evaluate the navigation model's 108 effectiveness. The threshold is determined by the automation module 112 based on historical data, expert judgment, and one or more specific requirements of the factory environment 100. For instance, if the historical data shows that an AGV 104 with a resilience parameter above 80 operates efficiently without frequent collisions or deviations, the threshold may be set at 80. In one example, the threshold is received from a user.

[0204] Advantages of using both the plurality of real-world performance metrics and the plurality of simulated performance metrics to determine the resilience Siemens Aktiengesellschaft

[0205] 39 parameter, instead of only the plurality of simulated performance metrics, are significant. By incorporating real-world performance metrics, the automation module 112 captures actual operational conditions and challenges faced by the AGV 104. Thus, the resilience parameter accurately reflects the navigation model 108 performance in both simulated and real- world environments. For example, real-world performance metrics reveals issues such as unexpected obstacles or changes in the factory environment 100 that were not present in the simulation, providing a more holistic evaluation.

[0206] When the resilience parameter is lesser than the threshold, it signifies that the navigation model 108 is not performing optimally and may struggle to handle certain scenarios or obstacles effectively. The automation module 112 further causes the processing unit 202 to initiate a reinforcement learning procedure to retrain the navigation model 108 if the resilience parameter is determined to be lesser than the threshold. Initiating the reinforcement learning procedure only after determining that the resilience parameter is lesser than the threshold offers several advantages. Advantageously, computational resources are used efficiently, focusing on improving the navigation model 108 only when necessary. For example, if the resilience parameter falls below the threshold, the reinforcement learning procedure can be initiated to retrain the model, incorporating new data and scenarios to enhance its performance. Thus, a more resilient and capable navigation model 108 is generated, ultimately improving the ability of the AGV 104 to navigate the factory environment 100 safely and efficiently.

[0207] In other words, the automation module 112 causes the processing unit 202 to initiate a reinforcement learning procedure to retrain the navigation model 108 based on an analysis of the generated navigational decision. The retrained navigation model 108 is deployable in the AGV (104). The reinforcement learning procedure directly addresses the real-world obstacles represented by the plurality of synthetic obstacles 114. The reinforcement learning procedure enhances the navigation model 108 by incorporating diverse and dynamic scenarios associated with the plurality of synthetic obstacles 114. The navigation model 108 is retrained to recognize, interpret, and respond to the plurality of synthetic obstacles 114 as real-world challenges, thereby improving adaptability and decision-making capabilities of the AGV 104. The retrained navigation model 108, enriched with experiences from the plurality of synthetic obstacles 114, becomes more robust and effective in navigating the factory environment 100. By deploying the retrained navigation model 108 in the AGV 104, the automation Siemens Aktiengesellschaft

[0208] 40 module 112 guarantees that the AGV 104 can handle a wide range of real-world obstacles with greater precision and efficiency.

[0209] The navigation model 108 is retrained to configure the navigation model 108 to generate an optimal response to the simulated real-world condition. In the reinforcement learning procedure, where the navigation model 108 is repeatedly exposed to the plurality of synthetic obstacles 114 embedded in the visual feed 120. The training data comprises the visual feed captured by the at least one sensor 106, each frame containing the plurality of synthetic obstacles 114 in different positions and configurations. The plurality of synthetic obstacles 114 are digital representations of a set of real-world obstacles. The plurality of synthetic obstacles 114 simulate a real- world condition. For example, the simulated real-world condition includes scenarios such as a moving cart suddenly appearing in the path of the AGV 104, with variations in direction, speed, and behavior to mimic the unpredictable nature of the factory environment 100.

[0210] During retraining, the navigation model 108 processes the modified visual feed 116 to identify the plurality of synthetic obstacles 114 and generate appropriate navigational decisions. The automation module 112 is configured to evaluate the effectiveness of the generated navigational decisions by monitoring the response of the AGV 104, including a plurality of metrics such as collision occurrences, stop times, and deviations from the intended path. The navigation model 108 receives feedback based on the plurality of metrics and uses reinforcement learning algorithms to adjust one or more training parameters and improve decision-making strategies. Such an iterative process continues until the navigation model 108 demonstrates a high level of proficiency in handling the set of real-world obstacles represented by the plurality synthetic obstacles 114.

[0211] The automation module 112 further causes the processing unit 202 to deploy the retrained navigation model 108 in an operational environment of the AGV 104 after completion of the reinforcement learning procedure. Retraining the navigation model 108 through the reinforcement learning procedure offers several advantages. The retraining process allows the model to learn from new data, including recent real-world performance metrics and simulated performance metrics, thereby improving its ability to handle various scenarios and obstacles more effectively. For example, if the AGV 104 encountered a previously unseen obstacle during its operations, the retrained model would incorporate information about the unseen obstacle, enabling the AGV 104 to navigate similar obstacles more efficiently in the future. Siemens Aktiengesellschaft

[0212] 41

[0213] Deploying the retrained navigation model 108 in the operational environment enhances performance of the AGV 104 by leveraging the updated strategies and knowledge gained during the reinforcement learning procedure. The retrained model is better equipped to make accurate and timely navigational decisions, reducing the likelihood of collisions, stops, and deviations. For instance, if the retrained model has learned to recognize and avoid a new type of obstacle that frequently appears in the factory, the AGV 104 will navigate around it more smoothly, improving overall operational efficiency.

[0214] The automation module 112 further causes the processing unit 202 to display a resilience report based on the determined resilience parameter. The displayed resilience report is a comprehensive document that provides detailed insights into the performance and robustness of the navigation model 108. The resilience report is generated based on the determined resilience parameter, which quantifies how effectively the navigation model 108 handles various scenarios and obstacles.

[0215] The resilience report contains a detailed analysis of the plurality of real-world performance metrics and the plurality of simulated performance metrics. The resilience report includes counts of collision occurrences, stop times, and deviations from the intended path of the AGV 104. For example, the resilience report may present data showing that the AGV 104 experienced five collisions, ten stop times, and three deviations during a specific evaluation period. Additionally, the resilience report includes graphical representations such as charts and graphs to visualize the performance trends over time.

[0216] Moreover, the resilience report offers actionable insights and suggestions for optimizing the performance of the AGV 104. For example, the resilience report might recommend retraining the navigation model 108 with additional data or adjusting certain parameters to improve obstacle detection accuracy. The resilience report also includes comparisons between the performance of the AGV 104 in the real-world and simulated environments, providing a holistic view of its capabilities and areas for enhancement.

[0217] The present invention offers numerous advantages that significantly enhance the operational efficiency and safety of AGVs 104 in the factory environment 100. A primary benefit is the ability to conduct real-time testing in an actual operational environment of the AGV 104, rather than relying solely on simulated environments. Thus, the AGV 104 is exposed to realistic and dynamically changing conditions, allowing them to better adapt to unforeseen obstacles and Siemens Aktiengesellschaft

[0218] 42 changes in the factory layout. By introducing the plurality of synthetic obstacles 114 directly into the visual feed 120 captured by the sensor 106 of the AGV 104, the automation module 112 can test the AGV's 104 response to a wide range of scenarios without the need for physical alterations to the factory environment 100. The present invention not only saves time and resources but also minimizes the risk of damage to the AGV 104 and factory equipment during testing.

[0219] The present invention allows for continuous and comprehensive monitoring of the AGV's 104 performance by collecting detailed response data to each synthetic obstacle 114. This data-driven approach enables the identification of specific weaknesses in the AGV's 104 navigation and navigation models, facilitating targeted retraining and improvement. By utilizing the central monitoring station 102 to manage the retraining, the automation module 112 ensures seamless integration and coordination, providing a robust framework for evaluating and enhancing the resilience of the AGV 104. Additionally, the flexibility to conduct resilience testing before planned changes in the factory environment 100, such as altering the color of workers' uniforms, ensures that the AGV 104 is adequately prepared for new conditions, thereby reducing operational downtime and financial losses. Overall, the automation module 112 offers a proactive and efficient solution to improving the adaptability and robustness of AGVs 104, leading to safer and more reliable operations in dynamic factory environments.

[0220] The present invention incorporates several advanced technical features that enhance its efficacy and robustness. One key feature is the use of the sophisticated automation module 112 that orchestrates the entire testing process by dynamically inserting the plurality of synthetic obstacles 114 into the visual feed 120 and collecting detailed performance metrics. The automation module 112 leverages high-performance image processing units and machine learning algorithms to generate realistic synthetic obstacles 114 that accurately mimic real-world conditions. Additionally, the automation module 112 employs state-of- the-art reinforcement learning models to analyze the navigation process of the AGV 104 in response to the plurality of synthetic obstacles 114, providing precise feedback for further training. The visual feed 120 is rerouted through the automation module 112 using secure, low-latency communication protocols, ensuring seamless integration and real-time processing.

[0221] Introduction of the plurality of synthetic obstacles 114 into the factory environment 100 allows for comprehensive testing of response mechanisms of the AGV 104 in a controlled manner, without risking physical damage and harm to factory personnel or industrial devices in the factory environment 100. Thus, the Siemens Aktiengesellschaft

[0222] 43

[0223] AGV 104 is enabled to navigate around new and unexpected obstacles with greater precision, thereby reducing the likelihood of collisions that could result in injuries or damage to valuable assets. The ability of the automation module 112 to simulate a wide range of scenarios, including potential safety hazards, allows for thorough validation of the AGV's 104 navigation algorithms, ensuring that the vehicle can make safe and effective navigational choices in real-time. Enhanced monitoring tools, such as real-time error detection and decision latency tracking, further contribute to overall safety by providing continuous oversight of the AGV's 104 operations. By addressing these safety concerns, the invention not only improves operational efficiency but also fosters a safer work environment, promoting confidence in the deployment of AGVs 104 within dynamic and complex factory settings.

[0224] In one example, the plurality of synthetic obstacles 114 are inserted by the automation module 112 into the visual feed 120 in real-time, during the operation of the AGV 104 in the factory environment 100. The automation module 112 is thus enabled to perform dynamic testing of the ability of the AGV 104 to navigate around new and unforeseen obstacles without disrupting the factory environment 100. The visual feed 120 may be a live visual feed captured by the sensor 106 in real-time. By integrating synthetic obstacles 114 into the live visual feed 120, the AGV 104 is configured to perceive the inserted plurality of synthetic obstacles 114 as part of the natural surroundings of the AGV 104, providing a more accurate assessment of its resilience and navigation capabilities in real-time.

[0225] In one example, the central monitoring station 102 is configured to reroute the visual feed 120 from the AGV 104, process the visual feed 120 to insert the plurality of synthetic obstacles 114, and then send the modified visual feed 116 back to the AGV 104. Thus, the AGV 104 receives a continuous stream of modified visual data which has the plurality of synthetic obstacles 114. The CMS 102 also collects detailed logs of the response of the AGV 104, including actions taken, navigation times, and other critical performance metrics. Thus, the CMS 102 incorporates a real-time feedback loop which allows for immediate evaluation and adjustment of the navigation algorithm of the AGV 104, enhancing its ability to adapt to changing conditions on the factory floor.

[0226] In one example, the automation module 112 is enabled to conduct scenario-based resilience testing before implementing planned changes in the factory environment 100. For instance, if there is a plan to change the color of workers' uniforms, synthetic human figures with the new uniform color can be inserted Siemens Aktiengesellschaft

[0227] 44 into the visual feed 120 of the AGV 104. This proactive testing ensures that the AGV 104 can handle such changes without issues, thereby minimizing operational downtime and potential collisions. By simulating planned changes, the automation module 112 is enabled to gauge the preparedness of the AGV 104 as necessary, ensuring seamless adaptation to new conditions.

[0228] In one example, the CMS 102 comprises a comprehensive library of synthetic objects 114, which can be used to create specific test cases for resilience testing. The synthetic objects 114 are meticulously designed to mimic real-world obstacles, including their shapes, sizes, and movement patterns. Each test case is logged with the response of the AGV 104, including successful navigation or collision, which helps in identifying weaknesses in the algorithm of the AGV 104. This structured approach to resilience testing allows for targeted improvements and more effective retraining of the AGV 104, ensuring it can handle a wide range of scenarios.

[0229] In one example, the automation module 112 is configured to execute batch processing of the set of visual frames 120A to give the AGV 104 a perception of encountering a real 3D object. By processing multiple frames, the automation module 112 creates a dynamic and realistic representation of the synthetic obstacle 114, ensuring that the AGV 104 perceives them as genuine parts of the factory environment 100.

[0230] In one example, the resilience report comprises logs of interactions of the AGV 104 with the plurality of synthetic obstacles 114. Each interaction with the plurality of synthetic obstacles 114 is considered as a test case of a plurality of test cases. Each test case is marked as failed if the AGV 104 collides with the synthetic obstacle 114 or temporarily halts the operation of the AGV 104. Thus, the resilience report comprises a robust dataset that can be used to retrain the navigation model 108 of the AGV 104. By analyzing the resilience report, a user is enlightened about specific areas where the resilience of the AGV 104 needs improvement and thereby enables the user to implement targeted training to address one or more weaknesses of the AGV 104.

[0231] FIG 2 is a block diagram of the central monitoring station 102, such as those shown in FIG 1, in which an embodiment of the present invention can be implemented. In FIG 2, the central monitoring station 102 includes a processing unit 202, an accessible memory 204, a storage unit 206, a communication interface 208, an input-output unit 210, a network interface 212 and a bus 214. Siemens Aktiengesellschaft

[0232] 45

[0233] The processing unit 202, as used herein, means any type of computational circuit, such as, but not limited to, a microprocessor unit, microcontroller, complex instruction set computing microprocessor unit, reduced instruction set computing microprocessor unit, very long instruction word microprocessor unit, explicitly parallel instruction computing microprocessor unit, graphics processing unit, digital signal processing unit, or any other type of processing circuit. The processing unit 202 may also include embedded controllers, such as generic or programmable logic devices or arrays, application specific integrated circuits, single-chip computers, and the like.

[0234] The memory 204 may be non-transitory volatile memory and non-volatile memory. The memory 204 may be coupled for communication with the processing unit 202, such as being a computer-readable storage medium. The processing unit 202 may execute machine-readable instructions and / or source code stored in the memory 204. A variety of machine-readable instructions may be stored in and accessed from the memory 204. The memory 204 may include any suitable elements for storing data and machine-readable instructions, such as read only memory, random access memory, erasable programmable read only memory, electrically erasable programmable read only memory, a hard drive, a removable media drive for handling compact disks, digital video disks, diskettes, magnetic tape cartridges, memory cards, and the like. In the present embodiment, the memory 204 includes a control unit 216. The control unit 216 includes the automation module 112 stored in the form of machine-readable instructions on any of the above-mentioned storage media and may be in communication with and executed by the processor(s) 202.

[0235] The storage unit 206 may be a non-transitory storage medium configured for storing a database which comprises server version of the plurality of periodically generated plan view received from the plurality of vehicles.

[0236] The communication interface 208 is configured for establishing communication sessions between the central monitoring station 102 and the AGV 104 and one or more external data sources.

[0237] The input-output unit 210 may include input devices a keypad, touch-sensitive display, camera (such as a camera receiving gesture-based inputs), etc. capable of receiving one or more input signals, such as user commands to navigate the AGV 104. The bus 214 acts as inter-connect between the processing unit 202, the memory 204, and the input-output unit 210. Siemens Aktiengesellschaft

[0238] 46

[0239] The network interface 212 is configured to handle network connectivity, bandwidth and network traffic with a network 218, The network 218 is used to communicate with the AGV 104.

[0240] Those of ordinary skilled in the art will appreciate that the hardware depicted in FIG 2 may vary for particular im-plementations. For example, other peripheral devices such as an optical disk drive and the like, Local Area Network (LAN), Wide Area Network (WAN), Wireless (e.g., Wi-Fi) adapter, graphics adapter, disk controller, input / output (I / O) adapter also may be used in addition or in place of the hardware depicted. The depicted example is provided for the purpose of explanation only and is not meant to imply architectural limitations with respect to the pre-sent disclosure.

[0241] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all data processing systems suitable for use with the present disclosure is not being depicted or described herein. Instead, only so much of the central monitoring station 102 as is unique to the present disclosure or necessary for an understanding of the present disclosure is depicted and described. The remainder of the construction and operation of the central monitoring station 102 may conform to any of the various current implementation and practices known in the art.

[0242] FIG 3 is a block diagram of an automation module 112, such as those shown in FIG 2, in which an embodiment of the present invention can be implemented. In FIG 3, the automation module 112 comprises a request handler module 302, an elevation profiling unit 304, an analysis module 306, a prompt module 308, a obstacle database 310, a validation module 312 and a navigation module 314. FIG. 3 is explained in conjunction with FIG. 1 and FIG. 2.

[0243] The request handler module 302 is configured for receiving user instructions to control the vehicle 104 (as shown in FIG. 1A).

[0244] The elevation profiling unit 304 is configured to determine an elevation profile of a specific area captured in the field-of-view 114 by the at least one sensor 106. The elevation profiling unit 304 is further configured to dynamically transmit the determined elevation profile to the AGV 104.

[0245] The analysis module 306 is configured for analyzing the visual feed to insert the plurality of synthetic obstacles 114 into the visual feed 120 (as shown in FIG. IB) Siemens Aktiengesellschaft

[0246] 47

[0247] The prompt module 308 is configured to generate a plurality of prompts for the LLM.

[0248] The obstacle database 310 is configured for storing a comprehensive library of synthetic obstacles.

[0249] The validation module 312 is configured to determine the resilience parameter of the navigation model 108.

[0250] The navigation module 314 is configured for analyzing the navigation decision generated by the navigation model 108.

[0251] FIG 4 is a process flowchart 400 illustrating an exemplary method of retraining the AGV 104, according to an embodiment of the present invention.

[0252] At step 402, the visual feed 120 is received from the at least one sensor 106. The at least one sensor 106 is configured to capture visual data of the factory environment 100.

[0253] At step 404, the plurality of synthetic obstacles 114 is inserted into the received visual feed 120 to generate the modified visual feed 116. The insertion of synthetic obstacles 114 involves embedding one or more digital representations of real-world obstacles into the visual feed 120 to simulate real-world conditions.

[0254] At step 406, the modified visual feed 116 is analyzed by applying the reinforcement learning-based navigation model 108. The navigation model 108 generates a navigational decision based on the detection of the plurality of synthetic obstacles 114.

[0255] At step 408, a reinforcement learning procedure is initiated to retrain the navigation model 108 based on an analysis of the generated navigational decision.

[0256] The present invention can take a form of a computer program product comprising program modules accessible from computer-usable or computer-readable medium storing pro-gram code for use by or in connection with one or more computers, processors, or instruction execution system. For the purpose of this description, a computer-usable or computer-readable medium can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The Siemens Aktiengesellschaft

[0257] 48 medium can be electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation mediums in and of themselves as signal carriers are not included in the definition of physical computer-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, random access memory (RAM), a read only memory (ROM), a rigid magnetic disk and optical disk such as compact disk read-only memory (CD-ROM), compact disk read / write, and DVD. Both processors and program code for implementing each aspect of the technology can be centralized or distributed (or a combination thereof) as known to those skilled in the art.

[0258] While the present invention has been described in detail with reference to certain embodiments, it should be appreciated that the present invention is not limited to those embodiments. In view of the present disclosure, many modifications and variations would be present themselves, to those skilled in the art without departing from the scope of the various embodiments of the present invention, as described herein. The scope of the present invention is, therefore, indicated by the following claims rather than by the foregoing description. All changes, modifications, and variations coming within the meaning and range of equivalency of the claims are to be considered within their scope. All advantageous embodiments claimed in meth-od claims may also be apply to system / apparatus claims.

[0259] Siemens Aktiengesellschaft

[0260] 49

[0261] List of reference numerals ■

[0262] 100 Factory environment

[0263] 102 Central monitoring station (CMS)

[0264] 104 Automated Guided Vehicle (AGV)

[0265] 106 Sensor (e.g., LIDAR sensor, depth sensing camera)

[0266] 108 Navigation model

[0267] 110 Real-world object

[0268] 112 Automation module

[0269] 114 Synthetic obstacle

[0270] 114A First configuration and position of synthetic obstacle

[0271] 114B Second configuration and position of synthetic obstacle

[0272] 116 Modified visual feed

[0273] 118 Field of view (of the AGV)

[0274] 120 Visual feed

[0275] 120A Set of visual frames

[0276] 122 Modified set of visual frames

[0277] 202 Processing unit

[0278] 204 Memory

[0279] 206 Storage unit

[0280] 208 Communication interface

[0281] 210 Inp ut ■ outp ut unit

[0282] 212 N etwork interface

[0283] 214 Bus

[0284] 216 Control unit

[0285] 218 Network

[0286] 302 Request handler module

[0287] 304 Elevation profiling unit

[0288] 306 Analysis module

[0289] 308 Prompt module

[0290] 310 Obstacle database

[0291] 312 Validation module

[0292] 314 N avigation m o dule

[0293] 400 Process flowchart

Claims

Siemens Aktiengesellschaft50 CLAIMS1. A method for retraining an Automated Guided Vehicle (AGV) (104) in a factory environment (100), comprising: obtaining a visual feed (120) captured by at least one sensor (106), wherein the at least one sensor (106) is configured to capture visual data from the factory environment (100); inserting a plurality of synthetic obstacles (114) into the received visual feed (120) to generate a modified visual feed (116) comprising the plurality of synthetic obstacles (114), wherein insertion of the plurality of synthetic obstacles (114) comprises embedding one or more digital representations of real-world obstacles into the visual feed (120) to simulate a real-world condition; analyzing the modified visual feed (116) by applying a reinforcement learning-based navigation model (108) on the modified visual feed (116), wherein the navigation model (108) is configured to generate a navigational decision based on detection of the plurality of synthetic obstacles (114); and initiating a reinforcement learning procedure to retrain the navigation model (108) based on an analysis of the generated navigational decision, wherein the retrained navigation model (108) is deployable in the AGV (104).

2. The method of claim 1, wherein the navigation model (108) is retrained to configure the navigation model (108) to generate a response to the simulated real-world condition.

3. The method of claim 1, wherein the at least one sensor (106) is at least one of a LIDAR sensor (106) and a depth sensing camera (106).

4. The method according to any of claims 1 to 3, wherein the at least one sensor (106) is mounted on the AGV (104) and the at least one sensor (106) is configured to capture a field-of-view (118) from the AGV (104).

5. The method in accordance with any of claims 1 to 4, further comprising: executing the generated navigational decision by controlling the movement of the AGV (104), based on the navigational decision; and monitoring a response of the AGV (104) to the navigational decision, wherein the response is monitored based on a plurality of real-world performance metrics comprising at least one of a count of collision occurrences, a count of stop times, and a count of deviations from an intended path of the AGV (104).

6. The method according to any of claims 1 to 5, further comprising:Siemens Aktiengesellschaft51 generating a simulation instance of the factory environment (100), wherein the simulation instance includes the plurality of synthetic obstacles (114), and a digital twin of the AGV (104); simulating a movement of the AGV (104) in response to the generated navigational decision, wherein the movement is simulated within the simulation instance; and monitoring the simulated movement of the AGV (104) in the simulation instance, wherein the simulated movement is monitored based on a plurality of simulated performance metrics of the AGV (104).

7. The method in accordance with any of the claims 5 and 6, wherein analyzing the generated navigational decision comprises: determining a resilience parameter of the navigation model (108) based on an analysis of the plurality of real-world performance metrics and the plurality of simulated performance metrics; and determining whether the resilience parameter is lesser than a threshold, wherein the reinforcement learning procedure is initiated based on a determination that the resilience parameter is lesser than the threshold.

8. The method of claim 1, further comprising utilizing a Large Language Model (LLM) (not numbered) to generate each of the plurality of synthetic obstacles (114) based on one or more user-defined criteria, wherein each of the plurality of synthetic obstacles (114) is associated with at least one user-defined criteria of the one or more user- defined criteria.

9. The method in accordance with any of claims 1 to 8, wherein the plurality of synthetic obstacles (114) comprises three-dimensional models which are configured to move, and wherein a shape and a position of at least one of the plurality of synthetic obstacles (114) changes with time.

10. The method in accordance with any of claims 6 to 9, wherein the inserting each synthetic obstacle of the plurality of synthetic obstacles (114) comprises: determining, from the received visual feed (120), a set of visual frames (120A) for the synthetic obstacle, wherein the set of visual frames (120A) are determined based on the one or more user-defined criteria associated with the synthetic obstacle; generating a series of configurations and a series of positions of the synthetic obstacle by application of the LLM on the one or more user- defined criteria, wherein each of the series of configurations and the series of positions isSiemens Aktiengesellschaft52 indicative of a configuration or position of the synthetic obstacle, at a time instance corresponding to each frame of the set of visual frames (120A); and embedding the synthetic obstacle into an appropriate visual frame of the set of visual frames (120A), based on the generated series of configurations and the series of positions of the synthetic obstacle.

11. The method of claim 9, wherein the one or more user-defined criteria associated with the synthetic obstacle comprises at least one of: a specific type of obstacle to be simulated; a location within the factory environment (100) where the synthetic obstacle (114) is to appear; a timing and duration for which the synthetic obstacle is to be present in the visual feed (120); a movement pattern of the synthetic obstacle; information associated with a type of factory layout changes, one or more historical collision data, one or more training objectives for the AGV (104), and an interaction behavior of the synthetic obstacle, including changes in shape, size, or color over time.

12. The method in accordance with any of claims 1 to 11, wherein applying the reinforcement learning-based navigation model (108) on the modified visual feed (116) comprises: detecting the plurality of synthetic obstacles (114) by application of an object detection algorithm on the modified visual feed (116); and analyzing the detected plurality of synthetic obstacles (114) using the reinforcement learning-based navigation model (108) to generate the navigation decision.

13. The method of claim 1, further comprising deploying the retrained navigation model (108) in an operational environment of the AGV (104) after completion of the reinforcement learning procedure.

14. A central monitoring station (102) for retraining an Automated Guided Vehicle (AGV) (104) in a factory environment (100), comprising: one or more processor(s) (202); and a memory (204) coupled to the one or more processor(s) (202), wherein the memory (204) comprises an automation module (112) stored in the form of machine-readable instructions executable by the one or more processor(s) (202), wherein the automation module (112) is capable of performing a method according to any of the claims 1'13.Siemens Aktiengesellschaft5315. An Automated Guided Vehicle (104) deployable in a factory environment (100), wherein the Automated Guided Vehicle (104) comprises an automation module (112), the automation module (112) being configured to perform a method according to any of the claims 1 to 13.

Images