Anomaly detection for component using pose-dependent machine learning models

A mobile robot with pose-dependent machine learning models for anomaly detection in industrial facilities addresses the inefficiencies of fixed sensors by capturing images in multiple poses, enhancing accuracy and reducing maintenance, while minimizing sensor requirements and wiring.

JP2024157547AActive Publication Date: 2024-11-07YOKOGAWA ELECTRIC CORP
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
JP2024070659
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-04-25
Filing Date
2024-04-24
Publication Date
2024-11-07
Estimated Expiration
2044-04-24

AI Technical Summary

Technical Problem

Industrial facilities require numerous fixed sensors for anomaly detection, which are costly, require extensive wiring, and are prone to failure, leading to inefficiencies and maintenance challenges.

Method used

Implement a mobile robot equipped with a camera to capture images of components in various poses, using pose-dependent machine learning models to detect anomalies, reducing the need for fixed sensors and improving detection accuracy by selecting appropriate models based on the image's capture pose.

Benefits of technology

This approach reduces the number of sensors needed, minimizes wiring, enhances anomaly detection accuracy, and decreases the occurrence of false positives and negatives, thereby preventing unnecessary remedial actions and ensuring safer operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024157547000001_ABST
    Figure 2024157547000001_ABST
Patent Text Reader

Abstract

To use pose-dependent machine learning (ML) models in determining whether an anomaly is present within an environment.SOLUTION: A first ML model is used to generate first output based on processing a first image that is captured by a camera of a mobile robot when the camera is at a first pose. The first ML model is used based on the first image being captured at the first pose and the first ML model being trained for the first pose. A second ML model is used to generate second output based on processing a second image that is captured by the camera when the camera is at a distinct second pose. The second ML model is used based on the second image being captured at the second pose and the second ML model being trained for the second pose. The first and second outputs can be used in determining whether an anomaly is present.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to component anomaly detection using pose-dependent machine learning models. [Background technology]

[0002] Complex industrial facilities, such as petrochemical refineries, chemical plants, and the like, may contain numerous components utilized in the processing of liquids, gases, and / or other substances involved in the industrial process of the industrial facility. It is important to ensure that the components involved in the industrial process are operating as intended and / or that the substances involved in the industrial process are in the intended state.

[0003] In industrial facilities, various sensors are utilized to monitor such components and / or materials for anomalies, including temperature sensors utilized to monitor the temperature anomalies of the components and / or materials, light sensors utilized to monitor the composition of materials (e.g., based on exciting the materials with a light source), and the like.

[0004] Industrial facilities may utilize a variety of sensors to monitor various anomalies, with such sensors typically each located at a corresponding fixed location within the industrial facility. Thus, a very large number of sensors may be required to efficiently monitor the industrial facility. Furthermore, providing power and / or communications to each of the sensors may require extensive wiring throughout the industrial facility. Furthermore, the large number of sensors are each prone to failure or malfunction and must be repaired or replaced when they do occur, which may be difficult in large industrial facilities that include a large number of fixed sensors. Summary of the Invention [Means for solving the problem]

[0005] Implementations of the present disclosure are directed to utilizing a mobile robot to capture images of components and / or substances in an environment, such as an industrial facility or other environment, at a given pose via a camera of the mobile robot. The implementations are further directed to processing the captured images to generate ML outputs using a machine learning (ML) model corresponding to the given pose. The machine learning model can be a neural network model, such as a convolutional neural network (CNN) model including one or more convolution layers. The images are processed using the ML model in response to the images captured at the given pose and in response to the ML model corresponding to the given pose at which the images were captured. The implementations are further directed to determining whether there is an anomaly in the components and / or substances captured by the images based on the generated ML output, and if there is an anomaly, causing one or more repair actions to be performed. Executing a repair action can include, for example, rendering an alarm, stopping a process, and / or performing other corrective actions.

[0006] As referenced above, the image is processed using the ML model in response to the image captured at a given pose and also in response to the ML model corresponding to the given pose in which the image was captured. For example, metadata can indicate that the image was captured at a given pose, and the ML model can select from among multiple candidate ML models to process the image based on the metadata indicating that the image was captured at the given pose and the ML model that also corresponds to the given pose. The metadata can be included as data incorporated as part of the image or as data separate from but associated with the image. The metadata can directly or indirectly reflect a given position and / or a given orientation in which the image was captured. The given position and / or the given orientation reflected by the metadata can be relative to a world frame, such as a relative world frame. In other words, the given pose can be relative to some absolute or relative world frame, rather than only relative to the mobile robot. For example, if the camera that captures the images is non-adjustably coupled to the robot (i.e., it is always at a fixed pose relative to the robot), then a given pose (due to the non-adjustable coupling) will change as the robot moves, even though the camera remains at a fixed pose relative to the robot.

[0007] The ML model may correspond to a given pose based on having been trained using ground truth images captured at or near the given pose (e.g., within a threshold of the given pose). For example, the ML model may be trained at least in part based on negative training instances, each of which includes a training instance input of a corresponding image captured at or near the given pose when the anomaly was not present and a labeled training instance output indicating that the anomaly was not present. Also, for example, the ML model may additionally or alternatively be calibrated at least in part based on positive calibration instances, each of which includes a calibration instance input of a corresponding image captured at or near the given pose when the anomaly was present and a labeled calibration instance output indicating that the anomaly was present. The ML model may be indexed or otherwise associated with data indicating that it corresponds to the given pose and / or a pose within a threshold of the given pose. For example, the ML model may be indexed with data indicative of the given pose based on having been trained for the given pose, and may be used to process an image based on being indexed with such data and also based on the image being associated with metadata indicative of the given pose.

[0008] In particular, as referenced above, multiple candidate ML models can be provided, each of the candidate ML models can correspond to a different corresponding pose. For example, a first ML model can correspond to a first pose based on being trained based on images captured at or near the first pose, a second ML model can correspond to a second pose based on being trained based on images captured at or near the second pose, etc. For example, the first ML model can be trained only based on images captured at or near the first pose, and the second ML model can be trained only based on images captured at or near the second pose, etc. Also, for example, the first ML model can be a given ML model fine-tuned based on images captured at or near the first pose, the second ML model can be a given ML model fine-tuned based on images captured at or near the second pose, etc. For example, the first ML model can be fine-tuned based on training instances each including corresponding images captured at or near the first pose and corresponding supervised labeled outputs indicating whether an anomaly is present in the corresponding images. For example, the corresponding supervised labeled output can be based on input from a human operator after reviewing the corresponding image, where the input indicates whether an anomaly is present in the image.

[0009] Thus, which of a plurality of candidate ML models is utilized in processing a given image captured by a mobile robot may depend on a given pose in which the image was captured. For example, if a given image was captured at or near a first pose, a first ML model corresponding to the first pose may be utilized. On the other hand, if the given image was instead captured at or near a second pose, a second ML model corresponding to the second pose may be utilized. Thus, the ML model utilized in processing an image to generate an output used to determine whether an anomaly exists in a component and / or material captured by the image may be selected depending on the pose of the image and may be selected based on the ML model having been trained on ground truth images also captured at or near the pose. Thus, the ML model utilized is applied to the pose of the image, resulting in a more accurate output when the image is processed using the ML model. The more accurate output may mitigate the occurrence of false positive and / or false negative anomaly detections. Mitigating the occurrence of false positives may prevent utilization of network and / or computing device resources in erroneously performing remedial actions, such as, for example, rendering false positive alerts. Reducing the occurrence of false negatives can prevent the occurrence of unsafe conditions in an industrial facility and / or damage to components and / or materials in the industrial facility.

[0010] In various implementations of the present disclosure, multiple ML models and images, each corresponding to a different respective pose, are utilized in determining whether an anomaly exists in a given component and / or material. As an example, a first ML model can be trained using a first ground truth image capturing a given liquid tank from at or near a first pose. The first ML model can be used to process the first image capturing the given liquid tank from the first pose to generate a first output indicating whether an anomaly exists associated with the given liquid tank (e.g., an anomaly related to the given liquid tank itself and / or the liquid contained therein). Continuing the example, a second ML model can be trained using a second ground truth image capturing the same given liquid tank from at or near a separate second pose. The second ML model can be used to process a second image capturing the same given liquid tank from a separate second pose to generate a second output indicating whether an anomaly exists associated with the given liquid tank. Additionally, both the first output from the first ML model and the second output from the second ML model can be considered in determining whether an anomaly exists for the liquid tank. For example, an anomaly can be detected if (a) the first model output meets a lower anomaly threshold and the second model output also meets a lower anomaly threshold, (b) either the first model output or the second model output meets an upper anomaly threshold, (c) and / or an average (or other combination) of the first and second model outputs meets a threshold (e.g., a lower, upper, or other threshold).

[0011] By considering both the first output and the second output in determining whether an anomaly exists with respect to the liquid tank, the occurrence of false positives can be mitigated. For example, it may be the case that the first image includes significant glare (e.g., due to current sunlight conditions), such that the first output (alone) may indicate an anomaly despite the absence of a true anomaly with respect to the liquid tank. However, the second image may include less glare (or no glare), such that the second output (alone) may not indicate an anomaly. Considering both the first output and the second output in such a situation may result in a correct no anomaly determination, whereas considering only the first output alone may result in an incorrect anomaly determination. As another example, if each of the first and second outputs meets a lower anomaly threshold and / or if the combination of the first and second outputs meets a lower anomaly threshold, an anomaly may be detected and the lower anomaly threshold may be set more aggressively as a result of considering both the first and second outputs. Such a more aggressive lower anomaly threshold may mitigate the occurrence of false negatives. In other words, if only a single output is considered, the lower anomaly threshold needs to be less aggressive (i.e., a higher lower bound) to prevent an excess of false positives.

[0012] Many of the above examples describe the utilization of a first ML model and a second ML model in determining whether an anomaly exists in a given component and / or material. However, it should be noted that in various implementations, more than two ML models may be utilized for a given component and / or material.

[0013] The mobile robot utilized in capturing images from various poses may be a four-legged robot, a wheeled robot, an unmanned aerial vehicle, a powered guided robot, or any other robot that moves itself in an environment. The images are captured via the vision component of the mobile robot. Each image is captured when the vision component is in a corresponding pose (i.e., a given position and orientation). The pose of the vision configuration when the image is captured at a given time is a function of the pose of the robot at the given time. If the vision component is in a fixed pose relative to the robot, the pose of the vision component is purely a function of the pose of the robot. If the pose of the vision component is independently adjustable relative to the robot, the pose of the vision component is a function of the pose of the robot and the pose of the vision component relative to the robot. The images utilized herein may include, for example, an RGB image including red, green, and blue channels and captured by a monographic RGB camera, an RGB-D image including a depth channel in addition to red, green, and blue channels and captured by a stereographic camera, or a thermal image including one or more thermal channels and captured by a thermal camera.

[0014] By utilizing a mobile robot that includes a vision component and moves around an environment to capture images of components and / or objects of interest for anomaly monitoring / detection, the number of sensors monitoring the environment can be reduced and / or the extensive wiring required for such sensors can be avoided. Furthermore, the accuracy of anomaly detection based on processing each image can be improved by processing each image captured at a given pose of one or more captured images of the components and / or material using a pose-dependent ML model trained to process images corresponding to the given pose for anomaly detection. This allows anomaly detection to be more robust and / or more accurate.

[0015] It should be understood that all combinations of the foregoing concepts, and additional concepts described in more detail herein, are contemplated as being part of the subject matter disclosed herein, for example, all combinations of claimed subject matter appearing at the end of this disclosure are contemplated as being part of the subject matter disclosed herein. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 illustrates a schematic diagram of an environment in which selected aspects of the present disclosure can be implemented, according to various implementations. [Diagram 2] FIG. 1 illustrates a schematic diagram of an example of how the techniques described herein may be implemented, according to various implementations. [Diagram 3] FIG. 1 illustrates an exemplary method for carrying out selected aspects of the present disclosure. [Figure 4] FIG. 1 illustrates generally an exemplary computer architecture in which selected aspects of the present disclosure may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Implementations described herein relate to monitoring and selectively detecting anomalies for components (e.g., liquid tanks, piping, wiring) in an environment (e.g., an industrial automation facility) using pose-dependent machine learning (ML) models. In various implementations, images of the components are captured by a vision component (e.g., a camera) carried by or integrated with a mobile robot movable within the environment. Images are captured by the vision component in various poses (i.e., various positions and orientations), and ML models corresponding to the poses are selected to process the images. For example, a first image captured by the vision component in a first pose can be processed using a first ML model trained for anomaly detection based on images captured at or near the first pose, and a second image captured by the vision component in a second pose can be processed using a second ML model trained for anomaly detection based on images captured at or near the second pose. Based on processing the images with the selected ML models (e.g., processing the first image using the first ML model and processing the second image using the second ML model), one or more anomalies (if any) associated with the component may be identified / detected, and one or more remedial actions may be performed to address the detected one or more anomalies.

[0018] Referring now to FIG. 1, an example environment 100 in which various aspects of the disclosure may be implemented is generally illustrated. In some implementations, the example environment 100 may be or include an industrial facility. The industrial facility may take a variety of forms and may be designed to implement any number of at least partially automated processes. The industrial facility may take the form of a chemical processing plant, an oil or natural gas refinery, a catalyst factory, a manufacturing facility, an offshore oil platform, etc.

[0019] The exemplary environment 100 can include one or more client devices (e.g., local client devices 103-A and 103-B) operably coupled to a process automation network 106 within an industrial facility. The client devices 103-A and 103-B can be implemented as computers (e.g., laptops, desktops, notebooks), tablets, robots, smart appliances (e.g., smartphones), messaging devices, wearable devices (e.g., watches), or any other applicable devices. The process automation network 106 can be implemented using various wired and / or wireless communication technologies, including, but not limited to, cellular networks such as the Institute of Electrical and Electronics Engineers (IEEE) Standard 802.3 (Ethernet), IEEE 802.11 (Wi-Fi), 3GPP® Long Term Evolution ("LTE") or other wireless protocols designated as 3G, 4G, 5G, and beyond, and / or other types of communication networks of various types of topologies (e.g., mesh).

[0020] The exemplary environment 100 may further include a mobile robot 101 having or carrying a vision component 1011. The mobile robot 101 may be a four-legged robot (e.g., a robot dog), a wheeled robot, an unmanned aerial vehicle, a robot that moves along elevated and / or non-elevated tracks in the environment, or any other applicable robot that can move in an industrial facility. The vision component 1011 may be a monographic camera, a stereographic camera, a thermal camera, or any other applicable vision component to capture one or more images of one or more specific components of the industrial facility (e.g., a liquid tank or tube 102 that stores or transports a liquid substance). The vision component 1011 may be removably coupled to the mobile robot 101 or may be integrated as a non-removable adhesive part of the mobile robot 101. In some implementations, the vision component 1011 may be independently positionally and / or orientated relative to the robot frame of the mobile robot 101, for example by rotation or other movement via an actuator that independently controls the vision component 1011. In addition to the vision component 1011, the mobile robot 101 may include one or more additional vision components to navigate within an industrial facility, sense static or dynamic objects, and / or capture images.

[0021] The exemplary environment 100 may further include a server device 105. The server device 105 may include a machine learning (ML) engine 1051 and an anomaly detection engine 1052. The server device 105 may further include or otherwise access one or more pose-dependent machine learning (ML) models 1053. The server device 105 may communicate with one or more local client devices (e.g., 122-A and 122-B) and / or with one or more remote client devices (not shown). The local client devices 122-A or 122-B may connect to the server device 105 via one or more local area networks (e.g., the process automation network 106), and the remote client devices may connect to the server device 105 via one or more wide area networks (e.g., the Internet). The local and remote client devices may be operable by personnel, such as a system integrator, to configure and / or interact with various aspects of the exemplary environment 100.

[0022] In some implementations, the server device 105, in addition to the ML engine 1051 and the anomaly detection engine 1052, may include a database (not shown) that stores information used by the ML engine 1051 and / or the anomaly detection engine 1052 to implement selected aspects of the present disclosure. Various aspects of the server device 105, such as the ML engine 1051 and / or the anomaly detection engine 1052, may be implemented using any combination of hardware and software. In some implementations, the ML engine 1051, the anomaly detection engine 1052, or one or more pose-dependent ML models 1053 may be implemented across multiple computer systems as part of what is often referred to as a "cloud infrastructure" or simply a "cloud." However, this is not required, and in FIG. 1, for example, the ML engine 1051 is implemented within an industrial facility, e.g., within a single building, or across a single canvas of a building or other industrial facility. In such implementations, the ML engine 1051 may be implemented on one or more local computing systems, such as one or more local server computers.

[0023] In some implementations, the mobile robot 101 can move through an industrial facility and arrive at a designated spot or waypoint. The vision component 1011 of the mobile robot 101 can be configured to a given pose to capture an image of the liquid tube 102 at the given pose (position and / or orientation). The vision component 1011 can be configured to a given pose as a result of the mobile robot 101 being in a corresponding pose and / or as a result of the vision component 101 being independently adjusted (if the vision component 1011 is independently adjustable relative to the robot frame of the mobile robot 101). The image captured by the vision component 1011 can capture and include pixels or other data corresponding to a region of interest for the liquid tube 102 (e.g., a joint J as shown in FIG. 1, or a region where a wet spot or crack was previously identified that indicates a potential leak, etc.). Based on a given pose at which an image is captured by the vision component 1011 and / or based on the type of image captured, the ML engine 1051 may select a pose-dependent ML model from the ML models 1053 that corresponds to the given pose for processing the captured image.

[0024] The captured image can be processed by the ML engine 1051 as an input to a selected pose-dependent ML model corresponding to the given pose to generate an ML output of the selected pose-dependent ML model. The ML output indicates whether the region of interest of the liquid tube 102 includes one or more anomalies. In some implementations, the pose-dependent ML model corresponding to the given pose can be trained using training instances, each of which includes (1) an image captured by the visual component (or additional visual component) at a given pose (i.e., at a given location and a given orientation) and (2) a ground truth label indicating whether an anomaly is present in the image (e.g., no anomaly is present or one or more specific types of anomalies are present). For example, the pose-dependent ML model corresponding to the given pose can be trained using multiple training instances, each of which includes an image captured at a given pose and a ground truth label indicating whether an anomaly is present in the image captured at the given pose. In this case, the aforementioned ML output can indicate whether the region of interest of the liquid tube 102 includes any anomaly. Training the pose-dependent ML model based on such training instances may include training the pose-dependent ML model only on such training instances, or fine-tuning the pose-dependent ML model by training on such training instances after initial training on non-pose-dependent training instances.

[0025] In some implementations, each ground truth label can be a single corresponding value, such as either an “anomaly present” label (e.g., “1”) or an “anomaly free” label (e.g., “0”). In those implementations, the output generated by processing the image using the ML model similarly generates a single corresponding value (e.g., a value from “0” to “1”), which single value can indicate a corresponding likelihood that an anomaly is present. In some other implementations, each ground truth label can include multiple values, such as an “anomaly present” label or an “anomaly free” label for each of a number of regions. For example, the images of the training instances can be 512 pixels by 512 pixels (or other size), and each ground truth label can include nine (or other amount) values, each of which can indicate a corresponding likelihood that an anomaly is present in a portion of the image. For example, the image can be divided into a 3×3 grid, and each of the values ​​can indicate a corresponding likelihood that an anomaly is present in one of the nine cells of the 3×3 grid.

[0026] In some implementations, a pose-dependent ML model corresponding to a given pose can be trained using multiple training instances, each including an image captured at the given pose and a ground truth label indicating a particular type of anomaly present in the image captured at the given pose. In some of these implementations, such an ML output can indicate whether a region of interest in the liquid tube 102 includes a particular type of anomaly.

[0027] In various implementations, based on the ML output, the anomaly detection engine 1052 can determine whether an anomaly exists within a region (e.g., a region of interest) of the liquid tube 102 captured in the image processed by the selected pose-dependent ML model. For example, based on the ML output indicating that an anomaly exists within the captured image, the anomaly detection engine 1052 can determine that the liquid tube 102 includes an anomaly, and in response, can cause a remedial action to be performed. For example, the anomaly detection engine 1052 can generate, execute, or otherwise communicate with other engines / components (not shown) to perform one or more additional remedial actions (e.g., display an alarm message 107 via the client devices 103-A and / or 103-B and / or pause an industrial process involving the liquid tube 102).

[0028] Although only a single mobile robot 101 is shown in FIG. 1, it is understood that multiple mobile robots can be deployed in an industrial environment and utilized in the implementations disclosed herein. For example, each of the multiple mobile robots can include a corresponding vision component used to capture images, and images from the multiple mobile robots are transmitted (along with pose data) to the server device 105 for processing by the ML engine 1051 and the anomaly detection engine 1052. Also, while the ML engine 1051, the ML model 1053, and the anomaly detection engine 1052 are shown in FIG. 1 as being implemented separately from the mobile robot 101, in some implementations, all or aspects can be implemented by the mobile robot 101. For example, the mobile robot 101 can include the ML engine 1051, the anomaly detection engine 1052, and at least a subset of the ML model 1053. For example, the mobile robot 101 may include a subset of the ML models 1053 at a given time based on the subset corresponding to a pose at which the mobile robot 101 captures an image in a mission to be performed by the mobile robot 101 at the given time.

[0029] FIG. 2 illustrates an example of how the techniques described herein can be implemented according to various implementations. As illustrated in FIG. 2, the vision component 1011 of the mobile robot 101 can capture a first image 201 corresponding to a first region (e.g., a first side) of a liquid tube 102 in an industrial facility (sometimes referred to as an "industrial automation facility"), the first image 201 being captured by the vision component 1011 when the vision component 1011 is in a first pose in the environment. The mobile robot can be controlled (e.g., autonomously, optionally based on a previously recorded mission) to move from a first side to a second side of a particular component to enable the vision component 1011 to capture a second image 203, the second image 203 being captured by the vision component 1011 when the vision component 1011 is in a second pose in the environment that is different from the first pose. The second image 203 can correspond to a second region (e.g., the right side) of the liquid tube 102. In some implementations, the second region can optionally supplement the first region such that a portion of the liquid tube 102 is captured in its entirety for anomaly detection. In some implementations, the first and second regions can include the same region of the liquid tube 102, but captured from different perspectives (i.e., as a result of being captured from different poses of the vision component 1011).

[0030] In response to the first image 201 being captured in a first pose, the first ML model 211 may be selected from a plurality of trained ML models. The plurality of trained ML models may be trained based on images captured in a corresponding range of poses (e.g., the first pose, the second pose, etc.) to detect anomalies (cracks, leaks, thermal anomalies, corrosion, impurities, etc.) in the industrial facility. The first ML model 211 may be selected based on being trained to process images captured in the first pose and / or based on a type of anomaly to be detected from the first image 201. In response to the second image 203 being captured in a second pose, the second ML model 213 may be selected from the plurality of trained ML models based on being trained to process images captured in the second pose and / or based on a type of anomaly to be detected from the second image 203.

[0031] In some implementations, the first image 201 can be determined to have been captured in a first pose based on metadata associated with the first image and received from the mobile robot indicating that the first image was captured in a first pose. In those implementations, selecting a first ML model for processing the first image is responsive to determining that the first image was captured in a first pose based on metadata associated with the first image. In some implementations, the second image can be determined to have been captured in a second pose based on metadata associated with a second image and received from the mobile robot indicating that the second image was captured in a second pose. In those implementations, selecting a second ML model for processing the second image is responsive to determining that the second image was captured in a second pose based on metadata associated with the second image.

[0032] The first image 201 can be processed using the first ML model 211 to generate a first ML output 221 of the first ML model 211. The first ML output 221 can indicate whether an anomaly is present in the first image 201. In some implementations, the first ML output 221 can indicate whether a particular type of anomaly is present in the first image 201. The second image 203 can be processed using the second ML model 213 as an input to generate a second ML output 223 of the second ML model 213. The second ML output 223 can indicate whether an anomaly is present in the second image 203. In some implementations, the second ML output 223 can indicate whether a particular type of anomaly is present in the second image 203.

[0033] Based on the first ML output 221 and / or the second ML output 223, it can be determined whether an anomaly (possibly of a particular type) has been detected for the particular component. For example, the first and second regions can be different regions of one or more particular components, and based on the first ML output 221 indicating an anomaly (e.g., a leak) and the second ML output 223 indicating no anomaly, the anomaly detection engine (e.g., 1052 in FIG. 1 ) can determine that an anomaly has been detected for the particular component (in the first region). In this example, the anomaly detection engine and / or other components can perform one or more remedial actions, including, but not limited to, generating and delivering (audibly and / or visually) an alarm message to one or more client devices of relevant personnel in the industrial facility to schedule maintenance or more detailed investigation, pausing or stopping any process involving the particular component, and / or controlling a mobile robot to capture additional images of higher resolution or different type (than the first image 201 and the second image 203).

[0034] In some implementations, the anomaly detection engine can determine whether an anomaly has been detected for a particular component (e.g., determine whether to trigger the rendering of an alert indicating that an anomaly is present) by determining whether the first ML output 221 (e.g., a value indicating the presence or absence of an anomaly) meets a threshold and determining whether the second ML output 223 (the “second output”) meets a threshold.

[0035] In some implementations, the anomaly detection engine may determine to cause the rendering of an alert only if both the first output (e.g., 0.8, where a number closer to "1" is more indicative of an anomaly being present and a value closer to "0" is less indicative of an anomaly) meets a threshold (e.g., 0.7) and the second output (e.g., 0.9) meets the threshold (0.7). In some implementations, the anomaly detection engine may determine to cause the rendering of an alert if either the first output (e.g., 0.8) or the second output (e.g., 0.6) meets the threshold (0.7).

[0036] In some implementations, the anomaly detection engine can determine a combined output based on the first output and the second output, and can determine whether to trigger rendering of an alarm based on the combined output. In some implementations, the anomaly detection engine can determine a level of an alarm to be rendered based on both the first output and the second output. For example, if the combined output determined from the first and second outputs is determined to meet a first threshold (e.g., 0.7) but not a second threshold (e.g., 0.8), a first type of alarm (e.g., a non-emergency alarm) can be determined and visually rendered via one or more displays of one or more computing devices in the industrial automation facility. If the combined output is determined to meet a second threshold (e.g., 0.8) but not a third threshold (e.g., 0.9), a second type of alarm (e.g., an emergency alarm) can be determined and visually rendered via one or more displays. If the combined output is determined to meet a third threshold (e.g., 0.9), a third type of alert (e.g., an emergency alert) may be determined and visually rendered via one or more displays and a text alert may be delivered to a telephone number of a person responsible for the particular component. If the combined output is determined to not meet the first threshold (e.g., 0.7), no alert is generated or rendered.

[0037] As another example, if the first and second outputs are each determined to meet a first threshold (e.g., 0.7) but not a second threshold (e.g., 0.7), a first type of alert (e.g., a non-emergency alert) may be determined and visually rendered via a display of a computing device in the industrial automation facility. If the first and second outputs are each determined to meet a second threshold (e.g., 0.8) but not a third threshold (e.g., 0.9), a second type of alert (e.g., an emergency alert) may be determined and rendered via a display. If the first and second outputs are each determined to meet a third threshold (e.g., 0.9), a third type of alert (e.g., an emergency alert) may be determined and rendered via a display and / or via an additional computing device. If either the first or second outputs do not meet the first threshold (e.g., 0.7), no alert is generated or rendered.

[0038] In some implementations, the threshold value of the first output, the second output, or the combined output that triggers an alarm may be different from the threshold value (if any) that triggers the suspension of the process involved in the particular component in which the anomaly is detected. For example, the threshold value of the first, second, and / or combined output for triggering the suspension of the process involved in the particular component may be a higher numerical value (e.g., 0.95) than the aforementioned first, second, and / or third threshold value. In other words, in some implementations, in response to the combined output meeting the threshold value for triggering the suspension (e.g., 0.95) only if the combined output (or alternatively, the first and / or second output) meets / exceeds the numerical value of about 0.95, all processes involved in the particular component are suspended.

[0039] The example of FIG. 2 is described with respect to capturing images of a particular component from a first pose and a second pose, using a first ML model to process the first pose images, and using a second ML model to process the second pose images, and using the output generated from the processing in determining whether an anomaly exists. However, in various implementations, more ML models, or only a single ML model, may be utilized in determining whether an anomaly exists associated with a given component. For example, there may be three (or four or more) ML models, each trained for a given component, but each for a different corresponding pose of the visual component. In such an example, images of a given component may be captured from a first pose, a second pose, and a third pose, a first ML model is used to process the first pose images, a second ML model is used to process the second pose images, and a third ML model is used to process the third pose images, and the output generated from the processing may be used in determining whether an anomaly exists.

[0040] 3 is a flowchart illustrating an example method 300 for implementing selected aspects of the present disclosure, according to implementations disclosed herein. For convenience, the operations of the flowchart are described with reference to a system that performs the operations. The system may include various components of various computer systems, such as one or more components of the server device 105 (and / or additional computing devices, such as the mobile robot 101 or the client device 103-A), including the ML engine 1051 and / or the anomaly detection engine 1052. Furthermore, although the operations of the method 300 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added.

[0041] In block 302A, the system identifies a first image captured by the camera of the mobile robot when the camera is in a first pose. In block 302B, the system identifies a second image captured by the camera of the mobile robot when the camera is in a second pose that is different from the first pose. The first image and the second image both capture one or more particular components of the industrial automation facility, but capture the one or more particular components from different perspectives. Optionally, the second image can be captured by an additional mobile robot carrying an additional camera that is the same as, similar to, or different from the camera.

[0042] In some implementations, the mobile robot is a wheeled robot, an unmanned aerial vehicle, or a four-legged robot that is movable within an industrial automation facility. The mobile robot can be controlled, for example, to move along a predefined path and / or for the camera (or any other vision component referenced above) to pause at one or more predefined spots to capture an image at a given pose (e.g., a first or second pose). In some implementations, the camera is fixed relative to the mobile robot such that controlling the mobile robot can control or adjust the pose of the camera. In some implementations, the camera is movable (e.g., rotationally and / or translationally) relative to the mobile robot while operably coupled to the mobile robot. In this case, after pausing the mobile robot at a given pose, the camera can be further controlled to capture an image at the given pose (first or second pose). In some implementations, two or more mobile robots can be deployed in an industrial automation facility to capture one or more images (e.g., in parallel). In some implementations, the camera of the mobile robot is a monographic camera, a stereographic camera, or a thermal camera.

[0043] In some implementations, the mobile robot can be controlled to move from a first predefined spot to a second predefined spot such that the camera can capture a first image in a first pose and further capture a second image in a second pose. In some implementations, the first image and the second image can capture the same region of the one or more specific components at different resolutions. In some implementations, the first image and the second image can capture different regions of the one or more specific components. The different regions may include overlapping regions, although this is not necessarily the case in some circumstances. For example, the first image can capture a first portion of the one or more specific components and the second image can capture a second portion of the one or more specific components that complements the first portion, such that the one or more specific components are captured in their entirety. In some implementations, the first image and the second image can be of different types, although this is not necessarily the case.

[0044] At block 304A, a first machine learning (ML) model may be selected for processing the first image. In response to the first image being captured in the first pose, the system may select the first ML model based on the first ML model being trained based on training instances corresponding to the first pose. At block 306A, the system may process the first image using the selected first ML model to generate a first output indicating whether an anomaly is present in the first image.

[0045] In some implementations, the first ML model is trained using one or more training instances, each of which includes (1) an image captured by a camera and determined to correspond to a first pose in the absence of an anomaly for one or more particular components, and (2) a ground truth label indicating that no anomaly is present for the one or more particular components.

[0046] At block 304B, the system may select, by a server, such as server device 105, a second ML model for processing the second image, where selecting the second ML model is responsive to the second image being captured at a second pose and the second ML model being trained based on training instances corresponding to the second pose. At block 306B, the system may process the second image using the selected second ML model to generate a second output indicating whether an anomaly is present in the second image.

[0047] In some implementations, the second ML model is trained using one or more training instances, each of which includes (1) an image captured by a camera and determined to correspond to a second pose in the absence of an anomaly for one or more particular components, and (2) a ground truth label indicating that no anomaly is present for the one or more particular components.

[0048] At block 308, the system may determine whether to cause the rendering of an alarm indicating that an anomaly exists for one or more particular components based on both the first output and the second output. In some implementations, the system may determine whether to cause the rendering of an alarm indicating that an anomaly exists by determining whether the first output meets a threshold, determining whether the second output meets a threshold, and determining to cause the rendering of an alarm if either the first output or the second output meets the threshold.

[0049] In some implementations, the system can determine whether to cause the rendering of an alarm indicating that an anomaly exists by determining whether the first output meets a threshold, determining whether the second output meets a threshold, and determining to cause the rendering of an alarm only if both the first output meets the threshold and the second output meets the threshold. In some implementations, the system can determine whether to cause the rendering of an alarm indicating that an anomaly exists by determining a combined output based on the first output and the second output, and determining whether to cause the rendering of an alarm based on the combined output.

[0050] In some implementations, the system may render an alert in response to a determination to cause rendering of an alert. The alert may be rendered audibly and / or visually. For example, the alert may be rendered audibly via one or more speakers located within the industrial automation facility. Additionally or alternatively, the alert may be rendered in the form of a message to one or more computing devices within or outside the industrial automation facility. For example, a desktop in a monitoring room may receive an alert message indicating that an anomaly has been detected for one or more particular components, the type of anomaly, and / or contact information for a person responsible for the one or more particular components.

[0051] Alternatively or in addition to determining to cause the rendering of an alarm, the system may perform one or more other remedial actions. The one or more other remedial actions may include pausing or stopping one or more industrial processes (e.g., automation processes) involving the one or more particular components in which the anomaly was detected. For example, the system may determine to cause a pause of a process involving one or more particular components within the industrial automation facility based on both the first output and the second output.

[0052] The methods described herein or in other aspects of the present disclosure may be executed via one or more processors of one or more computing devices separate from and not attached to the mobile robot, in which case the first and second images are transmitted by the mobile robot and identified by the one or more computing devices after being transmitted by the mobile robot.

[0053] 4 is a block diagram of an exemplary computing device 410 that may be optionally utilized to perform one or more aspects of the techniques described herein. The computing device 410 typically includes at least one processor 414 that communicates with several peripheral devices via a bus subsystem 412. These peripheral devices may include, for example, a storage subsystem 424 including a memory subsystem 425 and a file storage subsystem 426, a user interface output device 420, a user interface input device 422, and a network interface subsystem 416. The input and output devices allow a user to interact with the computing device 410. The network interface subsystem 416 provides an interface to external networks and is coupled to corresponding interface devices in other computing devices.

[0054] The user interface input devices 422 may include a keyboard, a pointing device such as a mouse, a trackball, a touchpad, or a graphics tablet, a scanner, a touch screen integrated into a display, an audio input device such as a voice recognition system, a microphone, and / or other types of input devices. In general, use of the term "input device" is intended to include all possible types of devices and methods for inputting information into the computing device 410 or a communications network.

[0055] The user interface output devices 420 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a flat panel display such as a cathode ray tube (CRT), a liquid crystal display (LCD), a projection device, or some other mechanism for creating a visual image. The display subsystem may also provide a non-visual display, such as via an audio output device. In general, use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computing device 410 to a user or to another machine or computing device.

[0056] Storage subsystem 424 stores programming and data structures that provide the functionality of some or all of the modules described herein. For example, storage subsystem 424 may include logic for implementing the various components shown in Figures 1-2 as well as for performing selected aspects of the method of Figure 3.

[0057] These software modules are generally executed by the processor 414 alone or in combination with other processors. The memory 425 used within the storage subsystem 424 may include several memories, including a main random access memory (RAM) 430 for storing instructions and data during program execution, and a read-only memory (ROM) 432 in which fixed instructions are stored. The file storage subsystem 426 may provide persistent storage for program and data files and may include hard disk drives, floppy disk drives, along with associated removable media, CD-ROM drives, optical drives, or removable media cartridges. Modules implementing the functionality of a particular implementation may be stored by the file storage subsystem 426 within the storage subsystem 424, or may be stored within other machines accessible by the processor 414.

[0058] The bus subsystem 412 provides a mechanism for allowing the various components and subsystems of the computing device 410 to communicate with each other as intended. Although the bus subsystem 412 is shown generally as a single bus, alternative implementations of the bus subsystem may use multiple buses.

[0059] The computing device 410 can be of various types, including a workstation, a server, a computing cluster, a blade server, a server farm, or any other data processing system or computing device. Due to the ever-changing nature of computers and networks, the description of the computing device 410 shown in Figure 4 is intended only as a specific example for purposes of illustrating some implementations. Many other configurations of the computing device 410 are possible, having more or fewer components than the computing device shown in Figure 4.

[0060] Although several implementations have been described and illustrated herein, various other means and / or structures for performing the functions and / or obtaining one or more of the results and / or advantages described herein may be utilized, and each such variation and / or modification is deemed to be within the scope of the implementations described herein. More generally, all parameters, dimensions, materials, and configurations described herein are meant to be exemplary, and the actual parameters, dimensions, materials, and / or configurations will depend on the particular application in which the teachings are used. Those skilled in the art will recognize, or be able to ascertain using no more than routine experimentation, many equivalents to the specific implementations described herein. Thus, the foregoing implementations are presented by way of example only, and it should be understood that within the scope of the appended claims and their equivalents, implementations may be practiced otherwise than as specifically described and claimed. Implementations of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. Additionally, any combination of two or more such features, systems, articles, materials, kits, and / or methods is included within the scope of the present disclosure, if such features, systems, articles, materials, kits, and / or methods are not mutually inconsistent.

[0061] In some implementations, a method implemented using a processor is provided, the method including identifying a first image captured by a camera of the mobile robot when the camera is in a first pose, and identifying a second image captured by the camera of the mobile robot when the camera is in a second pose different from the first pose. The first image and the second image both capture one or more particular components of the mobile robot's environment, but capture the one or more particular components from different perspectives. The mobile robot can be, for example, a wheeled robot, an unmanned aerial vehicle, or a four-legged robot. The camera of the mobile robot can be, for example, a monographic camera, a stereographic camera, or a thermal camera. The one or more processors can be, for example, of one or more computing devices separate from and not attached to the mobile robot, and the first and second images can be transmitted by the mobile robot and identified by the computing devices after being transmitted by the mobile robot.

[0062] In various implementations, the method may further include selecting a first machine learning (ML) model for processing the first image, where selecting the first ML model is responsive to the first image being captured at the first pose and the first ML model being trained based on training instances corresponding to the first pose. In those implementations, the method may further include processing the first image with the selected first ML model to generate a first output indicating whether an anomaly (e.g., associated with one or more particular components of the industrial automation facility) is present.

[0063] In various implementations, the method may further include selecting a second ML model for processing the second image, where selecting the second ML model is responsive to the second image being captured at a second pose and the second ML model being trained based on training instances corresponding to the second pose. In those implementations, the method may further include processing the second image with the selected second ML model to generate a second output indicating whether an anomaly is present.

[0064] In various implementations, the method may further include determining, based on both the first output and the second output, whether to cause rendering of an alarm indicating that an anomaly exists. In some implementations, determining, based on both the first output and the second output, whether to cause rendering of an alarm indicating that an anomaly exists may include determining whether the first output meets a threshold, determining whether the second output meets a threshold, and determining to cause rendering of an alarm if either the first output or the second output meets the threshold. In these or other implementations, the method may further include rendering the alarm in response to the determination to cause rendering of the alarm.

[0065] In some implementations, determining whether to cause the rendering of an alert indicating that an anomaly exists based on both the first output and the second output can include determining whether the first output meets a threshold, determining whether the second output meets a threshold, and determining to cause the rendering of the alert only if both the first output meets the threshold and the second output meets the threshold.

[0066] In some implementations, determining whether to cause rendering of an alert indicating that an anomaly exists based on both the first output and the second output may include determining a combined output based on the first output and the second output, and determining whether to cause rendering of an alert based on the combined output.

[0067] In some implementations, the first ML model is trained using one or more training instances, each of which includes (1) an image captured by a camera and determined to correspond to a first pose in the absence of an anomaly associated with one or more particular components, and (2) a ground truth label indicating the absence of an anomaly associated with one or more particular components.

[0068] In some implementations, the second ML model is trained using one or more training instances, each of which includes (1) an image captured by the camera or an additional camera corresponding to a second pose in the absence of an anomaly associated with one or more particular components, and (2) a ground truth label indicating the absence of an anomaly associated with the one or more particular components.

[0069] In some implementations, instead of or in addition to determining to cause the rendering of an alert, the method may include determining, based on both the first output and the second output, to cause a suspension of a process involving one or more particular components within the industrial automation facility.

[0070] In addition, some implementations include one or more processors of the mobile robot and / or one or more computing devices, the one or more processors operable to execute instructions stored in associated memory, the instructions configured to cause performance of any of the methods disclosed herein. Some implementations also include one or more transitory or non-transitory computer-readable storage media that store computer instructions executable by the one or more processors to perform any of the methods disclosed herein.

[0071] In some implementations, a system is provided that includes one or more processors and a memory storing instructions that, responsive to execution by the one or more processors, cause the one or more processors to identify a first image captured by a camera of the mobile robot when the camera is in a first pose, identify a second image captured by the camera of the mobile robot when the camera is in a second pose that is different from the first pose, select a first machine learning (ML) model for processing the first image, process the first image using the selected first ML model to generate a first output indicating whether an anomaly is present, select a second ML model for processing the second image, process the second image using the selected second ML model to generate a second output indicating whether an anomaly is present, and determine, based on both the first output and the second output, whether to trigger rendering of an alert indicating the anomaly is present.

[0072] In various implementations, execution of the instructions by the one or more processors further causes the one or more processors to determine, based on the first output and / or the second output indicating there is an anomaly associated with the one or more particular components, to cause rendering of an alarm, visually and / or audibly. ...

[0073] In some implementations, a mobile robot is provided that includes a camera, a memory that stores instructions and that stores a trained machine learning (ML) model, and one or more processors. The processor is operable to execute instructions to receive from the camera a first image captured when the camera is in a first pose, receive from the camera a second image captured by the camera of the mobile robot when the camera is in a second pose different from the first pose, select a first machine learning (ML) model for processing the first image, process the first image using the selected first ML model to generate a first output indicative of whether an anomaly is present, select a second ML model for processing the second image, process the second image using the selected second ML model to generate a second output indicative of whether an anomaly is present, and determine whether to trigger rendering of an alert indicating the anomaly is present based on both the first output and the second output. [Explanation of symbols]

[0074] 100 Environment 101 Mobile Robot 102 Tube, Liquid Tube 103-A Local Client Device, Client Device 103-B Local Client Device, Client Device 105 Server Device 106 Process Automation Network 107 Warning Message 122-A Local Client Device 122-B Local client device 201 1st image 203 Second Image 211 The first ML model 213 Second ML Model 221 First ML output 223 Second ML Output 410 Computing Devices 412 Bus Subsystem 414 processor 416 Network Interface Subsystem 420 User Interface Output Device 422 User Interface Input Devices 424 Memory Subsystem 425 Memory Subsystem 426 File Storage Subsystem 430 Main Random Access Memory (RAM) 432 Read-Only Memory (ROM) 1011 Visual Components 1051 Machine Learning (ML) Engine, ML Engine 1052 Anomaly Detection Engine 1053 Pose-dependent machine learning (ML) model, Pose-dependent ML model, ML model

Claims

1. 1. A method implemented by one or more processors, comprising: identifying a first image captured by a camera of a mobile robot when the camera is in a first pose; identifying a second image captured by the camera of the mobile robot when the camera is in a second pose different from the first pose; the first image and the second image both capture one or more particular components, but capture the one or more particular components from different perspectives; selecting a first machine learning (ML) model for processing the first image, wherein selecting the first ML model is responsive to the first image being captured in the first pose and the first ML model being trained based on an image corresponding to the first pose; processing the first image using the selected first ML model to generate a first output indicative of whether an anomaly is present; and selecting a second ML model for processing the second image, wherein selecting the second ML model is responsive to the second image being captured in the second pose and the second ML model being trained based on an image corresponding to the second pose; processing the second image using the selected second ML model to generate a second output indicative of whether an anomaly exists for the one or more particular components; determining whether to cause the rendering of an alarm indicating that the anomaly exists based on both the first output and the second output; A method comprising:

2. determining whether to cause the rendering of the alarm indicating that the anomaly exists based on both the first output and the second output; determining whether the first output satisfies a threshold; determining whether the second output satisfies the threshold; determining if either the first output or the second output satisfies the threshold to trigger the rendering of the alert; 2. The method of claim 1, comprising:

3. determining whether to cause the rendering of the alarm indicating that the anomaly exists based on both the first output and the second output; determining whether the first output satisfies a threshold; determining whether the second output satisfies the threshold; determining to cause rendering of the alert only if both the first output meets the threshold and the second output meets the threshold; 2. The method of claim 1, comprising:

4. determining whether to cause the rendering of the alarm indicating that the anomaly exists based on both the first output and the second output; determining a combined output based on the first output and the second output; determining whether to trigger rendering of the alert based on the combined output; 2. The method of claim 1, comprising:

5. The method of claim 1 , wherein the mobile robot is a wheeled robot, an unmanned aerial vehicle, or a four-legged robot.

6. The method of claim 1 , wherein the camera of the mobile robot is a monographic camera, a stereographic camera, or a thermal camera.

7. 2. The method of claim 1 , wherein the first ML model is trained using one or more training instances, each of which includes (1) an image captured by the camera or an additional camera corresponding to the first pose in the absence of an anomaly associated with the one or more particular components, and (2) a ground truth label indicating the absence of an anomaly associated with the one or more particular components.

8. 2. The method of claim 1 , wherein the second ML model is trained using one or more training instances, each of which includes (1) an image captured by the camera or an additional camera corresponding to the second pose in the absence of an anomaly associated with the one or more particular components, and (2) a ground truth label indicating that no anomaly is present.

9. 2. The method of claim 1, wherein the first image captures a first portion of the one or more particular components and the second image captures a second portion of the one or more particular components that is complementary to the first portion.

10. The method of claim 1 , wherein the first image and the second image capture the same region of the one or more particular components.

11. The method of claim 1 , further comprising the step of rendering the alert in response to a decision to cause rendering of the alert.

12. based on both the first output and the second output, The method of claim 1 , further comprising determining to cause a suspension of processes involving the one or more particular components within an environment.

13. 2. The method of claim 1 , wherein the one or more processors are of one or more computing devices separate from and not attached to the mobile robot, and the first and second images are transmitted by the mobile robot and identified by the computing devices after being transmitted by the mobile robot.

14. one or more computing devices including a camera and in at least selective network communication with a mobile robot deployed in an environment, the one or more computing devices comprising: receiving a first image from the mobile robot captured by the camera of the mobile robot when the camera is in a first pose; receiving a second image from the mobile robot captured by the camera of the mobile robot when the camera is in a second pose different from the first pose; receiving the first image and the second image, both capturing one or more particular components, but capturing the one or more particular components from different perspectives; selecting a first machine learning (ML) model for processing the first image, wherein selecting the first ML model is responsive to the first image being captured in the first pose and the first ML model being trained based on an image corresponding to the first pose; processing the first image using the selected first ML model to generate a first output indicative of whether an anomaly is present; and selecting a second ML model for processing the second image, wherein selecting the second ML model is responsive to the second image being captured in the second pose and the second ML model being trained based on an image corresponding to the second pose; processing the second image using the selected second ML model to generate a second output indicative of whether an anomaly is present; and determining whether to cause the rendering of an alarm indicating that an anomaly exists for the one or more particular components based on both the first output and the second output; A system that performs the above.

15. The system of claim 14 , wherein the environment is an industrial automation facility and the one or more computing devices are located within the industrial automation facility.

16. In determining whether to cause the rendering of the alert, the one or more computing devices: determining whether the first output meets a threshold; determining whether the second output satisfies the threshold; and determining if either the first output or the second output satisfies the threshold to cause the rendering of the alert; The system of claim 14, further comprising:

17. In determining whether to cause the rendering of the alert, the one or more computing devices: determining whether the first output meets a threshold; determining whether the second output satisfies the threshold; and determining to cause rendering of the alert only if both the first output meets the threshold and the second output meets the threshold; The system of claim 14, further comprising:

18. In determining whether to cause the rendering of the alert, the one or more computing devices: determining a combined output based on the first output and the second output; determining whether to trigger rendering of the alert based on the combined output; The system of claim 14, further comprising:

19. the one or more computing devices; determining that the first image was captured in the first pose based on metadata associated with the first image and received from the mobile robot, wherein selecting the first ML model for processing the first image is responsive to determining that the first image was captured in the first pose. The system of claim 14, further comprising:

20. the one or more computing devices; determining that the first image was captured in the second pose based on metadata associated with the second image and received from the mobile robot, wherein selecting the second ML model for processing the second image is responsive to determining that the second image was captured in the second pose. The system of claim 14, further comprising:

Citation Information

Patent Citations

  • Robot control system

    JP2019087114A

  • Plant health monitoring method and apparatus

    JP2020531014A

  • Estimating device, learning device, estimating method, and program

    JP2021165984A

  • System, method and device for automatic meter reading for smart field patrol

    JP2023029227A

  • Autonomous and teleoperated sensor pointing on a mobile robot

    US20220388174A1