Automated image clustering-based image-to-asset linking
An automated clustering-based image-to-asset linking system addresses the inefficiencies of manual labeling by using machine learning to efficiently cluster and link images with assets, enhancing accuracy and reducing resource consumption.
Patent Information
- Application Number
- PCT/US2026/028454
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-15
- Filing Date
- 2026-05-15
- Publication Date
- 2026-11-19
AI Technical Summary
Current asset management systems face challenges in accurately and reliably linking images to assets, leading to inefficiencies, resource wastage, and inaccuracies due to environmental conditions, equipment limitations, and human errors in manual labeling processes.
An automated clustering-based image-to-asset linking system utilizing statistical analysis and machine learning techniques to efficiently cluster and associate images with assets, reducing the need for manual labeling and optimizing computing, network, and storage resources.
The system enables reliable, efficient, and accurate linking of images to assets, minimizing data processing and storage requirements while improving operational efficiency and reducing errors.
Smart Images

Figure US2026028454_19112026_PF_FP_ABST
Abstract
Description
AUTOMATED IMAGE CLUSTERING-BASED IMAGE-TO-ASSET LINKINGFIELD
[0001] The present disclosure relates generally to systems and methods for managing equipment, and automated maintenance related activities therefore. In particular, the present disclosure is related to techniques utilized to interface with, and obtain from, various types of devices, data associated with the equipment. The techniques include automatically managing images of the equipment. The present disclosure also relates to image linking techniques that include leveraging various ty pes of information about the images to automate generation of links between the images and the equipment.BACKGROUND
[0002] Various ty pes of data are utilized for purposes of maintaining and managing equipment. Equipment management and maintenance documentation is utilized for different purposes pertaining to the equipment, in various types of scenarios, and / or under various conditions. The documentation associated with the equipment includes various types of documentation that have different purposes and / or are available for utilization by different types of personnel. Data is generated and utilized to track guidelines, procedures, and schedules for maintaining and managing equipment throughout its lifecycle. The equipment related data is utilized to identify details and characteristics associated with functions and / or properties of the equipment. The details and characteristics associated with the equipment are tracked to determine operational conditions of the equipment.
[0003] Various types of contents are included in the various types of equipment data. The equipment data includes identifiers utilized to manage and maintain the data. The data includes other information pertaining to the equipment and utilized to describe aspects of the equipment, and the status thereof. The data is also utilized to track information associated with where the equipment is located and what type of condition the equipment is in. Tasks for maintaining and managing the equipment are performed utilizing the data that is gathered for the equipment. The tasks are performed at various times and by various types of personnel.SUMMARY
[0004] Aspects and advantages of embodiments of the disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.
[0005] One example aspect of the present disclosure is directed to an asset condition monitoring system. The asset condition monitoring system can include one or more processors. The system can also include one or more databases that collectively store a plurality of machine-health diagnostic files and a plurality of asset identifiers (ids) respectively associated with a plurality of machine assets. The system can also include one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the asset condition monitoring system to perform operations. The operations can include obtaining a plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files. The operations can include clustering the plurality of machine-health diagnostic files into a plurality of clusters based on the plurali ly of sets of metadata respectively associated with the plurality of machine-health diagnostic files. The operations can include associating at least one of the plurality of clusters with one or more of the asset ids associated with one or more of the plurality of machine assets.
[0006] The condition monitoring system, where at least a subset of the plurality’ of machine-health diagnostic files can include a plurality of machine-health diagnostic images; each of the machine-health diagnostic images can include a visible-light image, an acoustic image, or a thermal image; and each of the machine-health diagnostic images depicts one or more of the machine assets or an area adjacent to the one or more of the machine assets. The respective set of metadata associated each of the machine-health diagnostic images can include a capture time associated with the machine-health diagnostic image; and clustering the plurality’ of machine-health diagnostic files can include clustering the plurality' of machine-health diagnostic images. Clustering the plurality of machine-health diagnostic images can include clustering the plurality of machine-health diagnostic images based on a user-defined hyperparameter of maximum time difference, the user-defined hyperparameter of maximum time difference specifying a maximum permissible time difference betweencapture times of any two machine-health diagnostic images contained in the same cluster. Clustering the plurality of machine-health diagnostic files can include generating a plurality of image embeddings respectively for the plurality of machinehealth diagnostic images, where the respective image embedding for each machinehealth diagnostic image can include a numerical vector that represents visual features contained in the machine-health diagnostic image: and clustering the plurality of machine-health diagnostic images based on the plurality of image embeddings.Clustering the plurality of machine-health diagnostic files can include generating a plurality of extracted text strings respectively for the plurality' of machine-health diagnostic files, where the respective extracted text strings for each machine-health diagnostic file can include strings of text extracted from the machine-health diagnostic file; and clustering the plurality of machine-health diagnostic files based on the plurality of extracted text strings. Clustering the plurality of machine-health diagnostic files based on the plurality' of extracted text strings can include determining a plurality’ of edit distances values between the plurality of extracted text strings; and clustering the plurality of machine-health diagnostic files based on the plurality of edit distance values. Clustering the plurality of machine-health diagnostic files based on the plurality7of extracted text strings can include: receiving a user input that specifies a particular text string; searching the extracted text strings for the plurality' of machine-health diagnostic files to identify any machine-health diagnostic files that contain the particular text string; and inserting into one of the plurality of clusters, any machine-health diagnostic files that were identified as containing the particular text string. Clustering the plurality of machine-health diagnostic files can include generating, for each of the machine-health diagnostic files, multiple scores according to multiple different clustering criteria, the clustering criteria including asset locations and geospatial information associated with the plurality of machine assets; determining, for each of the machine-health diagnostic files, a score from the multiple scores; and clustering the plurality of machine-health diagnostic files based on the scores. Clustering the plurality of machine-health diagnostic files can include providing a drop down menu that includes a number of clustering criteria: receiving a user input that selects one of the clustering criteria; and performing said clustering based on the selected one of the clustering criteria. The operations further can includeproviding a graphical user interface that graphically depicts the plurality of clusters of machine-health diagnostic files; receiving a user input that selects one of the clusters from the graphical user interface; and visually presenting, within the graphical user interface, at least one of each individual machine-health diagnostic file contained within the selected cluster, data associated with the machine-health diagnostic file, or content associated with the machine-health diagnostic file. Associating the at least one of the plurality of clusters with the one or more of the asset ids associated with the one or more of the plurality of machine assets can include providing a graphical user interface that enables a user to search for machine assets using one or more search strings; receiving one or more user inputs that define one or more search strings; identifying the one or more asset ids based on the one or more search strings; and associating the at least one of the plurality of clusters with the one or more of the asset ids identified using the one or more search strings. Associating the at least one of the plurality of clusters with the one or more of the asset ids associated with the one or more of the plurality of machine assets can include automatically associating the at least one of the plurality of clusters with the one or more of the asset ids based on a comparison of the respective sets of metadata associated with the machine-health diagnostic files contained in the at least one of the plurality of clusters and metadata associated with the one or more of the plurality of machine assets.
[0007] Another example of the present disclosure is directed to a method to associate a plurality of asset-health diagnostic files and a plurality of asset identifiers (IDs) respectively associated with a plurality of asset. The method can include obtaining, by a computing system can include one or more computing devices, a plurality of sets of metadata respectively associated with the plurality of asset-health diagnostic files; clustering, by the computing system, the plurality of asset-health diagnostic files into a plurality of clusters based on the plurality of sets of metadata respectively associated with the plurality of asset-health diagnostic files; and associating, by the computing system, at least one of the plurality of clusters with one or more of the asset ids associated with one or more of the plurality of assets.
[0008] The computer-implemented method, where at least a subset of the plurality of asset-health diagnostic files can include a plurality of asset-health diagnostic images; each of the asset-health diagnostic images can include a visible-light image,an acoustic image, or athermal image; and clustering, by the computing system, the plurality of asset-health diagnostic fdes can include clustering, by the computing system, the plurality of asset-health diagnostic images. Clustering, by the computing system, the plurality of asset-health diagnostic images can include clustering, by the computing system, the plurality of asset-health diagnostic images based on a user-defined hyperparameter of maximum time difference, the user-defined hyperparameter of maximum time difference specifying a maximum permissible time difference between capture times of any two asset-health diagnostic images contained in the same cluster. Clustering, by the computing system, the plurality of asset-health diagnostic files can include generating, by the computing system, a plurality of image embeddings respectively for the plurality of asset-health diagnostic images, where the respective image embedding for each asset-health diagnostic image can include a numerical vector that represents visual features contained in the asset-health diagnostic image; and clustering, by the computing system, the plurality of assethealth diagnostic images based on the plurality of image embeddings. Clustering, by the computing system, the plurality of asset-health diagnostic files can include generating, by the computing system, a plurality' of extracted text strings respectively for the plurality' of asset-health diagnostic files, where the respective extracted text strings for each asset-health diagnostic file can include strings of text extracted from the asset-health diagnostic file; and clustering, by the computing system, the plurality of asset-health diagnostic files based on the plurality of extracted text strings.Clustering, by the computing system, the plurality of asset-health diagnostic files can include identifying, by the computing system, a search string associated with at least one of text, an image, a timestamp, or a location, where clustering, by the computing system, the plurality of asset-health diagnostic files can include clustering, by the computing system, the plurality of asset-health diagnostic files further based on the search string. Clustering, by the computing system, the plurality' of asset-health diagnostic files can include identifying, by the computing system, a plurality of locations associated with the plurality of assets, respectively; identifying at least two locations from among the plurality of locations that are separated from one another by less than a distance threshold; and clustering, by the computing system, at least two of the plurality' of asset-health diagnostic files associated with the at least two locations.
[0009] Other example aspects of the disclosure are directed to other systems, methods, apparatuses, tangible non-transitory computer-readable media, and devices for performing functions described herein. These and other features, aspects, and advantages of various implementations will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate implementations of the disclosure and, together with the description, help explain the related principles.BRIEF DESCRIPTION OF THE DRAWINGS
[0010] A full and enabling disclosure of the present defect event detection systems and methods, including the best mode of making and using the present systems and methods, directed to one of ordinary skill in the art, is set forth in the specification, which makes reference to the appended figures, in which:
[0011] FIG. 1 illustrates an example automated clustering-based image-to-asset linking networked computing environment in accordance with an exemplary aspect of the present disclosure;
[0012] FIGS. 2A-2E illustrate block diagrams of components of an example system utilized for automated clustering-based image-to-asset linking in accordance with embodiments of the present disclosure;
[0013] FIG. 3 is an example user interface (UI) utilized to identify images, and clusters of images utilized, for image-to-asset linking in accordance with embodiments of the present disclosure;
[0014] FIG. 4 is an example UI utilized to output images utilized for image clustering, and image-to-asset linking, in accordance with embodiments of the present disclosure;
[0015] FIG. 5 illustrates an example UI utilized for file selection for image-to-asset linking in accordance with embodiments of the present disclosure;
[0016] FIG. 6 illustrates an example UI utilized for asset selection for image-to-asset linking in accordance with embodiments of the present disclosure;
[0017] FIG. 7 illustrates an example UI utilized for outputting selected asset information in accordance with embodiments of the present disclosure;
[0018] FIG. 8 illustrates example sets of images for outputting linked asset information in accordance with embodiments of the present disclosure;
[0019] FIGS. 9 and 10 illustrate flow diagrams of methods of automated clustering-based image-to-asset linking in accordance with embodiments of the present disclosure; and
[0020] FIG. 11 illustrates a block diagram of a computing system that performs image clustering, and image-to-asset linking in accordance with embodiments of the present disclosure.DETAILED DESCRIPTION
[0021] Reference now will be made in detail to example systems and methods for automated clustering-based image-to-asset linking, one or more examples of which are illustrated in the drawings.
[0022] The technical problem that this invention aims to solve is the inability of current asset management systems to accurately and reliably link images to assets. Errors, omissions, and / or inaccuracies of linking images to assets occur in existing systems, possibly due to environmental conditions, equipment limitations, service mishaps, and / or various other factors. As such, although related technologies focus on various strategies, such as manual labeling, to link images to assets, unsuccessful linking that occurs causes unnecessary consumption and expenditures of computing, network, and storage resources. Existing solutions are unable to overcome certain defects in generating, updating, and removing image links to assets. Furthermore, current techniques are cumbersome, resource heavy, and error prone, resulting in lower quality, increased costs, and inefficiencies in operations performed by devices and equipment utilized for image-to-asset linking.
[0023] Current systems undergo difficulties in organizing various types of images, such as thermal and acoustic images. Existing techniques for linking images to assets are inconvenient. According to conventional techniques, technicians performing image-to-asset linking manually experience difficulties in managing images due to strenuous levels of their jobs and time constraints of overly full maintenance schedules. Variations, such as natural inconsistencies and human errors, which arise in service personnel-based image management result in problems and negativerepercussions in image-to-asset linking scenarios. Technicians occasionally forget to label images and / or capture images identifying assets and / or labels of assets.
[0024] According to conventional techniques, technicians occasionally capture asset label information incorrectly, such as by capturing information that is not accurate and / or poor in quality. Numbers of assets being tracked may be extremely large. Mechanisms of assets utilized to capture label information, such as heavy and / or awkward doors that are designed to be swing open and closed for each asset, occasionally make asset information difficult to capture in a timely manner. Asset identifiers being placed in inconsistent or unexpected locations on the assets may result in technicians being unable to locate the identifiers. Technicians occasionally mishandle captured information. Finally, even in cases in which technicians complete image-to-asset linking tasks, inaccuracies or quality shortfalls result in adverse effects on linking operations due to quantities of image-to-asset linking information being extremely large and / or complex to organize.
[0025] The technical solution provided by example implementations of the present disclosure lies in the application of automated clustering-based image-to-asset linking. Statistical analysis techniques and / or machine learning techniques can be utilized to automate the process of clustering-based image-to-asset linking. This solution overcomes the limitations of previous systems which relied on less optimal techniques of manual labeling, inspecting, organizing, and itemizing of assets.
[0026] In the case of image-to-asset linking, example implementations of the present disclosure can be applied during the organizing, maintaining, repairing, operating, and / or overseeing processes for linking images to assets. Additionally or alternatively, the invention can be applied as an auditing tool for historical data associated with image-to-asset linking operations. Classification of historical data can be used to train machine learned models for future image-to-asset linking operations.
[0027] Technical solutions provided by example implementations of the present disclosure can implement a variety of improvements affecting computing, network, and storage resources upon linking of images to assets. For example, the above-mentioned limitations of existing technologies can be mitigated by utilizing clustering of images to reliably, accurately, and efficiently link images to assets. Technicians are not required to capture, record, and organize asset identifiers for every individualimage. According to the techniques discussed herein, image-to-asset linking can be performed and utilized to achieve optimal levels of utilization of the computing, network, and storage resources.
[0028] In contrast to existing systems that are unable to be utilized to link certain images to assets, linking of complete sets of images (e.g. visible, thermal, and / or acoustic images) to assets according to techniques discussed herein is accomplished with clustering-based image linking capabilities more efficiently. Utilizing clustering techniques enables images to be quickly, easily, and effortlessly linked by users of the system. Furthermore, because total numbers of images needed to be captured, processed, rendered, etc., according to the techniques discussed herein are decreased, total amounts of data associated therewith that are required to be processes are lowered. Relatively fewer computing resources are consumed. Relatively fewer network resources are expended due to reliable capturing and linking of images resulting in overall fewer numbers of images needing to be captured and linked. Relatively fewer storage resources are required to be utilized by systems operating according to the techniques discussed herein, which are able to accurately capture and link images utilized for managing assets, and expel and / or relocate any unnecessary redundant images, data related thereto, and / or information generated therefrom.
[0029] Referring now to the drawings, FIG. 1 illustrates an example automated clustering-based image-to-asset linking networked computing environment 100 in accordance with an exemplary aspect of the present disclosure. More particularly, the automated clustering-based image-to-asset linking networked computing (also referred to herein simply as “environment"’) 100 can include an automated clusteringbased image-to-asset linking system (also referred to herein simply as “system"’) 102.
[0030] The system (or “asset condition monitoring system”) 102 can utilize one or more models 104 to manage automated clustering-based image-to-asset linking. Any of the model(s) 104 (e.g., the timestamp model 106, the visual model 108, the text model 110, the search model 112, the location model 114, one or more other models, as discussed below in further detail, or any combination thereof) can include any type of model. In some examples, individual ones of model(s) 104 can include corresponding machine learned models, corresponding deterministic models, corresponding stochastic models, corresponding models of other types, or anycombination thereof. In those or other examples, any of the model(s) 104 can, alternatively or additionally, include a timestamp model, a Euclidian distance (e.g.. for a location) model, any other type of model, or any combination thereof. Any of the model(s) 104 can be separate from any other model, and / or integrated with any other of the model(s) 104.
[0031] The model(s) 104 can include a timestamp model 106. a visual model 108, a text model 110, a search model 112, a location model 114, one or more of various types of other models, or any combination thereof. The system 102 can be utilized to manage (e.g., identify, determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) image data, which can include one or more images 116 and / or 118. For example, managing of the image(s) 116 and / or 118 can be performed using the model(s) 104. In some cases, individual ones of the image(s) 116 / 118 can include an asset-health diagnostic image, such as a machine-health diagnostic image.
[0032] In some examples, the image(s) 116 can include one or more thermal images 116(A) and / or one or more acoustic images 116(B) (image(s) 116(A) / ! 16(B) being also referred to herein, as appropriate, simply as “image(s) 116,” for convenience and simplicity of explanation). In those or other examples, the image(s) 118 can include one or more thermal images 118(E), one or more acoustic images 118(F) (image(s) 118(E) / 118(F) being also referred to herein, as appropriate, simply as “image(s) 118,” for convenience and simplicity of explanation), one or more other images of various types, such as one or more visual images, or any combinations thereof. For example, any of the images (e.g., the image(s) 116 / 118, and / or any other image) can have one or more layers (e.g., individual ones of layers of any type, such as a thermal layer, a acoustic layer, a visual layer, etc., and / or with data of any type, such as thermal data, acoustic data, visual data, etc.), one or more dimensions (e.g., width, height, depth, etc., or any combination thereof), or any combination thereof.
[0033] Any of the image(s) 116 may be different from, or the same as, any of the image(s) 118. In some examples, all of the image(s) 116 may be different from all of the image(s) 118. In those or other examples, any of the image(s) 116 may be the same ty pe as, or a different type from, any of the image(s) 118. In those or other examples, one or more characteristics (or “image characteristic(s)”) of any of the image(s) 116 may be the same as, or a different from, one or more characteristics ofany of the image(s) 118. Individual ones of the image characteristic(s) include, for example, a layer (e.g., a thermal layer, an acoustic layer, a visual layer), a dimension (e.g., a height, a width, a depth), a size, a resolution, a scale, one or more colors, a tone, a contrast, a nature, a position, detail, noise, distortion, etc., or any combination thereof.
[0034] Various images, such as the image(s) 116 / 118, can be captured in various ways. For example, individual ones of the image(s) 116 / 118 can be captured by one or more cameras, such as a camera 120 of a user (e.g., a technician), one or more other cameras, or any combination thereof. The camera(s), such as the camera 120, can include any of various types of cameras, such as a thermal camera, an acoustic camera, a visual camera, any other types of cameras, or any combination thereof.
[0035] The image(s) 116 / 118 can be utilized to identify various candidate assets (e.g., suggested assets identified by the system 102), and / or data associated with the candidate assets. Individual ones of the image(s) 116 and / or 118 can be utilized to identify any of the asset(s) 122 / 124, any of one or more combinations thereof, one or more other assets, and / or one or more other combinations thereof, as the candidate asset(s). In some examples, individual ones of the image(s) 116 and / or 118 can be utilized to identify any of the candidate assets, which can then be identified as one or more assets. For instance, a candidate asset from among the asset(s) 122, which is identified based on the image(s) 116 may be identified as an asset 134. In such an instance or another instance, a candidate asset from among the asset(s) 124, which is identified based on the image(s) 118 may be identified as an asset 136. The assets 134 and 136 may be part of assets (e.g., inventoried assets, database assets, datastore assets, etc., or any combination thereof) identified by asset identifiers in an asset list (or "inventoried asset list”) managed by the system 102.
[0036] Individual ones of the asset(s) 122 / 124 can include an asset of any type, such as equipment (e.g., one or more pieces of equipment). An asset (e.g., individual ones of the asset(s) 122 / 124), such as the equipment, for example, can include one or more panels (e.g., one or more electrical / mechanical panels), one or more cabinets, one or more machines, one or more other types of equipment, or any combination thereof.
[0037] Any of the asset(s) 122 may be different from, or the same as, any of the asset(s) 124. In some examples, all of the asset(s) 122 may be different from all of the asset(s) 124. In those or other examples, any of the asset(s) 122 may be the same type as, or a different type from, any of the asset(s) 124. In those or other examples, one or more characteristics (or “asset characterise c(s)”) of any of the asset(s) 122 may be the same as, or a different from, one or more characteristics of any of the asset(s) 124.
[0038] The system 102 may manage one or more databases that collectively store a plurality of asset-health diagnostic files and a plurality of asset identifiers (IDs) respectively associated with a plurality of machine assets (e.g., the assets 122 / 124). The plurality of asset-health diagnostic files, which can include, in some instances, a plurality of machine-health diagnostic files, may manage, and / or be utilized to manage, the asset characteristic(s).
[0039] In some cases, the system can be utilized to manage metadata associated with the asset(s) 122 / 124. The metadata can be managed by the models (e.g., the model(s) 104). The metadata may be associated with the machine-health diagnostic files. In some examples, managing the metadata by the model(s) 104 can include receiving metadata associated with any number of various images, such as the image(s) 116 and / or the image(s) 118. The model(s) 104 can process the metadata and input images to act on the input images to output links between the input images and assets. The model(s) 104 can process the metadata and the images to identify clusters of the input images. Any initially unlinked images in clusters can be linked based on other linked images (e.g., pre-linked / initially linked images, and / or currently linked images) in the corresponding clusters. The initially unlinked images being clustered with the linked images can be utilized to generate an inference result. The inference result can be utilized to link the initially unlinked images to the same assets in the cluster as for the linked images.
[0040] Individual ones of the asset characteristic(s) include, for example, an asset identifier (e.g., ID, a size, a location, a type, a brand), a condition (e.g., a level of cleanliness, upkeep, maintenance, etc.), an amount of wear, a label (e.g., identifier label), location information (e.g., a location (or “asset location”)), geospatial information,, a name of a premises on which the corresponding asset is located, an owner, a lessee, a lessor, an organization (e.g., a company, a business, etc.) of anyty pe with which the corresponding asset is associated, a previous capture (e.g., image) of the corresponding asset, a date of the previous capture, a number of captures (e.g., images) of the corresponding asset previously performed, dates of the captures, a repair schedule, a last service date, etc., or any combination thereof. The label may include, for example, a unique identifying label (e.g., a unique alphanumeric string, serial number, etc., or any combination thereof) on an asset (e.g., any of the asset(s) 122 / 124), such as an electric panel or any other type of asset. In some cases, the identifier on the label (e.g., possibly captured in image(s) 116 / 118) may be utilized by an individual (e.g., a technician, an operator, etc.) to identify, in or out of the field, the asset identifier (e.g., stored in the system 102 or any other system) of the corresponding asset(s) 122 / 124. The asset characteristic(s) can be included in, and / or managed via, the metadata associated with the plurality of machine-health diagnostic files. For example, a plurality of sets of the metadata may be respectively associated with the plurality of machine-health diagnostic files.
[0041] In some cases, at least a subset of the plurality of machine-health diagnostic files include a plurality of machine-health diagnostic images (e.g.. the image(s) 116 / 118). For example, each of the machine-health diagnostic images may include a visible-light image, an acoustic image, or a thermal image. Each of the machine-health diagnostic images may depict one or more of the machine assets (e.g., the assets 122 / 124) and / or an area adjacent to the one or more of the machine assets.
[0042] The model(s) 104 can be utilized to manage (e.g., identify, determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) one or more clusters of the various images, such as the image(s) 116 / 118. In some examples, the cluster(s), which can be managed by the model(s) 104, can include one or more clusters 126 and / or one or more cluster(s) 128. For instance, individual ones of the cluster(s) 126 can correspond to, be generated with, and / or be associated with, the image(s) 116. In such an instance or another instance, a single cluster 126 can include one of the image(s) 116(A) as an image 126(H); and the cluster 126 can include at least one of the image(s) 116(B) as at least one corresponding image 126(1).
[0043] For instance, individual ones of the cluster(s) 128 can correspond to, be generated with, and / or be associated with, the image(s) 118. In such an instance or another instance, a single cluster 128 can include at least one of the image(s) 118(E)as at least one image 128(L); and the cluster 128 can include one of the image(s) 118(F) as an image 128(M). In various cases, any of the image(s) (e.g., the image(s) 116 / 118), and / or any combination thereof, can correspond to, be utilized to generate, be associated with, any of the cluster(s) 126 / 128, and / or any combination thereof.
[0044] In some examples, a respective set of metadata associated with each of the machine-health diagnostic images includes a capture time associated with the machine-health diagnostic image. Clustering the plurality of machine-health diagnostic files may include clustering the plurality of machine-health diagnostic images (e.g., the image(s) 116 / 118) based on the capture times. At least one of each of the plurality of machine-health diagnostic images (e.g., the image(s) 116 / 118) may be clustered into the cluster(s) 126 / 128. for example, based on the capture times.Clustering the plurality of machine-health diagnostic images (e.g., the image(s) 116 / 118) based on the capture times may include clustering the plurality of machinehealth diagnostic images (e.g., the image(s) 116 / 118) based on a parameter (e.g., a user-defined hyperparameter or any other type of parameter) of maximum time difference. For example, the parameter (e.g., the user-defined hyperparameter) of maximum time difference may specify a maximum permissible time difference between the capture times of any two machine-health diagnostic images (e.g., at least one of the image(s) 116 and / or at least one of the image(s) 118) contained in the same cluster (e.g., the cluster 126 and / or the cluster 128, respectively).
[0045] In some examples, clustering the plurality of machine-health diagnostic images may be performed by generating, for each of the machine-health diagnostic files, scores (e.g., multiple distance scores) according to multiple different clustering criteria. The system 102 may determine, for each of the machine-health diagnostic files, a score (e.g., an average distance score) from among the scores (e.g., the multiple distance scores). The system 102 may cluster the plurality of machine-health diagnostic files based on the scores (e.g., the average distance scores). A cluster of the plurality of machine-health diagnostic files can include files with corresponding scores being less than a threshold.
[0046] The system 102 may determine and / or utilize, for any of the machinehealth diagnostic files, any type of score from among the distance scores in a similar way as for the average distance score. In some examples, individual ones of thedistance scores can include corresponding average distance scores (e.g., average scores), corresponding median scores, corresponding minimums, corresponding maximums, corresponding percentiles (e.g., corresponding first quartiles, corresponding second quartiles, corresponding third quartiles, etc.), corresponding scores of other types, or any combination thereof, respectively for the machine-health diagnostic files. The scores can represent a level of similarity between multiple ones (e.g., a pair, a group, etc.,) of the machine-health diagnostic files.
[0047] In some examples, clustering the image(s) 116 / 118 into the clusters ) / 126 / 128 may include generating a plurality of image embeddings respectively for the plurality' of machine-health diagnostic images. The respective image embedding for each machine-health diagnostic image may include a numerical vector that represents visual features contained in the machine-health diagnostic image. The plurality of machine-health diagnostic images may be clustered based on the plurality of image embeddings. For example, any number / combination of the plurality of machine-health diagnostic images may be clustered using a cosine similarity and / or a vector dot product to identify an amount by which any number / combination of the plurality of image embeddings (e.g., vectors) are aligned.
[0048] In some examples, clustering the image(s) 116 / 118 into the clusters ) / 126 / 128 may include generating a plurality' of extracted text strings respectively for the plurality of machine-health diagnostic files (e.g., such as, based on the metadata of the image(s) 116 / 118). The respective extracted text strings for each machine-health diagnostic file may include strings of text extracted from the machine-health diagnostic file. The plurality of machine-health diagnostic images may be clustered based on the plurality of extracted text strings.
[0049] In some cases, clustering the plurality of machine-health diagnostic files based on the plurality of extracted text strings includes receiving a user input (e.g., such as via the window 302, as discussed below with reference to FIG. 3) that specifies a particular text string (e.g., via the search field 308, and / or any other search field). The system 102 can search the extracted text strings for the plurality of machine-health diagnostic files to identify any machine-health diagnostic files that contain the particular text string. The system 102 can insert into one of the plurality ofclusters (e.g., the cluster(s) 126 / 128), any machine-health diagnostic files that were identified as containing the particular text string.
[0050] In various examples, clustering the image(s) 116 / 118 into the cluster(s) / 126 / 128 may include determining a plurality of edit distances values between the plurality of extracted text strings. The plurality of machine-health diagnostic images may be clustered based on the plurality of edit distance values. The edit distance values may be utilized to indicate and / or quantify how dissimilar the extracted text strings (e g., words) are to one another.
[0051] The various types of data can be utilized by the system 102 to manage (e.g., identify, determine, generate, capture, analyze, modify, delete, organize, sort, provide, output, etc.) the cluster(s) 126 and / or 128. Managing the image(s) 116 / 118 and / or the cluster(s) 126 / 128 can include analyzing the image(s) 116 / 118, which can include assigning one or more groups of the image(s) 116 / 118 into corresponding ones of the cluster(s) 126 and / or 128, and / or corresponding combinations thereof.
[0052] For example, managing the image(s) 116 and / or the cluster(s) 126 can include assigning individual groups of the image(s) 116 into a corresponding one of the cluster(s) 126, and / or corresponding combinations thereof. In such an example or another example, managing the image(s) 118 and / or the cluster(s) 128 can include assigning individual groups of the image(s) 118 into a corresponding one of the cluster(s) 128, and / or corresponding combinations thereof. In some instances, the group(s) of the image(s) 116 can be assigned to, as, and / or within, the cluster from among the cluster(s) 126, or the combination thereof. In those or other instances, the group(s) of the image(s) 118 can be assigned to, as, and / or within, the cluster from among the cluster(s) 128, or the combination thereof.
[0053] The timestamp model 106 can be utilized to manage (e.g., identify, determine, generate, capture, modify7, delete, organize, sort, provide, output, etc.) timestamp data, which can include, and / or be identified via, one or more timestamps associated with the image(s) 116 / 118. For example, the timestamp model 106 can be utilized to identify individual ones of the timestamp(s) for a corresponding image from among the image(s) 116 / 118. A timestamp can include a date and / or a time (e.g., hours, minutes, seconds, etc.) that identifies when the corresponding image 116 / 118 was captured, such as by the camera 120.
[0054] Assigning the group(s) of the image(s) 116 / 118 to the corresponding cluster(s), from among the cluster(s) 126 and / or 128, or the corresponding combination thereof, can be performed based on the timestamp data. In some examples, assigning a group of the image(s) 116 to a cluster 126 can be performed based on timestamp data, such as one or more timestamps of corresponding image(s) 116. In alternative or additional examples, assigning a group of the image(s) 118 to a cluster 128 can be performed based on timestamp data, such as one or more timestamps of corresponding image(s) 118.
[0055] The visual model 108 can be utilized to manage (e.g., identify, determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) visual model data, which can include, and / or be identified via, one or more visual model metrics associated with the image(s) 116 / 118. For example, the visual model 108 can be utilized to identify individual ones of the visual model metrics(s) for a corresponding image from among the image(s) 116 / 118. A visual model metric can include, for an image from among the image(s) 116 (e.g., and / or from among the image(s) 118), a visual similarity metric associated with a level of visual similarity between the image and at least one other image from among the image(s) 116 (e.g., and / or from among the image(s) 118).
[0056] Assigning the group(s) of the image(s) 116 / 118 to the corresponding cluster(s), from among the cluster(s) 126 and / or 128, or the corresponding combination thereof, can be performed based on the visual model metric data. In some examples, assigning a group of the image(s) 116 to a cluster 126 can be performed based on visual model metric data, such as one or more visual similarity metrics of corresponding image(s) 116. In alternative or additional examples, assigning a group of the image(s) 118 to a cluster 128 can be performed based on visual model metric data, such as one or more visual similarity metrics of corresponding image(s) 118.
[0057] The text model (e.g., a text matching model) 110 can be utilized to manage (e.g., identify, determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) text model data (e.g., text matching data), which can include, and / or be identified via, one or more text model metrics associated with the image(s) 116 / 118. For example, the text model 110 can be utilized to identify individual ones of the textmodel metrics(s) for an image from among the image(s) 116 / 118. A text model metric can include, for an image from among the image(s) 116 (e.g., and / or from among the image(s) 118), a text similarity metric associated with a level of text similarity between the image and at least one other image from among the image(s) 116 (e.g., and / or from among the image(s) 118). The text similarity metric (or “text matching metric’') (e.g., the level of text similarity) may be utilized to identify how much (e.g.. an amount, a percentage, etc.) of text associated with the image matches text associated with the at least one other image from among the image(s) 116 (e.g., and / or from among the image(s) 118).
[0058] Assigning the group(s) of the image(s) 116 / 118 to the corresponding cluster(s), from among the cluster(s) 126 and / or 128, or the corresponding combination thereof, can be performed based on the text model metric data. In some examples, assigning a group of the image(s) 116 to a cluster 126 can be performed based on text model metric data, such as one or more corresponding text similarity metrics of the image(s) 116. In alternative or additional examples, assigning a group of the image(s) 118 to a cluster 128 can be performed based on text model metric data, such as one or more corresponding text similarity metrics of the image(s) 118.
[0059] The search model (e.g., a test search model) 112 can be utilized to manage (e.g., identify, determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) search model data (e.g.. text search data), which can include, and / or be identified via, one or more search model metrics (e.g., one or more text search metrics, one or more image search metrics (e.g., for one or more image searches), one or more timestamp search metrics (e.g., for one or more timestamp searches), one or more location search metrics (e.g., for one or more location searches), etc., or any combination thereof) associated with the image(s) 116 / 118.
[0060] The search model 112 can identify, using the search model metric(s), one or more search result images (e.g., one or more system images managed via a datastore in the system 102; one or more database images managed via a database, in turn being managed by the system 102, etc., or any combination thereof), and / or one or more identifiers thereof. The search model 112 can identify the search result image(s), such as from a repository of images (e.g., the repository including the image(s) 304, as discussed above with reference to FIG. 3), which may include theimage(s) 116 / 118. The search model 112 can identify the search result image(s) using the search model metric(s). For example, the search model 112 can identify the search result image(s), based on metadata associated with the image(s) 116 / 118. For example, the search model 112 can be utilized to identify individual ones of the search model metrics(s) for a corresponding image from among the image(s) 116 / 118.
[0061] The search model 112 can select a search result image from among a repository of images, based on one or more metrics (e.g., one or more metrics identified in the metadata, such as at least one of text, an image, a timestamp, or a location associated with an image matching the corresponding search model metric(s) of the image(s) 116 / 118. A search model metric can include, for an image from among the image(s) 116 (e.g., and / or from among the image(s) 118), a search match metric (e.g., a text / image / timestamp / location search match metric) associated with a level of similarity between the search model metric(s) associated with the image and the search model metric(s) associated with the search result image, which may be identified via a search query generated by a computing device (e.g.. of a user, operator, etc., of the system 102 and / or of one or more other computing devices communicatively connected to the system 102).
[0062] Assigning the group(s) of the image(s) 116 / 118 to the corresponding cluster(s), from among the cluster(s) 126 and / or 128, or the corresponding combination thereof, can be performed based on the search model metric data. In some examples, assigning a group of the image(s) 116 to a cluster 126 can be performed based on search model data (e.g., text search data), which can include search model metric data, such as one or more corresponding search model metrics of the image(s) 116. In alternative or additional examples, assigning a group of the image(s) 118 to a cluster 128 can be performed based on search model metric data, such as one or more search model metrics of corresponding image(s) 118.
[0063] The location model 114 can be utilized to manage (e.g., identify', determine, generate, capture, modify, delete, organize, sort, provide, output, etc.) location model data, which can include, and / or be identified via, one or more locations associated with the asset(s) 122 / 124 in the image(s) 116 / 118. For example, the location model 114 can be utilized to identify individual ones of the location(s) for a corresponding asset (e.g., any of the asset(s) 122 / 124) in a corresponding imagefrom among the image(s) 116 / 118. A location can include, for an image from among the image(s) 116 (e.g., and / or from among the image(s) 118), a GPS location (e.g., coordinates, such as a latitude and a longitude), a relative location (e.g., a distance and a direction with respect to a landmark, etc.), a vector-based location (e g., a location using point(s), line(s), and / or polygon(s) to represent the location), any other type of location, or any combination thereof.
[0064] Assigning the group(s) of the image(s) 116 / 118 to the corresponding cluster(s), from among the cluster(s) 126 and / or 128, or the corresponding combination thereof, can be performed based on the location model metric data. In some examples, assigning a group of the image(s) 116 to a cluster 126 can be performed based on location model data (e.g., location data), which can include location metric data, such as one or more corresponding location model metrics of the image(s) 116. In alternative or additional examples, assigning a group of the image(s) 118 to a cluster 128 can be performed based on location model metric data, such as one or more location model metrics of corresponding image(s) 118.
[0065] The system 102 can be utilized to generate various types of data (or “linking data’) for purpose of identifying the candidate asset(s) (e.g., from among the asset(s) 122 and / or 124, and / or one or more other assets) with which the cluster(s) 126 / 128 are associated, and, therefore, with which the image(s) 116 / 118 are associated. In some examples, the linking data can include cluster linking data that identifies one or more candidate assets (e.g., from among one or more various ones of the asset(s) 122 and / or 124, and / or one or more other assets) with which individual ones of the cluster(s) 126 / 128 are associated.
[0066] In those or other examples, the linking data, such as the cluster linking data, can be utilized to generate image linking data. The image linking data being identified based on the cluster linking data (e.g., for the cluster(s) 126 / 128) can be utilized to identify which of the image(s) 116 / 118 are within the cluster(s) 126 / 128; and that those identified image(s) 116 / 118 are, therefore, also associated with the identified candidate asset(s) for the cluster(s) 126 / 128. For instance, the image linking data can identify one or more candidate assets (e g., from among one or more various ones of the asset(s) 122 and / or 124, and / or one or more other assets) with which individual ones of the image(s) 116 / 118 are associated. In some examples, theindividual ones of the image(s) 116 / 118 being identified based on the linking data as being associated with the identified candidate asset(s) may be utilized as a representative image (e.g., a token image, a cover image, etc.) of a corresponding one of the cluster(s) 126 / 128.
[0067] The system 102 can be utilized to associate at least one of the plurality of clusters (e.g., the cluster(s) 126 / 128) with one or more of the asset IDs associated with one or more of the plurality of machine assets (e.g., one or more of the asset(s) 122 / 124). For example, the system 102 can be utilized to generate various types of data that links the cluster(s) 126 / 128 (e.g., and therefore any of the image(s) 116 / 118 in the cluster(s) 126 / 128), to one or more candidate assets (e.g., from one or more various ones of the asset(s) 122 and / or 124, and / or one or more other assets).
[0068] Associating the at least one of the plurality of clusters (e.g., the cluster(s) 126 / 128) with the one or more of the asset IDs associated with one or more of the plurality of machine assets can be performed, for example, by generating the data that links the cluster(s) 126 / 128 (e.g., and therefore any of the image(s) 116 / 118 in the cluster(s) 126 / 128), to the candidate assets, which then can be identified as the asset(s) (e.g., the asset(s) 134 / 136). In some cases, the plurality of clusters (e.g., the cluster(s) 126 / 128) can be associated with the asset(s) (e.g., the asset(s) 134 / 136) without identify ing the candidate assets.
[0069] In various examples, the data that links the cluster(s) 126 / 128 (e.g., and therefore any of the image(s) 116 / 118 in the cluster(s) 126 / 128) to the asset(s) can include one or more links 130 and / or one or more links 132. For instance, the data that links the cluster(s) 126 (e.g., and therefore any of the image(s) 116 in the cluster(s) 126) to a candidate asset, which may be identified as the asset 134, can include the link(s) 130. In such an instance or another instance, the data that links the cluster(s) 128 (e g., and therefore any of the image(s) 118 in the cluster(s) 128) to a candidate asset, which may be identified as the asset 136 can include the link(s) 132.
[0070] Various types of one or more link(s) can be identified for the cluster(s) 126 / 128, and, thereby, the image(s) 116 / 118. For example, the link(s) 130 can include a link 130 between the cluster 126 (e.g., and, therefore, the image(s) 126(H) / 126(I)) and one of the asset(s) 122 as an asset 134. In such an example or another example,the link(s) 132 can include a link 132 between the cluster 128 (e.g., and, therefore, the image(s) 128(L) / 126(M)) and one of the asset(s) 124 as an asset 136.
[0071] While the system 102 (e.g.. the model(s) 104 may identify (e.g., link) the cluster(s) 126 / 128 (e.g., and thereby the image(s) 116 / 118) as discussed above in the current disclosure, it is not limited as such. In various implementations, while not required for cluster / image linking, the system 102 (e.g., the model(s) 104) may utilize additional data, such as ground truth data, for the linking. In various instances, the ground tmth data includes a ground truth asset list (e g., possibly including, but not requiring, any portion of the inventoried asset list) with one or more identifiers of corresponding ground truth assets (e.g., from among any of the inventoried assets).
[0072] Various types of data can be utilized to generate the link(s) 130 / 132 for the cluster(s) 126 / 128, and, thereby, for the image(s) 116 / 118. In some examples, the system 102 (e.g., the model(s) 104) can identify, based on the ground truth data (e.g., which may include one or more ground truth assets), at least one of the inventoried assets as at least one of the candidate asset(s). In those or other examples, at least one of the candidate asset(s) identified by the system 102 (e.g., the model(s) 104), based on the ground truth data, may possibly be used to identify any of the candidate asset(s) as a selected asset, such as a selected asset 134 and / or a selected asset 136. Alternatively, the system 102 (e.g., the model(s) 104) can identify', based on the ground truth data, one or more inventoried assets as a selected asset, such as the selected asset 138 and / or the selected asset 140, without previously identifying any candidate assets (e.g., without requiring identification of any of the candidate asset(s)).
[0073] In some examples, the system 102 (e.g., the model(s) 104) can identify the link 130 between the cluster 126 and an asset (e.g., a candidate asset or the asset 134) based on the ground truth asset 138. In those or other examples, the system 102 (e.g., the model(s) 104) can identify' the link 132 between the cluster 128 and an asset (e.g., a candidate asset or the asset 136) based on the ground truth asset 140.
[0074] The ground truth data can include one or more characteristics (e.g., asset characteristics) associated with the ground truth asset(s). Any of the characteristic(s) associated with the ground truth asset(s) can be of the same type as, or different type from, any of the asset characteristic(s), as discussed above.
[0075] The ground truth data can be utilized by the system 102 (e.g., the model(s) 104) to identify one or more scores associated with similarities between the asset characteristic(s) of assets portrayed in the image(s) 116 / 118 and one or more asset characteristics of the ground truth asset(s). For instance, in some cases with at least one of the image(s) 116 / 118 in at least one of the cluster(s) 126 / 128 having data (e.g., an amount, a quality, a clarity, etc., of data, or any combination thereof) that the system(s) (e.g.. the model(s) 104) identifies as being capable of (e.g.. sufficient for) being utilized to link the at least one of the image(s) 116 / 118 to at least one corresponding candidate asset and / or at least one corresponding asset, that data (e.g., the at least one image 116 / 118) may be utilized to link one or more remaining images 116 / 118 in the at least one cluster 126 / 128 to the at least one corresponding candidate asset and / or the at least one corresponding asset, as will be discussed below with reference to FIGS. 2-10, in further detail.
[0076] The candidate asset(s) may be identified (e.g., automatically identified, such as in response to a search query) as possible asset(s) by the system 102. By identifying the candidate asset(s), the system 102 can present the candidate asset(s) (e.g., such as in the candidate asset data 528, 530, and / or 532, as discussed below with reference to FIG. 5) to an operator to enable the operator to select the correct candidate asset as the asset (e.g., such as in the data 704, 706, and / or 708, as discussed below with reference to FIG. 7). However, the disclosure is not limited as such and, in some examples, the candidate asset can be identified by the system 102 as the asset (e.g., possibly without requiring selection via operator input of any candidate asset). For instance, a candidate asset can be identified from among one or more candidate assets by the system 102 as matching the asset portrayed in the images 116 / 118 based on the system 102 identifying a confidence level associated with the candidate asset. In such an instance or another instance, the system 102 can identify the candidate asset as matching the asset portrayed in the images 116 / 118 based on the confidence level being above a threshold (e.g., a confidence threshold).
[0077] For example, based on the representative image (e.g., the token, cover, and / or title image) 116 / 118 being identified in the group of image(s), the cluster 126 / 128 with which the group corresponds may be linked to the corresponding candidate asset. In those or other cases, the system 102 (e.g., the model(s) 104) canutilize, along with the timestamp data, the visual data, the text data, the search data, one or more other types of data, or any combination thereof, the ground truth data to link the remaining image(s) 116 / 118 in the at least one cluster 126 / 128 to the at least one corresponding candidate asset, as will be discussed below with reference to FIGS.2A-11, in further detail.
[0078] While one or more various functions may be performed by a technician, a user, and / or an operator, as discussed above in the current disclosure, it is not limited as such. In some examples, any function that may be performed by any person, such as the technician, the user, and / or the operator, as discussed throughout the current disclosure, can be performed, interchangeably, by one or more technicians, one or more users, one or more operators, one or more administrators, one or more persons of any other type. The terms '‘technician,” '‘user,” '‘operator,” '‘administrator,” etc., as mentioned throughout the current disclosure, are utilized for simplicity and clarity of explanation and are not intended to be limiting in any way.
[0079] While the term “machine-health” is utilized with respect to various features throughout the current disclosure, it is not limited as such. The term “machine-health” is utilized for purposes of simplicity and convenience of explanation and is not intended to be limited thereto. In some examples, the term “machine-health"’ may be interpreted as “health"’ or “asset-health,” with respect to any asset of any type, such as an asset that includes one or more doors, one or more windows, one or more roofs, one or more pieces of equipment (e.g., associated with a building, a building HVAC, etc.), one or more machines, one or more assets of any other type, or any combination thereof. In those or other examples, a file (e.g., an asset-health file) may be interpreted as referring to an asset, or vice versa, as appropriate, with respect to any of the techniques discussed herein.
[0080] FIGS. 2A-2E illustrate block diagrams of components of an example system utilized for automated clustering-based image-to-asset linking in accordance with embodiments of the present disclosure. In some examples, any portion (e.g.. a partial or an entire portion) of the system, which can be implemented as one or more systems utilized to implement any portion (e.g., a partial or an entire portion) of the system 102, as discussed above with reference to FIG. 1, and / or any portion (e.g., a partial or an entire portion) of one or more other systems and / or computing devicesintegrated within, or separate from, the system 102 (e.g., the other system(s) and / or computing devices being communicatively coupled to the system 102).
[0081] Referring to FIG. 2A, the components include a timestamp-based clustering component 202. In some examples, the timestamp-based clustering component 202 can be utilized to implement the timestamp model 106, as discussed above with reference to FIG. 1. The timestamp-based clustering component 202 can be utilized to manage timestamp data 204. visualization data 206. redundancy data 208, and / or one or more other types of timestamp data. In some examples, the timestamp data 204 can be implemented utilizing, and / or can include any of the data of, the timestamp data, as discussed above with reference to FIG. 1.
[0082] The timestamp data 204 can be utilized to identify whether to group the image(s) 116 / 118 into group(s) as the cluster(s) 126 / 128, respectively. The timestamp data 204 can be utilized to group together, into one or more sets (e.g., the set(s) 404 as discussed below7with reference to FIG. 4) associated with corresponding group(s) of the image(s) 116 / 118, atleast one of one or more shapes (e.g., the geometric shape(s), as discussed below w ith reference to FIG. 4). The timestamp data 204 can be utilized along with a user-defined hyperparameter of maximum time difference to generate the cluster(s) 126 / 128 and / or the set(s) 404. The hyperparameter may specify a maximum permissible time difference between the capture times of any tw o machine-health diagnostic images contained in the same cluster (e.g., the cluster 126 or 128) (e.g., and / or the same set 404).
[0083] The visualization data 206 can include, and / or be associated with, data output by the system(s). For example, the data included in the visualization data 206, and / or with which the visualization data 206 is associated, can be output by, one or more UIs (e.g., the LU 300 and / or the U1400, as discussed below with reference to FIGS. 3 and 4, respectively) of the system(s).
[0084] The data output by the UI(s) can be utilized by an operator to identify images, and clusters of images utilized, for image-to-asset linking. For example, the operator can utilize the UI 300 to identify at least one image from among one or more images (e.g., the image(s) 116, as discussed above with reference to FIG.1) output by the UI 300. In some instances, the operator can select the at least one image and request the system(s) to include the at least one image in a cluster (e.g., the cluster126, as discussed above with reference to FIG. 1). In those or other instances, the operator can request the system(s) to designate the at least one image as being included in the cluster 126. In those or other instances, the operator can request the system(s) to designate a group of images that includes the at least one image as the cluster 126.
[0085] In some examples, the operator can utilize the data output by the UI 400 to identify at least one geometric shape (also referred to herein simply as "‘shape”) from among one or more geometric shapes (e.g., the geometric shape(s) 404, as discussed below with reference to FIGS. 4), corresponding to, representative of, and / or associated with, at least one corresponding image from among one or more images (e.g., the image(s) 118. etc., as discussed above with reference to FIG.l). In various examples, the data output by the UI 400 can be utilized by the operator to select the at least one shape, such as in a group of the geometric shape(s). The operator can select the at least one shape in the group to identify', for instance, at least one corresponding image from among the image(s) 118 (e.g., with which the at least one shape is associated) as a cluster (e.g.. the cluster 128, etc., as discussed above with reference to FIG. 1). Any number of one or more group(s), which can include the group, can be identified in a similar way as the group.
[0086] The redundancy data 208 can identify' whether the cluster(s) 126 / 128 include any number of the image(s) 116 / 118 that are redundant. For example, the system(s) can analyze one or more time differences between any pairs of one or more consecutive (e.g., sequential) images (e.g., one or more images captured one after another by the camera 120, as discussed above with reference to FIG.l, such as without any intervening image(s) being captured by the camera 120 in between the consecutive images). The redundancy data 208 may identify at least one of the consecutive image(s) as redundant based on any of the time difference(s) between the any tw o of the consecutive image(s) being less than a time difference threshold.
[0087] The redundancy data 208 can be utilized in various ways to identify the redundant image(s). For example, the redundancy data 208 can include, and / or be utilized along with, any other data (e.g., any data in the component(s) 202, 210, 218, 232 and / or 240, as discussed above and / or below), to identify' whether any of the image(s) 116 / 118 (e.g., the consecutive image(s) and / or one or more non-consecutiveimages) are redundant or non-redundant. Individual ones of the image(s) 116 / 118 identified by the system(s) using the redundancy data 208 as being redundant can be archived, deleted, moved, separated, flagged, etc., or any combination thereof.
[0088] The system(s) can utilize one or more UIs to perform one or more actions based on the redundancy data 208. In some examples, the system(s) can output, via the UI(s). the image(s) 116 / 118, including the redundant image(s). In those or other examples, the system(s) output, via the UI(s), one or more flags associated with (e.g., and not being set for) the non-redundant image(s) and / or one or more other flags associated with (e.g., and being set for) the redundant image(s). In those or other examples, the system(s) can output, via the UI(s), one or more indicators (e.g., one or more colors, brightness levels, etc.) associated with the non-redundant image(s), and / or one or more other indicators (e.g., one or more different colors, different brightness levels, etc.) associated with the redundant image(s). In those or other examples, the system(s) can output, via the UI(s), the non-redundant images with formatting (e.g., size, shape, orientation, steadiness, blinking rate, etc.), and / or the redundant images with different formatting.
[0089] While various types of data can be utilized for timestamp-based clustering, such as by utilizing the timestamp-based cluster component 202, as discussed above in the current disclosure, it is not limited as such. In some examples, timestamp-based clustering, such as by the timestamp-based cluster component 202, can be performed without various types of data, such as the redundancy data 208. In those or other examples, for instance, identifying and / or utilizing the redundancy data 208 may be unnecessary with respect to performing timestamp-based clustering (e.g., timestampbased clustering can be performed without utilizing various types of data, such as the redundancy data 208).
[0090] Referring to FIG. 2B, the components include a visual-based clustering component 210. In some examples, the visual -based clustering component 210 can be utilized to implement the visual model 108, as discussed above with reference to FIG.1. The visual-based clustering component 210 can be utilized to manage visual model data 212, embeddings data 214, similarity data 216, and / or one or more other types of visual data. In some examples, the visual model data 212 can be implementedutilizing, and / or can include any of the data of, the visual model data, as discussed above with reference to FIG. 1.
[0091] The visual model data can include any data associated with one or more characteristics (or ‘'window character! stic(s)” or “UI characterise c(s)”) (e.g., in the window 402 of the UI 400, as discussed below with reference to FIG. 4). For example, the visual model data (e.g., the window characteristic(s)) can include one or more locations of the geometric shape(s) (e.g., in the window 402 of the UI 400, as discussed below with reference to FIG. 4), one or more distances between any number of the geometric shape(s), any other types of window characteristic(s), or any combination thereof.
[0092] The visual model data can be utilized to identify the group(s) of the geometric shape(s), and to identify, based on the group(s), at least one of the image(s) 118 as the cluster(s) 128, in various ways. In some examples, the visual model data utilize one or more characteristics (or “algorithm characteristic(s)”) of any type via one or more algorithms of any type, such as of the similarity algorithm(s), discussed below, one or more clustering algorithms, etc., or any combination thereof. In those or other examples, the clustering algorithm(s) (e.g., utilized to identity7the group(s)) can include one or more K-means clustering algorithm characteristics via one or more K-means clustering algorithms, one or more hierarchical cluster algorithm characteristics via one or more hierarchical clustering algorithms, and / or any other one or more types of clustering algorithm characteristics). Individual ones of any of the different types of algorithm(s) (e.g., the similarity7algorithm(s), the clustering algorithm(s), etc.) can utilize any data from the component(s) 202, 210, 218, 232, 240, and / or any other component(s) of the system(s). Individual ones of any of the different types of algorithm(s) (e.g., the similanty algorithm(s), the clustering algorithm(s), etc.) can output data utilized by the component(s) 202, 210, 218, 232, 240, and / or any other component(s) of the system(s).
[0093] The embeddings data 214 can include one or more embeddings (or “image embedding(s)”) of. and / or associated with, any of the image(s) 116 / 118, and / or data associated with the embedding(s). In some examples, clustering of the image(s) 116 / 118 can be performed utilizing the image embedding(s). For example, the system(s) can generate one or more numerical vectors representing one or more visualfeatures of individual ones of the image(s) 116 / 118. By analyzing the embedding(s), the system(s) can cluster the image(s) 116 / 118 that are visually similar.
[0094] The similarity data 216 can include data identified as reflecting at least one of the image(s) 116 / 118 being similar, based on the visual model data 212 and / or the embeddings data 214. In some examples, the similarity data 216 can include data output from the analyzing of the embedding(s). Utilizing the embedding(s) to cluster the image(s) 116 / 118 may be useful when dealing with large volumes of diagnostic images where visual inspection according to existing technology would otherwise be relatively more time-consuming and prone to errors. In some instances, image similarity may be performed by computing the image embedding(s) and then finding one or more cosine similarities between combinations of the image(s) 116 / 118.
[0095] In some examples, the similarity data can be identified utilizing one or more similarity' algorithms. For instance, the similarity algorithm(s) can be run on one or more visual light images (e.g., directly extracted from a thermal / acoustic camera, such as the camera 120) from among the image(s) 118, to identify similar images. At least one of the image(s) 116 and / or at least one of the image(s) 118 can be clustered based on any corresponding data identified by the visual-based clustering component 210.
[0096] Referring to FIG. 2C. the components include a text-based clustering component 218. In some examples, the text-based clustering component 218 can be utilized to implement the text model 110, as discussed above with reference to FIG. 1. The text-based clustering component 218 can be utilized to manage file data 220, extraction data 222, which can include optical character recognition (OCR) data 224, annotation data 226, and / or audio data 228; and / or to manage similarity data 230, and / or one or more other types of text data.
[0097] The file data 220 can include one or more identifiers of one or more files associated with one or more images. For example, the image(s) with which the file(s) are associated may include the image(s) 116 / 118, as discussed above with reference to FIG. 1. and / or one or more other images captured by one or more cameras, such as the camera 120 and / or one or more other cameras.
[0098] The extraction data 222 may include various types of data extracted from the image(s) with which the file(s) are associated. For example, the system(s) mayextract and output the data from the image(s) (e.g. , the image(s) 116 / 118) as the extraction data 222. The extraction data 222 can include any type of data, such as metadata (e.g., which may also include data provided by the user, and / or various other types of data).
[0099] The OCR data 224 can include data based on one or more character recognition algorithms, which can include OCR algorithms, one or more other types of character recognition algorithms, or any combination thereof. The OCR data 224 may include data (e.g., text, text strings, etc.) that is associated with text identified via the OCR algorithms in the image(s) 116 / 118. The OCR data 224 may be extracted by, output via analysis of, and / or associated with, individual ones of the image(s) 116 / 118.
[0100] While the system(s) can utilize to OCR data 224 to process the image(s) 116 / 118, as discussed above in the current disclosure, it is not limited as such. In some examples, various other types of data such as QR codes, barcodes, any other types of codes / data, or any other combination thereof, can be utilized in a similar way as the OCR data 224. The data (e.g.. the OCR data 224. the QR codes, barcodes, etc.), which may be captured in the image(s) 116 / 118, and / or scanned and stored as metadata (e.g., text metadata) directly by the camera(s) (e.g., the camera 120), can be utilized (e.g., by the model(s) 104) to process the image(s) 116 / 118.
[0101] The annotation data 226 may include one or more annotations associated with the image(s) with which the fde(s) are associated. The annotation(s) may be included in metadata identified for individual ones of the image(s) 116 / 118, such as by one or more machine learned models (e.g., one or more machine learned models of the model(s) 104). The annotation data 226 may include data that is associated with the annotation(s). The annotation data 226 may be extracted by, output via analysis of, and / or associated with, individual ones of the image(s) 116 / 118. The annotation data 226 may be input via the user (e.g., the operator) when the image(s) 116 / 118 are captured, e.g. via a virtual keyboard implemented in a camera UI (e.g., a UI of the camera 120). Alternatively or additionally, one or more selected tags and / or one or more selected labels may be generated by the camera UI. The tag(s) / label(s) may be generated based on individual ones of the image(s) 116 / 118 being tagged (e.g.,identified as a tagged image). The tag(s) / label(s) may include text metadata associated with the tagged image(s).
[0102] The audio data 228 may include one or more types of audio data associated with the image(s) 116 / 118 and utilized to identify text(s) associated with individual ones of the image(s) 116 / 118. One or more audio files may be created to include the audio data.
[0103] The audio data 228 may include one or more timestamps (or "audio timestamps”) in audio timestamp data. The audio timestamp(s) may represent audio data (e.g., an audio clip / snippet / note, etc.) created by an operator by using a device (e.g., the camera 120) to record and store a short audio clip with any of the image(s) 116 / 118. The audio note can include a recording of the operator identifying (e.g.. speaking) an asset with which the corresponding image 116 / 118 is associated. The audio note may include probl ems / issues, etc., identified by the operator. Speech-to-text (STT) models can be utilized to turn the audio clip(s) into text, which can be used for image linking (e.g.. linking the image(s) 116 / 118, via the cluster(s) 126 / 128, to the asset(s) 134 / 136). The audio timestamp(s) may be created for corresponding groups of data from among the audio data 228, and / or for corresponding files from among the audio files. For example, the audio data may be captured by one or more devices, such as the camera 120 and / or one or more other device(s). Any of the data (e.g., data from the component(s) 202, 210, 218, 232, and / or 240). such as the timestamp data 204 and the audio timestamp data, may be utilized to match the group(s) of audio data, the audio file(s), and / or the text associated therewith, with the image(s) 116 / 118.
[0104] The audio data 228 may be processed in various ways. For example, the system(s) can process the audio data 228 with the speech-to-text models to generate text data associated with the audio data 228. The audio data 228 and / or the text data can be utilized (e.g., by the model(s) 104) to process the image(s) 116 / 118. The audio data 228, and / or the corresponding text data, can be stored as audio files, and / or text files. Audio data and / or text data (e.g., text) can be extracted from the files and used like other text (e.g., the text model data, as discussed above with reference to the text model 110) to identify the asset associated with the image 116 / 118 (e.g., and / or other images with similar text, such as any images able to be added into a same cluster). The audio / text files can be automatically tagged / tied (e.g., by the camera 120) to theimage(s) 116 / 118 with which the audio / text files are associated. The audio / text files can be used for the image linking.
[0105] The similarity data 230 may be utilized to identify the groups of the image(s) 116 / 118 as the cluster(s) 126 / 128, in a similar way as the data of the component(s) 202 and / or 210. For example, the similarity data 230 may be utilized to identify the groups of the image(s) 116 / 118 as the cluster(s) 126 / 128 based on individual ones of one or more scores associated with a likelihood of the audio file, the text file, etc. , being associated with any of the image(s) 116 / 118 (e g. , the image to which the audio file / text file is tagged / tied, and / or any other image identified / predicted as being associated with a same candidate asset) being greater than a score threshold.
[0106] In some instances, individual ones of the score(s) may be generated based on a level of similarity between any combination of the text associated with the similarity data 230, text in corresponding data of the OCR data 224, text in corresponding data of the annotation data 226, etc. In some examples, the similarity’ data 230 may be utilized to identify whether there is a match, and / or a level of similarity’ above a threshold, between the text identified based on the OCR data 224, the text identified by the annotation data 226, the text identified by the audio data 228, etc.
[0107] The similarity’ data 230 may be generated based on one or more comparisons between obtained text in any of the different ty pes of extraction data 222 (e.g., the OCR data 224, the annotation data 226, the audio data 228, etc.), and other obtained text in any of the different types of extraction data 222. The similarity data 230 may be generated using, for a comparison, a string similarity metric such as a Levenshtein distance, to identify whether there is a match via a comparison between texts associated with a single image 116 / 118, or via a comparison between texts associated with different ones of the image(s) 116 / 118. In some examples, the string similarity metric such as the Levenshtein distance, can be utilized to identify, based on the comparison(s), whether there is a level of similarity between the texts being compared that is greater than a similarity threshold. At least one of the image(s) 116 and / or at least one of the image(s) 118 can be clustered based on any corresponding data identified by the text-based clustering component 218.
[0108] Referring to FIG. 2D, the components include a search-based clustering component 232. In some examples, the search-based clustering component 232 can be utilized to implement the search model 112, as discussed above with reference to FIG.1. The search-based clustering component 232 can be utilized to manage character data 234, query data 236, target data 238, and / or one or more other ty pes of search data.
[0109] The character data 234 can include one or more characters, such as in one or more corresponding strings utilized to perform one or more searches associated with the image(s) 116 / 118. The string(s) can include one or more alphanumeric search strings. In some examples, the character data 234 can be input to the system(s) as one or more selections received from a user (e.g., an operator).
[0110] The query' data 236 can include one or more queues. Individual ones of the query(ies) may' be associated with the image(s) 116 / 118 and / or with one or more ground truth images. The query(ies) can be managed (e.g., identified, determined, generated, output, updated, canceled, etc.) based on the selection(s) received from the user as the input to the system(s). In some examples, the query(ies), and / or any data utilized by any of the component(s) 202, 210, 218, 232, and / or 240 can be utilized to identify the image(s) 116 / 118 and / or perform image automated clustering-based image-to-asset linking.
[0111] Text associated with the query(ies) can be identified in, and / or based on, the query(ies). The text can be utilized to perform one or more comparisons with other text (e.g., text associated with the image(s) 116 / 118 and identified by the component(s) 202, 210, 218, 232, and / or 240). The comparison(s) can be performed in a similar way as any of the other comparison(s) utilizing text, as discussed above.
[0112] The system(s) can utilize the comparison(s) to group at least one of the image(s) 116 / 118 into the cluster(s) 126 / 128. The at least one of the image(s) 116 / 118 can be grouped into the cluster(s) 126 / 128 utilizing the comparisons with the text in the query(ies) in a similar way as discussed above for utilizing the component(s) 202, 210, and / or 218 to group the at least one of the image(s) 116 / 118 into the cluster(s) 126 / 128.
[0113] Referring to FIG. 2E, the components include a location-based clustering component 240. In some examples, the location-based clustering component 240 canbe utilized to implement the location model 114, as discussed above with reference to FIG. 1. The location-based clustering component 240 can be utilized to manage location data, such as geo-location data 242. The geo-location data 242 can include any of the location model data (e.g., the GPS location, the relative location, the vector-based location, any other type of location, or any combination thereof). At least one of the image(s) 116 and / or at least one of the image(s) 118 can be clustered based on corresponding location data. For example, one or more locations associated with assets being separated by less than a distance threshold can be utilized to group images associated with the assets. In such an example or another example, at least two images associated with the assets can be grouped based on at least two locations from among the plurality of locations being separated from one another by less than a distance threshold. For instance, grouping the at least two images can be utilized to identity7, in some cases, that the at least two images are associated with the same asset (e.g., based on the separation being less than the threshold).
[0114] While the system(s) can utilize various components, such as the timestamp-based clustering component 202. the visual-based clustering component 210, the text-based clustering component 218, the search-based clustering component 232, and / or the location-based clustering component 240 to process the image(s) 116 / 118 and cluster at least one of the image(s) 116 / 118, as discussed above in the current disclosure, it is not limited thereto. Any of the components 202. 210, 218, 232, and / or 240 can be utilized individually, or in combination with any of the others, to process the image(s) 116 / 118 and cluster at least one of the image(s) 116 / 118.
[0115] While various terms, such as “group(s),’' “cluster(s),” etc., are discussed with respect to various feature(s), characteristic(s), parameter(s), etc., (e.g., associated with various component(s), system(s), and / or functionality thereof), as discussed throughout the current disclosure, it is not limited thereto. Such terms are utilized merely for simplicity7, clarity7, and / or ease of explanation. In some examples, any ty pes of terms, such as “group(s),’' “cluster(s),” “set(s),'’ "subset(s)." ‘‘section(s),'’ ■'subsection(s).” etc., or any combination thereof, as pertaining to any features, may be interchangeable, and / or relevant with respect to any feature(s), characteristic(s), parameter(s), etc., as applicable, for purposes of performing any of the techniques, as discussed herein.
[0116] FIG. 3 is an example UI 300 utilized to identify images, and clusters of images utilized, for image-to-asset linking in accordance with embodiments of the present disclosure. In some examples, the UI 300 can be implemented utilizing the system(s), as discussed above with reference to FIGS. 2A-2E.
[0117] The UI 300 can include a window 302 utilized to output, as one or more images 304, one or more of the image(s) 116, as discussed above with reference to FIG. 1. The window 302 can output, from a repository of images stored in the system(s), the image(s) 304. The window 302 can graphically depict, based on one or more files (e.g., one or more machine-health diagnostic files) (e.g., one or more of the plurality' of machine-health diagnostic files, as discussed above with reference to FIG.1), the image(s) 304. In some examples, the system(s) can identify (e.g., select) one or more of the image(s) 116 to be output via the UI 300 as the image(s) 304. In those or other examples, one or more selections received from a user (e.g., an operator) as input to the system(s) can be utilized to identify one or more of the image(s) 116 to be output via the UI 300 as the image(s) 304. The selected image(s) 116 can be identified as image(s) 304.
[0118] The image(s) 116 can be identified and output by the UI 300 as the image(s) 304 based on various ty pes of input. The input may be received from the user and by a select all field (or “select all”) 306, a search field 308, a date field 310, a text field 312, a submit field (or “submit”) 314, a clustering graph field (or “clustering graph”) 316, and / or one or more other fields. In some examples, individual ones of the fields can include a button, a check box, a toggle switch, a text entry box, and / or any other ty pe of UI field and / or UI entry' tool. Input to the field(s) 306-316 can be utilized by the system(s) to search the repository of images for the image(s) 304.
[0119] The image(s) 116 can be identified and output by the UI 300 as the image(s) 304 based on input to the search field 308. In some examples, text input to the search field 308 may be based on input selecting a drop down arrow, then an item in a menu (e.g., a drop down menu) in a drop down box, and / or then text in the search field 308.
[0120] In some examples, the drop down menu can include a number of clustering criteria. User input can be received, such as based on the drop down menu being output. The user input can include one or more selections, such as by the user inputselecting at least one of the clustering criteria. For instance, based on the user input that selects a clustering criteria, the clustering criteria can be performed. The clustering criteria can include one or more identifiers, one or more cameras, one or more transcripts, one or more comments, one or more upload tags, one or more annotations, and / or one or more other types of clustering criteria. Text input into the search field 308 can correspond to, and / or be based on the selection of the search criteria via the drop down arrow and the drop dow n menu.
[0121] As an example, the text input to the search field 308 may include text input by the user to search for an identifier. Alternatively or in addition, the text input by the user to search for a camera (e.g., the camera 120, utilized to capture the image(s) 304). Alternatively or in addition, the text input to the search field 308 may include text input by the user to search for a transcript (e.g., generated by a user and / or automatically generated by the system(s), identifying details, such as a date, a time, a location, etc. associated with the image(s) 304 being captured). In some cases, content (e.g., text) of the transcript can be identified by performing an operation (e.g., an OCR operation, a speech-to-text (STT) operation) with data (e.g.. in the camera 120. and / or a device communicatively coupled thereto and / or associated therewith) corresponding to any of the image(s) 304. Alternatively or in addition, the text input to the search field 308 may include text input by the user to search for a comment (e.g., generated by a user and / or automatically generated by the system(s), identifying details, such as to identify one or more other images which individual ones of the image(s) 304 being captured may be associated).
[0122] Alternatively or in addition, the text input to the search field 308 may include text input by the user to search for an upload tag (e.g., automatically generated by the camera 120, and including details identifying one or more image characteristics associated with the image(s) 304, etc ). Alternatively or in addition the upload tag can be provided by the operator at the time they upload one or more image files (e.g., one or more files corresponding to the image(s) 116 / 118) into the system 102. Alternatively or in addition, the text input to the search field 308 may include text input by the user to search for an annotation (e g., metadata generated by the camera 120, and / or the system(s), and associated with the image(s) 304). The image(s) 116 output by the UI 300 as the image(s) 304 may be based on a scoreassociated with a similarity between the text in the search field 308 and other text in the corresponding data being searched, such as an identifier generated by the camera 120 and / or the system(s), an identifier of the camera 120 stored in the camera 120 and / or the system(s), a transcript generated by a user and / or automatically generated by the system(s), a comment generated by a user and / or automatically generated by the system(s), an upload tag automatically generated by the camera 120 and / or the system(s), and / or an annotation.
[0123] The image(s) 116 can be identified and output by the UI 300 as the image(s) 304 based on input to the date field 310. In some examples, input to the date field 310 (e.g., which may be implemented as a single field, or one or more date fields), may include a start date and / or an end date; and / or a start time and / or an end time (e.g., in the date field 310 or a separate time field). The image(s) 116 output by the UI 300 as the image(s) 304 may be based on a date at which the image(s) 116 were captured being between the start date and the end date.
[0124] The image(s) 116 can be identified and output by the UI 300 as the image(s) 304 based on input to the text field 312. In some examples, text input to the text field 312 may include text processed in a similar way as any of the text processed by the search-based clustering component 232. For instance, text input to the text field 312 may be utilized to scan other text in one or more file names, one or more annotations, one or more OCR-identified text strings, etc., and / or any other data associated with the component(s) 202, 210, 218, 232, and / or 240. The image(s) 116 output by the UI 300 as the image(s) 304 may be based on a score associated with a similarity between the text in the text field 312 and the other text being greater than a score threshold.
[0125] In some examples, identifying and / or output of the image(s) 304 can include receiving input via one or more user selections to the window 302, such as via a selection by the user to the submit field 314. In some examples, the image(s) 304 can be identified from among the image(s) 116 and output by the window 302 in response to receiving the submit selection (e.g., to the submit field 316). In those or other examples, the system(s) can determine to output the UI 400, as discussed below with reference to FIG. 4, such as by switching between output of the UI 300 to output of the UI 400. For instance, the system(s) can determine to output the UI 400 in31response to receiving input via the user selection(s) to the window 302, such as via a selection by the user to the clustering graph field 316.
[0126] In some examples, the system(s) can be triggered to output the U1300 in response to receiving the image(s) 116 / 118 via a transfer operation (e.g., a download operation, an upload operation, any other type of operation, or any combination thereof) performed with the camera 120. For instance, the system(s), such as a webapplication in the cloud, can be triggered to output the UI 300 based on the transfer operation, as a default operation (e.g., by any device such as the camera 120, and / or any computing device / PC communicatively coupled thereto). Alternatively, the system(s) can be triggered to output the UI 400 based on the transfer operation, as the default operation. Whether the default operation includes outputting the UI 300 or the UI 400 can possibly be set in one or more settings of the system(s).
[0127] The UI 300 may output the image(s) 304 in various orders and / or UI formats, and / or with at least one of various parameters for the image characteristic(s) as discussed above with reference to FIGS. 1-2E. For example, the window 302 may output the image(s) (or "candidate image(s)’?) 304 in one or more initial candidate groups. The candidate group(s) may be identified in a similar way, via the component(s) 202, 210, 218, 232, and / or 240, as for the group(s), as discussed above with reference to FIGS. 2A-2E, respectively. Such output techniques may enable operators to more easily identify, from among the image(s) 304 and / or the candidate group(s), individual ones of the cluster(s) 126 as possibly being associated with a same corresponding candidate asset.
[0128] In some examples, outputting the image(s) 304 in the candidate group(s) can include outputting the image(s) 304 in an order (e.g., in order of capture times of when the image(s) 304 were captured) and / or with a UI format (e.g., in rows and columns). In those or other examples, outputting the image(s) 304 in the candidate group(s) can include outputting any of the image(s) 304, one after the other, from left to right, and top to bottom, similar to text on a page (e.g., in a book). For instance, a sequence (e.g., order) of the image(s) 304 may include at least one of the image(s) 304 in a candidate group being output contiguously (e.g., sequentially), in order one after the other. In such an instance or another instance, the at least one of the image(s)304 in the candidate group may be output in order (e.g., to be grouped together), prior to at least one other of the image(s) 304 in another candidate group.
[0129] Alternatively or additionally, outputting the image(s) 304 in the candidate group(s) can include outputting any of the image(s) 304 within a candidate group in areas forming into geographic shapes of any types. For instance, a first candidate group of the image(s) 304 can be output in a first square area of the window 302 (e.g., the first square area being outlined, formatted, visually identified, and / or not being outlined, formatted, visually identified); a second candidate group of the image(s) 304 can be output in a second square area of the window 302 (e.g., the second square area being outlined, formatted, visually identified, and / or not being outlined, formatted, visually identified), and so on.
[0130] Any of the image characteristic(s) (e.g., sizes, shapes, formats, etc.) of any of any candidate group of the image(s) 116 may be the same to easily distinguish the candidate group from at least one other candidate group. In some cases, any of the image characteristic(s) (e.g., sizes, shapes, formats, etc.) utilized to output any of any candidate group of the image(s) 304 may indicate to a user a relative likelihood identified by the system(s) that the candidate group is associated with a same candidate asset. In those or other cases, any of the UI formats utilized to output any of any candidate group of the image(s) 304 may indicate to a user a relative likelihood identified by the system(s) that the candidate group is associated with a same candidate asset.
[0131] The image(s) 304 may be output via the window 302 in various ways, such as be the system(s) utilizing output of at least one of the component(s) 202, 210218, 232, and / or 240. For instance, the image(s) 304 may be output excluding the visual model data 212 (e.g., without utilizing the UI 400. as discussed below in further detail.
[0132] Individual ones of the candidate group(s) of the image(s) 304 can be identified as a group (e.g., the group the image(s) 126(H) / ! 24(1), as discussed above with reference to FIG. 1). For example, at least one of the image(s) 304 in the candidate group(s) can be selected via user input (e.g., a mouse click selecting a corresponding check box). The group of the selected at least one image 304 (e.g., theimage(s) 126(H) / 126(1)) can be identified (e.g., defined) as a cluster (e.g., the cluster 126, as discussed above with reference to FIG. 1).
[0133] The image(s) 126(H) / 126(1) can be identified and grouped into the cluster 126 by utilizing the window 302 with at least one of output by the component(s) 202, 210, 218, 232, and / or 240. In some examples, the image(s) 126(H) / 126(I) can be identified and grouped into the cluster 126, by utilizing the window 302 with less than all of the component(s) 202, 210, 218. 232, and / or 240. The image(s) 126(H) / 126(I) being identified without the data 212 / 214 / 216 may be based on the image(s) 304 being output in the window 302, and utilized to identify, such as via the user input, the at least one of the image(s) 304 as the image(s) 126(H) / 126(1).
[0134] In some examples, the candidate group(s) of the image(s) 304 can be identified as corresponding group(s) of the image(s) in a cluster (e.g., a cluster 126, as discussed above with reference to FIG. 1) based on user input. For instance, a candidate group of at least one of the image(s) 304 can be identified as a corresponding group of at least one corresponding image in the cluster 126 in response to receiving input by the user via at least one corresponding selection of a check box of the at least one corresponding image 304 (e.g., selecting the at least one corresponding image 304 as the at least one corresponding image grouped in the cluster 126).
[0135] Individual ones of the candidate group(s) of the image(s) 304 can be identified as a group (e.g., the group of image(s) 126(H) / ! 26(1), as discussed above with reference to FIG. 1). For example, at least one of the image(s) 304 in the candidate group(s) can be selected via user input (e.g., a mouse click selecting a corresponding check box). The group of the selected at least one image 304 (e.g.. the image(s) 126(H) / 126(1)) can be identified (e.g., defined) as a cluster (e.g., the cluster 126).
[0136] While the image(s) 304 can be shown and / or arranged (e.g., for presentation) in rows and / or columns (e.g., in a uniform arrangement, such as in a grid), as depicted in FIG. 3, and / or as discussed above in the current disclosure, it is not limited thereto. In some examples, any number / combination of the image(s) 304 can be shown, arranged, presented, etc., according to any of various spatial arrangements. For instance, any number / combination of the image(s) (e.g.,thumbnail(s)) 304 can be shown, arranged, presented, etc. in any of one or more clusters (e.g.. based on corresponding result(s) of any of the component(s) 202. 210, 218, 232, and / or 240), in any other format, or any combination thereof.
[0137] FIG. 4 is an example UI 400 utilized to output images utilized for image clustering, and image-to-asset linking, in accordance with embodiments of the present disclosure. In some examples, the UI 400 can be implemented utilizing the system(s), as discussed above with reference to FIGS. 2A-2E.
[0138] The UI 400 can include a window 402 utilized to output one or more sets 404 of one or more geometric shapes. In some examples, individual ones of the geometric shape(s) in the set(s) 404 can include a dot, a circle, a square, etc., and / or any other geometric shape. In those or other examples, individual ones of geometric shape(s) 404 may have one or more characteristics (or “shape characteristicis)") (e.g., outlined, enclosed, patterned, colored, etc., or any combination thereof). The shape characteristic(s), for example, may identify, correspond to, and / or be associated with, individual ones of one or more image characteristic(s) (e.g.. the image characteristic(s), as discussed above with reference to FIG. 1) associated with one or more images (e.g., the image(s) 116 / 118, as discussed above with reference to FIG.1).
[0139] In some examples, individual ones of the set(s) 404 have one or more characteristics (or “set characteristic(s)”) (e.g., outlined, enclosed, patterned, colored, etc., or any combination thereof). The set characteristic(s), for example, may identify, correspond to, and / or be associated with, individual ones of one or more image group characteristic(s) (“group characteristic(s)”). The set characteristic(s), can be the same as, or different from, any of the image characteristic(s), as discussed above with reference to FIG. 1.
[0140] In some examples, the window 402 can graphically depict a plurality of clusters of fdes (e.g., a plurality of clusters of machine-health diagnostic fdes), such as utilizing a plurality' of sets (also referred to herein as “clusters’") 404 to represent the clusters. For example, a cluster of images may be depicted via the set(s) 404. Associating the at least one of the plurality' of clusters with one or more asset IDs (e.g., the one or more of the asset IDs, as discussed above with reference to FIG. 1) associated with the one or more of the plurality' of machine assets includes providing agraphical user interface (e.g., one or more of the windows 302, 502, 602, and / or 702, as discussed above or below with reference to FIGS. 3 and 5-7) that enables the system(s) to identify machine assets (e.g., as candidate assets and / or selected assets), such as based on one or more user searches with one or more search strings. The system(s) can receive, via the window(s) 302 / 502 / 602 / 702, one or more user inputs that define one or more search strings. The system(s) can identify the one or more asset IDs based on the one or more search strings. The system(s) can associate the at least one of the plurality of clusters with the one or more of the asset IDs identified using the one or more search strings (e.g., such as by associating a cluster of image(s) 504 (e.g., a cluster of image(s) 504 and 506), as discussed in further detail below with reference to FIG. 5).
[0141] For example, individual ones of the set characteristic(s) can include a dimension (e.g., a height, a width, a depth), a size, a resolution, a scale, one or more colors, a tone, a contrast, a nature, a position, detail, noise, distortion, etc., or any combination thereof, which can correspond to a group of the image(s) 116, reflected by the shape(s) in a corresponding set 404. In some instances, individual ones of the set characteristic(s) may correspond to one or more similarities between the image(s) in the group, to which the corresponding set 404 corresponds. In those or other instances, a set characteristic may include a color corresponding to a similarity between the image(s) in the group, to which the corresponding set 404 corresponds, based on the similarity including the group having been identified as corresponding to the same candidate asset. In other words, a set 404 of the shape(s) can have a specific color to indicate that the set 404 corresponds to the image(s) 116 and the same candidate asset.
[0142] In some examples, any set 404 of shape(s) having one or more of the same, or similar characteristic(s) may be utilized to identify a candidate group of corresponding image(s) 116 associated with the shape(s) in the set 404 as having the same or similar image characteristic(s). In those or other examples, any set 404 of geometric shape(s) having one or more of the same, or similar characteristic(s) may be utilized to identify' the corresponding image(s) associated with the shape(s) in the set 404 as being associated with the same candidate asset.
[0143] The shape(s) in the set(s) 404 may be output (e.g., by the system(s)) in the window 402 based on output of one or more component(s) 202. 210, 218, 232, and / or 240, as discussed above with reference to FIGS. 2A-2E. respectively.
[0144] In some examples, the shape(s) in the set(s) 404 may be based on input to the UI 300, and / or output generated by the system(s) in response to receiving the input to the UI 300, as discussed above with reference to FIG. 3. The output may be generated by the system(s) in response to receiving input to the field(s) 306, 308. 310, 312, 314, and / or 316, as discussed above with reference to FIG. 3. The output may include identifying the image(s) 304, and possibly the candidate group(s) of the image(s) 304. The output may include the selection(s) by the user of at least one of the image(s) 304 as the at least one corresponding image 126.
[0145] In some examples, individual ones of the shape(s) in the set(s) 404 may represent, correspond to, and / or be associated with, a corresponding image 116 (e.g., a corresponding image 116 being automatically selected by the system(s)). Individual candidate groups may include at least one corresponding image from among the image(s) 116, based on the at least one corresponding image 116 being identified by the system(s) as being included in the candidate group. The at least one corresponding image 11 being identified as being included in any of the candidate group(s) may be based on output of the component(s) 202, 210, 218, 232, and / or 240, and / or data included therein. In some cases, the system(s) may automatically identify the at least one corresponding image 116 as being included in any of the candidate group(s). For instance, the system(s) may identify an image 116 as being included in a candidate group without input via the window 302.
[0146] While the shape(s) in the set(s) 404 may be associated with the image(s) 116, as discussed above, the UI 400 may output the shape(s) in the set(s) 404 based on the image(s) 116, the image(s) 118, and / or one or more other images. In some examples, individual ones of the shape(s) in the set(s) 404 may represent, correspond to, and / or be associated with, a corresponding image 304 (e.g., and a corresponding image 116). Individual candidate groups may include at least one corresponding image from among the image(s) 304.
[0147] The shape(s) can be selected by input received via one or more selections to the window 402. In some examples, data associated with individual ones of theshape(s) can be output (e.g., in the window 402 and / or in one or more other, different, windows) in response to receiving input selecting the corresponding shape. In those or other examples, data associated with a set 404 of the shape(s), such as corresponding to a group (e.g., in the cluster 126 and / or the cluster 128, associated with the image(s) 116 and / or the image(s) 118, respectively) can be output (e.g., in the window 402 and / or in one or more other, different, windows) in response to receiving input selecting the corresponding set 404.
[0148] The input selecting individual ones of the set(s) 404 associated with corresponding group(s) (e.g., the group(s) 126 / 128, and / or one or more other groups) can be received as input highlighting an area around a geometric shape and / or a set 404 of geometric shape(s) (e.g., such as a set 404 of the shape(s) associated with the corresponding group). For instance, input selecting a set (e.g., a set 404 of the shape(s) associated with the cluster 126, the cluster 128, or another cluster) can be received as input highlighting a geometric area (e.g., a box) around all of the shapes associated with the set 404. In various cases, input of a first type (e.g., a click and drag input) selecting a set 404 can be utilized to automatically output the data associated with the set 404.
[0149] In some examples, input of a first type (e.g., a click and drag input) selecting a first set 404 can be utilized to identify the first set 404, then input of the first type selecting a second set 404 can be utilized to identify the second set 404, possibly followed by more input of the first type, and then, the data associated with the groups can be output in response to input of a second type (e.g., a selection received by a “select” button, a single mouse click, etc.). Whether input of a particular type (e.g., the first type) is utilized to automatically output a set 404 and / or select a set 404 without automatic output of the set 404 may be controlled, for example, by one or more settings of the system(s).
[0150] FIG. 5 illustrates an example UI 500 utilized for file selection for image-to-asset linking in accordance with embodiments of the present disclosure. In some examples, the UI 500 can be implemented utilizing the system(s). as discussed above with reference to FIGS. 1-4.
[0151] In some examples, the UI 500 can include a window 502 being output (e.g., presented) by the UI 500 based on output received in response to one or moreoperations performed by one or more components (e.g., the component(s) 202, 210, 218, 232. and / or 240, as discussed above with reference to FIGS. 2A-2E) of the system(s), and / or data analyzed thereby. In those or other examples, the window 502 may be output (e g., presented) by the UI 500 based on output of both the window 302 and the window 402 (e.g., and not prior to both of output of the window 302 and output of the window 402). In those or other examples, the window 502 may be output (e.g., presented) by the UI 500 based on input received by both the window 302 and the window 402 (e.g., and not prior to input received by both the window 302 and the window 402).
[0152] Various options of automation can be provided by the system(s), such as based on one or more settings of the system(s). For example, the UI 500 can be output based on one or more operations performed by one or more components (e.g., the component(s) 202, 210, 218, 232, and / or 240), without requiring output of the window 302 and / or the window 402. In such an example or another example, the UI 500 can be output based on one or more operations performed by one or more components (e.g., the component(s) 202, 210. 218, 232. and / or 240), without requiring input received via the window 302 and / or the window 402.
[0153] The UI 500 can be utilized to output one or more images, such as an image 504, an image 506, an image 508, an image 510, an image 512, an image 514, and / or one or more other images. The image(s) 504, 508, and 512 may include a label (e.g.. the unique identifying label, as discussed above with reference to FIG. 1). The image(s) 504, 508, and 512, with a same type of data (e.g., visual data), may be output in a column. The image(s) 506, 510, and 514, with a same type of data (e.g., thermal data), may be output in a column.
[0154] Any of the image(s) 116 / 118 identified as any of the image(s) in the cluster(s) 126 / 128 may be output as any of the image(s) 504-514 based on the type of data in the image(s) 116 / 118. In some cases, all of the image(s) 504-514 may portray, and / or be associated with, a same asset (e.g., one of the asset(s) 122 / 124, as discussed above with reference to FIG. 1). Alternatively, at least one of the image(s) 504-514 may portray, and / or be associated with, a different asset from any of the others. For instance, a row of the image(s) 504-514, such as a row with the images 504 and 506 may portray, and / or be associated with, an asset (e.g., the asset 134). In such aninstance or another instance, another row with the images 508 and 510 may portray, and / or be associated with, a different asset (e.g.. the asset 136).
[0155] In some examples, for instance with one or more images (e.g., one or more images 116(A), as discussed above with reference to FIG. 1) including visual data, one of the image(s) 116(A) identified as one of the images 126(H) may be output as the image 504. In those or other examples, for instance with one or more images (e.g., one or more images 116(B), as discussed above with reference to FIG. 1) including thermal data, one of the image(s) 116(B) identified as one of the images 126(1) may be output as the image(s) 506.
[0156] In alternative examples, for instance with one or more images (e.g., one or more images 118(E). as discussed above with reference to FIG. 1) including visual data, one of the image(s) 118(E) identified as one of the images 128(L) may be output as the image(s) 508. In those or other examples, for instance with one or more images (e.g., one or more images 118(F), as discussed above with reference to FIG. 1) including thermal data, one of the image(s) 118(F) identified as one of the image(s) 126(M) may be output as on the image(s) 510.
[0157] The image(s) 504-514 can be identified and output by the window 502 based on various types of input. The input may be received from the user and by a select all field (or "select all'’) 516, a search field 518, a date field 520, a text field 522, a submit field (or “submit”) 524, a clustering graph field (or “clustering graph”) 526, and / or one or more other fields. In some examples, the select all field 516, the search field 518, the date field 520, the text field 522, the submit field 524, the clustering graph field 526, and / or the other field(s) can be utilized to implement, and / or include similar functionality as, the select all field 306, the search field 308, the date field 31 , the text field 312, the submit field 314, the clustering graph field 316, and / or the other field(s), respectively, as discussed above with reference to FIG.3. However, instead of searching a repository of images (e.g., the repository' including the image(s) 304, as discussed above with reference to FIG. 3), the window 502 can be utilized to search an asset list (e.g., the asset list as discussed above with reference to FIG. 1). The window 502 can be utilized to search the asset list to identify one or more candidate assets in the asset list as one or more candidate assets (e.g., the candidate asset(s), as discussed above with reference to FIG. 1). The window 502 canoutput the candidate asset data, as discussed below in further detail), identifying the candidate asset(s). In some instances, individual ones of the image(s) 504-514 can include a check box.
[0158] In some examples, individual ones of the image(s) 116 / 118 can be identified and output as a corresponding one of the image(s) 504-514 based on the image(s) 116 / 118 being identified as being a candidate image (e.g., one of the images 304, as discussed above with reference to FIG. 3). In those or other examples, individual ones of the image(s) 116 / 118 can be identified and output as a corresponding one of the image(s) 504-514 based on the image(s) 116 / 118 being identified as being in a set (e.g., one of the set(s) 404, as discussed above with reference to FIG. 4).
[0159] For instance, the individual ones of the image(s) 116 / 118 being identified and output as a corresponding one of the image(s) 504-514 may be based on the image linking data, as discussed above with reference to FIG. 1. In such an instance or another instance, at least one of the image(s) 116 / 118 may be utilized, for at least one of the image(s) 504-514. as a representative image (e.g., a token image, a cover image, a title image, etc.) of a corresponding one of the cluster(s) 126 / 128. In such an instance or another instance, the individual ones of the image(s) 116 / 118 can be identified and output as a corresponding one of the image(s) 504-514 based on the image 116 / 118 being clustered into one of the cluster(s) 126 / 128.
[0160] The window 502 can include one or more portions (or “asset data portion(s)”) of candidate asset data 528, 530, 532. The portion(s) (e.g., one or more windows) of candidate asset data 528, 530, 532 can be output by the window 502 in response to the system(s) loading one or more files associated with the corresponding portion(s) of candidate asset data 528, 530, 532. In some examples, the candidate asset data (or “candidate asset portion(s)”) 528, 530, and / or 532 can include initial candidate asset data, respectively corresponding to candidate assets (or “initial candidate assets’") initially identified by the system(s) (e.g., as being most likely to be the correct candidate assets).
[0161] The candidate asset data portion(s) can identify candidate asset data associated with corresponding candidate assets identified (e.g., by the system(s)) as having a likelihood of being corresponding assets that is greater than a likelihoodthreshold. In some examples, individual ones of the candidate asset data portion(s) 528 / 530 / 532 may include one or more details associated with one or more corresponding images.
[0162] In some examples, the candidate asset(s) can be identified based on similar data, or some or all of the same data (e.g., and / or same type of data), utilized to identify the candidate group(s) and / or set(s) 404, as discussed above with reference to FIGS. 3 and 4. For instance, any of the data (e.g., and / or type of data) utilized by any of the component(s) 202, 210, 218, 232, and / or 240 can be utilized to identify the candidate asset(s). Individual ones of the candidate asset(s) may be associated with a corresponding cluster (e.g., a cluster 126 or 128), and / or a corresponding set 404, as discussed above with reference to FIG. 4.
[0163] Individual ones of the candidate asset data portion(s) 528 / 530 / 532 can include, for a corresponding candidate asset, various types of data, such as text describing information (e.g., one or more characteristics, aspects, etc.) of the corresponding candidate asset. The candidate asset data portion(s) 528 / 530 / 532 can be retrieved from the asset list. The candidate asset data portion(s) 528 / 530 / 532 can include an identifier, a name, data, a serial number, quality information, a previously linked asset (e.g., if one is available), capabilities, features, compatibilities, etc., or any other type of asset information.
[0164] The candidate asset data portion(s) 528 / 530 / 532 can be retrieved based on data from any of the component(s) 202, 210, 218, 232, and / or 240. For instance, the data from any of the component(s) 202, 210, 218, 232, and / or 240 utilized to identify the candidate asset data portion(s) 528 / 530 / 532 can include, for a corresponding candidate asset, an identifier, a name, OCR data, a serial number, quality information, a previously linked asset (e.g., if one is available), capabilities, features, compatibilities, etc., or any other type of asset information.
[0165] While the portion(s) (e.g., one or more windows) of candidate asset data 528, 530, 532, and / or the data respectively associated therewith, can be output by the window 502. as discussed above with respect to the cunent disclosure, it is not limited as such. In some examples, the file(s), such as one or more machine-health diagnostic files associated with the corresponding portion(s) of candidate asset data 528, 530,532, can be visually presented in a similar way as the portion(s) of candidate asset data 528, 530, 532. and / or the data respectively associated therewith.
[0166] For example, the UI 500 can be utilized to visually present, within the Ul 500, each individual machine-health diagnostic file (e.g., associated with each of the portion(s) of the candidate asset data 528, 530, 532) contained within the selected cluster. In such an example or another example, any content (e.g.. a name, an identifier, a portion, a subset, any metadata, and so on, or any combination thereof) associated with any file (e.g., any machine-health diagnostic file), can be visually presented in a similar way as any file.
[0167] FIG. 6 illustrates an example UI 600 utilized for asset selection for image-to-asset linking in accordance with embodiments of the present disclosure. In some examples, the UI 600 can be implemented utilizing the system(s), as discussed above with reference to FIGS. 1-5.
[0168] The UI 600 can output one or more images, such as the image(s) 504-514, as discussed above with reference to FIG. 5. In some examples, the UI 600 may be the same as the UI 500, at a different stage (e.g., a subsequent stage). The UI 600 can output candidate asset identifier data 604. The candidate asset identifier data 604 can include one or more candidate asset identifiers associated with one or more candidate assets. For example, a candidate asset may include an asset (or ‘‘automatically selected assef ’) that is automatically selected by one or more systems (e.g.. the system 102, such as using the model (s) 104, as discussed above with reference to FIG. 1). In some instances, once selected by an operator, any of the candidate asset(s) can be used to identify an asset (e.g., as a selected asset, such as the selected asset 134 or 136, as discussed above with reference to FIG. 1) for at least one corresponding image from among the image(s) 504-514. In those or other instances, the candidate asset identifier(s) can be identified as one or more asset identifiers, respectively, based on one or more operator selections received via operator input, respectively. For example, any of the asset identifier(s) corresponding to the candidate asset identifiers 01 / 02 / 03 / etc. can be identified (e.g., via user input to the candidate asset identifier data 604) in asset data (e.g., any of the asset data 704 / 706 / 708, as discussed below with reference to FIG. 7), such as possibly with one of the candidate asset identifiers 01 / 02 / 03 / etc. being selected (e.g., via the user input to the candidate asset identifierdata 604) as an asset for the image(s) 504 / 506. The asset being selected via the user input to the candidate asset identifier data 604 may be indicated via the asset data (e.g., any of the asset data 704 / 706 / 708).
[0169] In some examples, a window including the candidate asset identifier data 604 can be output, possibly with a portion of the w indow that includes the candidate asset identifier data 604 overlapping a portion of at least one of the image(s) 504-514. The window with the candidate asset identifier data 604 can be output in a location indicating that the window is associated w ith one of the image(s) 504-514. The window with the candidate asset identifier data 604 may be associated with one of the image(s) 504-514 based on input received via a selection (e.g., a mouse click) of a user of the corresponding one of the image(s) 504-514.
[0170] In some examples, the UI 600 can be utilized to output one or more fields. The field(s) can include a select all field, a search field, a date field, a text field, a submit field, a clustering graph field, and / or one or more other fields. In those or other examples, the select all field, the search field, the date field, the text field, the submit field, the clustering graph field, and / or the other field(s) may be utilized to implement, and / or may include the same or different functionality as, the select all field 516, the search field 518, the date field 520, the text field 522, the submit field 524, the clustering graph field 526, and / or the other field(s), respectively .
[0171] In various examples, the candidate asset identifier data 604 can include the candidate asset identifier(s) 01 / 02 / 03 / etc. (e.g., as text, representative images / symbols, etc., any other type of identifier, or any combination thereof) associated with one or more candidate assets (e.g., one or more candidate assets identified from among a plurality of assets in an asset list, which may be stored in the system 102). The candidate asset identifier data 604 can be identified and / or output based on operator input selecting individual ones of the image(s) 504-514 and / or the candidate asset data 528 / 530 / 532 (e.g., one or more shapes, such as boxes utilized to output the candidate asset data 528 / 530 / 532).
[0172] Alternatively or additionally, candidate asset identifier data 604 can be identified and / or output based on operator input to at least one of the field(s) (e.g., in the UI 500 and / or the UI 600). The candidate asset identifier data 604 can be identified and / or output based one or more searches performed using the operatorinput to the at least one of the field(s). The search(es) may include any type of search, such as a text search (e.g., simple full-text search, Boolean full-text search, fuzzy search, wildcard search, phrase search, proximity search, range search, faceted search, etc., or any combination thereof) to identify the candidate asset identifier data 604.
[0173] Alternatively or additionally, any of the candidate asset identifier data 604 may be automatically identified by the system 102, such as by using image matching (e.g., between any of the image(s) 504-514 and any of one or more stored images of one or more assets in the asset list). Alternatively or additionally, any of the candidate asset identifier data 604 may be automatically identified by the system 102, such as by using OCR matching (e.g., between any text identified via OCR in the image(s) 504-514 and any text, such as OCR text or any other text, of any of the stored image(s) of the asset(s) in the asset list). Alternatively or additionally, any of the candidate asset identifier data 604 may be automatically identified by the system 102, such as by using metadata matching (e.g., between any metadata identified / provided for the image(s) 504-514 and any metadata of any of the stored image(s) of the asset(s) in the asset list). Alternatively or additionally, any of the candidate asset identifier data 604 may be automatically identified by the system 102, such as by using any other comparison (e.g., between any other data associated with the image(s) 504-514 and any other stored data associated with the asset(s) in the asset list).
[0174] The candidate asset identifier data 604 may be automatically identified by the system 102 based on likelihood of any of the asset(s) in the asset list being an asset associated with any of image(s) 504-514. For example, the system 102 can identify one or more scores (e.g., likelihood score(s) associated with one or more likelihood of any of the asset(s) in the asset list being an asset associated with any of image(s) 504-514. In some instances, a likelihood score may include, individually or in combination, a score associated with a result of a text search, a score associated with a result of image matching, a score associated with a result of OCR matching, a score associated with a result of the metadata matching, and / or a score associated with any other type of comparison. In those or other instances, a likelihood score may include a sum, an average, a maximum, a minimum, etc., or any other type of score calculated from a score associated with a result of a text search, a score associated with a result of image matching, a score associated with a result of OCR matching, ascore associated wi th a result of the metadata matching, and / or a score associated with any other type of comparison.
[0175] Operator input may be received, for example, based on an operator selecting, via user input corresponding to a candidate asset identifier, a candidate asset (e.g., to which the candidate asset identifier corresponds) as an asset. For example, operator input may be received, based on an operator identifying a candidate asset (e.g., such as an initial candidate asset, as discussed above with reference to FIG. 5, with which an initial candidate asset identifier, such as a candidate asset identifier 01, is associated) for at least one of the image(s) 504-514.
[0176] In some examples, operator input may be received, for example, based on an operator selecting a different candidate asset (e.g., such as any asset identified by any of the candidate asset identifier(s) 02 / 03 / etc.) as corresponding to at least one of the image(s) 504-514. The different candidate asset may be selecting as the asset based on the operator determining one of the candidate asset identifier(s) corresponds to at least one of the image(s) 116 / 118 (e.g., being identified and output as any of the image(s) 504-514).
[0177] Any of the candidate asset identifier(s) of the candidate asset(s) selected to be the asset identifier(s) in the candidate asset identifier data 604 may be the same as, or different from, the candidate asset identifier(s) initially selected as the candidate asset(s) output by the window 502. One or more same, or different, candidate asset identifier(s) can be selected based on input received via a selection (e.g., one or more mouse clicks, one or more keyboard keystrokes, and / or one or more selections of any ty pe) of a user of the corresponding one of the candidate identifier(s). The operator can select a different candidate asset identifier to change from an initial candidate asset to a new candidate asset, for example, if the operator determines that the initial candidate asset (e.g., the candidate asset identified in any of the candidate asset data 528 / 530 / 532) was incorrect. The candidate asset data 528 / 530 / 532 can be updated based on the new candidate asset(s) being identified as the corresponding asset(s) (e.g., the corresponding selected asset(s)).
[0178] FIG. 7 illustrates an example UI 700 utilized for outputting selected asset information in accordance with embodiments of the present disclosure. In someexamples, the UI 700 can be implemented utilizing the system(s), as discussed above with reference to FIGS. 1-6.
[0179] In some examples, the UI 700 can include a window 702 output (e.g., presented) by the UI 700 based on output received in response to one or more operations performed by one or more components (e.g., the component(s) 202, 210, 218, 232, and / or 240, as discussed above with reference to FIG. 2) of the system(s), and / or data analyzed thereby. In some examples, the UI 700 may be the same as the UI 500 and / or the UI 600, at a different stage (e.g., a subsequent stage). In those or other examples, the window 702 may be (e.g., presented) output by the UI 700 based on output received by the system(s) in response to input received by the window 302, the window 402, the window 502, the window 602, and / or one or more other windows.
[0180] The window 702 can be utilized to output the image(s) 504-514, as discussed above with reference to FIG. 5. In some examples, the window 702 can be utilized to output one or more portions (e.g.. one or more windows) of asset data 704, 706, and / or 708. In those or other examples, the asset data portion(s) 704 / 706 / 708 may represent, correspond to, and / or be associated with the candidate asset portion(s) 528, 530, and / or 532. The asset data portion(s) 704 / 706 / 708 may include similar, and / or same data, as the candidate asset portion(s) 528, 530, and / or 532. Alternatively or additionally, the asset data portion(s) 704 / 706 / 708 may include types of data that are similar, and / or same data, as the candidate asset portion(s) 528, 530, and / or 532.
[0181] While the candidate asset portion(s) 528, 530, and / or 532 may be intermediate / non-final, candidate, portions, the asset data portion(s) 704 / 706 / 708 may be selected based on the selection(s) by the user received via the window 602. In some examples, the asset data portion(s) 704 / 706 / 708 may be automatically identified based on the cluster(s) 126 / 128. Alternatively or in addition, the asset data portion(s) 704 / 706 / 708 may be automatically identified based on the data analyzed by the component(s) 202, 210, 218, 232, and / or 240. Alternatively or in addition, the asset data portion(s) 704 / 706 / 708 may be automatically identified based on the candidate asset portion(s) 528, 530, and / or 532, without operator input, by the system(s).
[0182] The window 702 can output a select all field 710, a search field 712, a date field 714, a text field 716, a submit field 718, a clustering graph field 720, and / or oneor more other fields. The select all field 710, a search field 712, a date field 714, a text field 716, a submit field 718. a clustering graph field 720, and / or the other field(s) may be utilized to implement, and / or may include the same or different functionality as, the select all field, the search field, the date field, the text field, the submit field, the clustering graph field, and / or the other field(s), as discussed above with reference to FIG(S). 5 and / or 6. However, instead of being utilized to identify the candidate asset(s) (e.g.. the candidate asset(s) with which the candidate asset data 528-532 is associated), the data associated with the candidate asset(s), the identifier(s) (e.g., the identifier(s) in the identifier data 604), and / or the data associated with the identifier(s), the select all field 710, a search field 712, a date field 714, a text field 716, a submit field 718, a clustering graph field 720, and / or the other field(s) may be utilized to request the system(s) to identify the asset(s) (e.g., the asset(s) in the asset data portion(s) 704 / 706 / 708).
[0183] At least one of the plurality of clusters (e.g., a cluster with images 504 and 506) may be associated with the one or more of the asset IDs associated with the one or more of the plurality of machine assets (e.g., an asset identified via the asset data 704). The system(s) may associated the at least one of the plurality of clusters (e.g., a cluster with images 504 and 506) with the one or more of the asset IDs automatically. The system(s) may automatically associate the at least one of the plurality of clusters with the one or more of the asset IDs based on a comparison of the respective sets of metadata associated with the machine-health diagnostic files contained in the at least one of the plurality7of clusters and metadata associated with the one or more of the plurality7of machine assets (e.g., the asset in the asset data 704).
[0184] While the asset data portion(s) 704 / 706 / 708 may be selected and / or output based on the selection(s) by the user received via the window 602 as discussed in the current disclosure, it is not limited as such. In some examples, the UIs 400 / 500 / 600 / 700 may be output any number of times, and / or in any order, to select, modify, delete, and / or update the candidate asset(s) identified for the image(s) 504-514, such as based on automated operations by the system(s) and / or user input to the window(s) 502, 602, and / or 702. In those or other examples, the UIs 500 / 600 / 700 may be output any number of times, and / or in any order, to select, modify, delete,and / or update the asset(s), such as based on automated operations by the system(s) and / or user input to the window(s) 502, 602, and / or 702.
[0185] While the Uls 400 / 500 / 600 / 700 may be output, as discussed above in the current disclosure, it is not limited thereto. In some examples, presentation of any of the Uls 400 / 500 / 600 / 700 can transition to, and / or be replaced by, presentation of another of the Uls 400 / 500 / 600 / 700, any number of times, any number of iterations, any number of combinations, etc., or any combination thereof. For instance, a user can switch back and forth between presentation of any of the Uls 400 / 500 / 600 / 700 to input one or more selections for identifying any of the result(s) via any of the Uls 400 / 500 / 600 / 700. By way of example, the user can switch back and forth between presentation of any of the Uls 400 / 500 / 600 / 700 while they work through the image(s) 304 so that any clusters of the image(s) can be identified.
[0186] FIG. 8 illustrates example sets of images 800 for outputting linked asset information in accordance with embodiments of the present disclosure. The sets of images 800 may include a set of images 802, which may include one or more images (e.g., the image(s) 504-514, as discussed above with reference to FIGS. 5-7). The sets of images 800 may include, alternatively or additionally to the set of images 802, a set of images 804.
[0187] The set of images 804 may be utilized by one or more systems (e.g., the system(s), as discussed above with reference to FIGS. 5-7) in a similar way as for the set of images 802. For example, the set of images 804 may be output by any of one or more windows (e.g., any of the window(s) 502, 602, and / or 702, as discussed above with reference to FIGS. 5-7). Additional information provided by the set of images 804 in comparison to the set of images 802 may enable an operator to more easily match any images in the set of images 804 to assets. The additional information can include additional views / portrayals of assets, such as internal views, top views, side views, etc., or any combination thereof. Any number of different types of images, such as with any other type being shown in a different column, can be output by the window(s) 502, 602, and / or 702. in a similar way as for the sets of images 802 and 804.
[0188] The set of images 804 may be output by the window(s) 502, 602, and / or 702 to enable an operator to view the set of images 804 and candidate asset dataassociated therewith, which maybe generated by the system(s). The operator may¬ input one or more selections to identify any of one or more candidate asset(s) in the candidate asset data as being one or more corresponding assets. The asset(s) can be output by the window 702 along with the set of images 804.
[0189] The set of images 804 may include, such as in a column, one or more images 806 and 810 with a same type of data as one another. The set of images 804 may include, such as in the column, an image 808 (e.g.. with the same type of data) representing a missing image. For example, the type of data in the image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include external views of one or more corresponding assets (e.g., one or more electrical panels). The type of data in the image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include visual data. The image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include the same type of data, and / or the same view, as for the image(s) 504-512. respectively, and / or the image(s) 506-514. respectively.
[0190] The image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include a different view from any of the image(s) 504-512 and / or the image(s) 506-514. For instance, any of the image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include an external view, an internal view, and so on, or any combination thereof. The type of data in the image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include visual data. However, in alternative or additional examples, the type of data in the image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include visual data, acoustic data, thermal data, another type of data, or any combination thereof.
[0191] Any of the image(s) 806, 808, and / or 810, such as the image 808, may represent a missing image or an image that is blank. The image(s) 806 and / or 810 (e.g., and the image 808, such as if the image 808 alternatively represents an actual image) may include visual data. The image(s) 806 and / or 810 (e.g., and the image808, such as if the image 808 alternatively represents an actual image) may include a label (e.g., the unique identifying label, as discussed above with reference to FIG. 1).
[0192] The set of images 804 may include, such as in a column, one or more images 812, 814, and / or 816 with a same type of data as one another. For example, the type of data in the image(s) 812, 814, and / or 816 may include external views of the corresponding asset(s) (e.g., the electrical panel(s)). Any of the images 812, 814, and / or 816 may represent a missing image or an image that is blank. The image(s) 812-816 may include the same type of data, and / or the same view, as for the image(s) 504-512, respectively, and / or the image(s) 506-514, respectively. The image(s) 812-816 may include a different view from any of the image(s) 504-512 and / or the image(s) 506-514. For instance, any of the image(s) 812-816 may include an external view, an internal view, and so on, or any combination thereof. The type of data in the image(s) 812, 814, and / or 816 may include acoustic data, thermal data. However, in alternative or additional examples, the ty pe of data in the image(s) 812-816 may include visual data, acoustic data, thermal data, another type of data, or any combination thereof.
[0193] The set of images 804 may include, such as in a column, one or more images 818, 820, and / or 822 with a same type of data as one another. For example, the type of data in the image(s) 818, 820, and / or 822 may include internal views of the corresponding asset(s) (e.g., the electrical panel(s)). Any of the images 818, 820, and / or 822 may represent a missing image or an image that is blank. The image(s) 818, 820, and / or 822 may include a same type of data (e.g., visual data) as the image(s) 504-512, respectively , and / or the image(s) 506-514, respectively. The image(s) 818, 820, and / or 822 may include a different view from any of the image(s) 504-512 and / or the image(s) 506-514. For instance, any of the image(s) 818, 820, and / or 822 may include an external view, an internal view, and so on, or any combination thereof. The type of data in the image(s) 818, 820, and / or 822 may include visual data. However, in alternative or additional examples, the ty pe of data in the image(s) 818, 820, and / or 822 may include visual data, acoustic data, thermal data, another ty pe of data, or any combination thereof.
[0194] Any of the image(s) 806 and 810-822 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may be associated with thesame type of asset as, or a different type of asset from, one another. Any of the image(s) 806 and 810-822 (e.g., e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may be associated with the same asset as, or a different asset from, one another. Any of the image(s) 806 and 810-822 (e g., and the image 808 such as if the image 808 alternatively represents an actual image) may be associated with the same type of asset as, or a different ty pe of asset from, any of the image(s) 504-512, respectively, and / or the image(s) 506-514, respectively. Any of the image(s) 806 and 810 (e.g., e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may be associated with the same asset as, or a different asset from, any of the image(s) 504-512, respectively, and / or the image(s) 506-514, respectively.
[0195] The image(s) 806 and 810-822 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may include various images, such as any of the image(s) 116 / 118 (e.g., being identified as any of the image(s) in the cluster(s) 126 / 128). For example, similarly as for the image(s) 504-514, any of the image(s) 116 / 118 (e.g., being identified as any of the image(s) 126(H), 126(1), 128(L). and / or 128(M)) may be output as any of the image(s) 806 and 810-822 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) based on the type of data in the image(s) 116 / 118. The image(s) 804 may include any number of images in any number of rows and in any number of columns, with any number of the image(s) 804 missing, or representing a blank image.
[0196] In some cases, all of the image(s) 806 and 810-822 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may portray, and / or be associated with, a same asset (e.g., one of the asset(s) 134 / 136, as discussed above with reference to FIG. 1). Alternatively, at least one of the image(s) 806 and 810-822 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may portray, and / or be associated with, a different asset from any of the others. For instance, a row of the image(s) 504-514, such as a row with the images 806, 812. and 818 may portray, and / or be associated with, an asset (e.g., the asset 134). In such an instance or another instance, another row with the images 814 and 820 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may portray, and / or be associated with, a different asset (e.g., the asset136). In such an instance or another instance, another row with the images 810, 816, and 822 may portray, and / or be associated with, a different asset. Any number of the image(s) 804 in any row and / or column may portray, and / or be associated with, a same asset or a different asset.
[0197] While the set of images 804 may include, in respective columns, the image(s) 806 and 810 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) with the same type of data as one another, the image(s) 812, 814, and 816 with the same type of data as one another, the image(s) 818, 820, and / or 822 with the same type of data as one another, as discussed above in the current disclosure, it is not limited as such. In some examples, any number of images of any type of data may be included in any column.
[0198] While the type of data in the image(s) 806 and 810 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) may include external views, the type of data in the image(s) 812, 814, and / or 816 may include external views, and / or the type of data in the image(s) 818, 820, and / or 820 may include views, as discussed above in the current disclosure, it is not limited as such. For example, data in the image(s) 806 and 810 (e.g., and the image 808 such as if the image 808 alternatively represents an actual image) can include any type of data, such as external views, internal views (e.g., internal thermal views of equipment, such as an electrical panel, etc.), any other ty pes of views, or any combination thereof.
[0199] FIGS. 9 and 10 illustrate flow diagrams of methods of automated clustering-based image-to-asset linking in accordance with embodiments of the present disclosure. Referring to FIG. 9, a method 900 of one embodiment of automated clustering-based image-to-asset linking is illustrated. In general, the method 900 will be described herein with reference to the system(s) as described above with reference to FIGS. 1-8. The method 900 may include at (902), storing a plurality of machine-health diagnostic files and a plurality7of asset identifiers (IDs) respectively associated with a plurality of machine assets.
[0200] The method 900 may include at (904), obtaining a plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files (e.g., files corresponding to images 116). The metadata may can be managed by the machine learned models (e.g., the machine learned model(s) in the model(s) 104).
[0201] The method 900 may include at (906), clustering the plurality of machinehealth diagnostic files (e.g., the images 116) into a plurality of clusters (e.g., the cluster 126) based on the plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files (e.g., the images 116). A plurality of machine-health diagnostic files (e.g., the images 118) may be clustered into a plurality of clusters (e.g., the cluster 128) based on the plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files (e.g.. the images 118).
[0202] The method 900 may include at (908), associating at least one of the plurality of clusters with one or more of the asset IDs associated with one or more of the plurality of machine assets. The plurality of clusters may include the cluster 126 being associated with one or more of the asset IDs, such as an asset ID of the asset 134. The plurality of clusters may include the cluster 128 being associated with one or more of the asset IDs, such as an asset ID of the asset 136.
[0203] Referring to FIG. 10, a method 1000 of one embodiment of automated clustering-based image-to-asset linking is illustrated. In general, the method 1000 will be described herein with reference to the system(s) as described above with reference to FIGS. 1-8. The method 1000 may include at (1002), obtaining, by a computing system (e.g., the system 102) that includes one or more computing devices, a plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files. The plurality of machine-health diagnostic files may include the images 116 and / or the images 118.
[0204] The method 1000 may include at (1004), clustering, by the computing system, the plurality of machine-health diagnostic files, such as the images 116, into a plurality of clusters, such as the cluster 126, based on the plurality of sets of metadata respectively associated with the plurality' of machine-health diagnostic files (e.g., the images 116). The plurality of machine-health diagnostic files, such as the images 118, may be clustered into the plurality of clusters, such as into the cluster 128, based on the plurality of sets of metadata respectively associated with the plurality of machinehealth diagnostic files (e.g., the images 118).
[0205] The method 1000 may include at (1006), associating, by the computing system, at least one of the plurality of clusters with one or more of the asset IDsassociated with one or more of the plurality7of machine assets. The cluster 126 may be associated with an asset ID, which may be associated with the selected asset 134. The image(s) 126(H) and 126(1) may be associated with the asset ID of the selected asset 134 based on the cluster 126 being linked to the selected asset 134. The cluster 128 may be associated with an asset ID, which may be associated with the selected asset 136. The image(s) 128(L) and 128(M) may be associated with the asset ID of the selected asset 136 based on the cluster 128 being linked to the selected asset 136.
[0206] FIG. 11 illustrates a block diagram of a computing system 10 that performs image clustering, and image-to-asset linking in accordance with embodiments of the present disclosure. The computing system 10 includes a first computing system 20 and a second computing system 40 that are communicatively coupled over a network 60.
[0207] The first computing system 20 may include one or more computing devices 21 or circuitry7. For instance, the first computing system 20 may include a processor 22 and a memory7device 23. In an embodiment, the processor 22 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In some implementations, the processor 22 may be programmed by one or more computer-readable or computer-executable instructions stored on the memory device 23.
[0208] In an embodiment, the memory7device 23 may include one or more non-transitory computer-readable medium, also referred to as a data storage device, which may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any7suitable combination thereof. The memory device 23 may form, e g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory7, a random access memory (RAM), a read-only memory7(ROM), an erasable programmable readonly memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory7(DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory7stick.
[0209] The memory7device 23 may store information that may be accessed by the processor 22. For instance, the memory device 23 may store data 24 that can be obtained, received, accessed, written, manipulated, created, and / or stored. The data 24 can include, for instance, any of the data or information described herein, including but not limited to any data utilized by the system 102, and / or data utilized by the component(s) 202, 210, 218, 232, and / or 240. In some implementations, the first computing system 20 can obtain data from one or more memory devices that are remote from the first computing system 20 (e.g., from second computing system 40).
[0210] The memory device 23 may also store computer-readable instructions 25 that can be executed by the processor 22. The instructions 25 may be software written in any suitable programming language or can be implemented in hardware. The instructions may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms '‘computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term '‘module” refers broadly to a collection of software instructions or code configured to cause the processor 22 to perform one or more functional tasks. The modules and computer-readable / executable instructions may be described as performing various operations or tasks when the processor 22 or other hardware component is executing the modules or computer-readable instructions.
[0211] The instructions 25 may be executed in logically and / or virtually separate threads on the processor 22. For example, the memory device 23 can store instructions 25 that when executed by the processor 22 cause the processor 22 to perform any of the operations, methods and / or processes described herein. In some cases, the memory device 23 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the methods of FIGS. 9 and 10.
[0212] In an embodiment, the first computing system 20 may store or include one or more machine learned models 26. For example, the machine learned models 26 may be or may otherwise include various machine learned models, including one ormore machine learned models of the model(s) 104. In an embodiment, the machine learned models 26 may include a supervised learning model (e.g.. for image clustering, and image-to-asset linking) or an unsupervised learning model (e.g., for image clustering, and image-to-asset linking). In an embodiment, the machine learned models 26 may include neural networks (e.g., deep neural networks) or other types of machine learned models, including non-linear models and / or linear models. Neural networks may include feed-forward neural networks, recurrent neural networks (e.g.. long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine learned models may leverage an attention mechanism such as self-attention. For example, some example machine learned models may include multi-headed self-attention models (e.g., transformer models).
[0213] In an embodiment, the one or more machine learned models 26 may be received from the second computing system 40 over one or more networks 60, stored in the first computing system 20 (e.g., in memory device 23), and then used or otherwise implemented by the processor 22. In an embodiment, the first computing system 20 may implement multiple parallel instances of a single model.
[0214] Additionally or alternatively, one or more machine learned models 46 may be included in or otherwise stored and implemented by the second computing system 40 that communicates with the first computing system 20 according to a client-server relationship. For example, the machine learned models 26 may be implemented by the second computing system 40 as a portion of a web service. Thus, one or more models 26 may be stored and implemented at the first computing system 20 and / or one or more models 46 may be stored and implemented at the second computing system 40.
[0215] The first computing system 20 may include a communication interface 27. The communication interface 27 may be used to communicate with one or more other systems (e.g., for receiving data from the camera 120, as discussed above with reference to FIG. 1). The communication interface 27 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 60). In some implementations, the communication interface 27 may include for example, one or more of a communications controller, receiver, transceiver,transmitter, port, conductors, software and / or hardware for communicating data / information.
[0216] The first computing system 20 may also include one or more user input components 28 that receives user input. For example, the input components 28 may be a touch-sensitive component (e.g., a touch-sensitive display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component may serve to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, cursor-device, joystick, or other devices (e.g., a mouse, etc.) by which a user (e.g., a vehicle operator, passenger, or other user) may provide user input. User input component(s) 28 may be used, for example, for image clustering, and image-to-asset linking, such as via the UIs 300-700, and / or one or more other UIs utilized by the system 102.
[0217] The first computing system 20 may include one or more output components 29. The output components 29 can include hardware and / or software for audibly or visually producing content. For instance, the output components 29 can include one or more speaker(s), earpiece(s), headset(s), handset(s), etc. The output components 29 can include a display device, which can include hardware for displaying a user interface and / or messages for a user. By way of example, the output component 29 can include a display screen, CRT, LCD, plasma screen, touch screen, TV, projector, tablet, and / or other suitable display components (e.g., augmented reality (AR) / virtual reality (VR) goggles, etc.).
[0218] The second computing system 40 can include one or more computing devices 41. In an embodiment, the second computing system 40 may include or is otherwise implemented by the one or more computing devices 41. In instances in which the second computing system 40 includes plural computing devices (e.g., server computing devices), such computing devices 41 may operate according to sequential computing architectures, parallel computing architectures, or some combination thereof.
[0219] The second computing system 40 can include a processor 42 and a memory device 43. In an embodiment, the processor 42 may include one or more processors (e.g., microprocessors), one or more processing cores, a programmable logic circuit (PLC) or a programmable logic / gate array (PLA / PGA), a fieldprogrammable gate array (FPGA), an application specific integrated circuit (ASIC), or any other control circuit. In an embodiment, the processor 42 may be programmed by one or more computer-readable or computer-executable instructions stored on the memory device 43.
[0220] In an embodiment, the memory device 43, also referred to as a data storage device, may include an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. The memory device 43 may form, e.g., a hard disk drive (HDD), a solid state drive (SDD) or solid state integrated memory, a random access memory (RAM), a read-only memory7(ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), dynamic random access memory (DRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), and / or a memory stick.
[0221] The memory7device 43 may store information that may be accessed by the processor 42. For instance, the memory device 43 may store data 44 that can be obtained, received, accessed, written, manipulated, created, and / or stored. The data 44 can include, for instance, any of the data or information described herein, including but not limited to and / or any data utilized by the system 102, and / or data utilized by the component(s) 202, 210, 218, 232, and / or 240. In some implementations, the second computing system 40 can obtain data from one or more memories that are remote from the second computing system 40.
[0222] The memory device 43 may also store computer-readable instructions 45 that can be executed by the processor 42. The instructions 45 may be software written in any suitable programming language or can be implemented in hardware. The instructions 45 may include computer-readable instructions, computer-executable instructions, etc. As described herein, in various embodiments, the terms '‘computer-readable instructions” and “computer-executable instructions” are used to describe software instructions or computer code configured to carry out various tasks and operations. In various embodiments, if the computer-readable or computer-executable instructions form modules, the term '‘module” refers broadly to a collection of software instructions or code configured to cause the processor 42 to perform one or more functional tasks. The modules and computer-readable / executable instructionsmay be described as performing various operations or tasks when the processor 42 or other hardware component is executing the modules or computer-readable instructions.
[0223] The instructions 45 may be executed in logically and / or virtually separate threads on the processor 42. For example, the memory device 23 can store instructions 45 that when executed by the processor 42 cause the processor 42 to perform any of the operations, methods and / or processes described herein. In some cases, the memory device 43 may store computer-executable instructions or computer-readable instructions, such as instructions to perform at least a portion of the methods of FIGS. 9 and 10.
[0224] The second computing system 40 may store or otherwise include one or more machine learned models 46 including one or more machine learned models of the model (s) 104. The machine learned models 46 may include or be the same as the models 26 stored in first computing system 20. In an embodiment, the machine learned models 46 can include a supervised learning model or an unsupervised learning model. In an embodiment, the machine learned models 46 can include neural networks (e.g., deep neural networks) or other types of machine learned models, including non-linear models and / or linear models. Neural networks may include feedforward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks or other forms of neural networks. Some example machine learned models may leverage an attention mechanism such as self-attention. For example, some example machine learned models may include multi-headed self-attention models (e.g., transformer models).
[0225] The second computing system 40 may include a communication interface 49. The communication interface 49 may be used to communicate with one or more other systems. The communication interface 49 may include any circuits, components, software, etc. for communicating via one or more networks (e.g., networks 60). In some implementations, the communication interface 49 may include for example, one or more of a communications controller, receiver, transceiver, transmitter, port, conductors, software and / or hardware for communicating data / information.
[0226] The first computing system 20 and / or the second computing system 40 may train the models 26, 46 via interaction with a model trainer 47 that trains the machine learned models 26, 46 stored at the first computing system 20 and / or the second computing system 40 using various training or learning techniques. The model trainer 47 can utilize training techniques, such as backwards propagation of errors. For example, a loss function may be backpropagated through a model to update one or more parameters of the models (e.g., based on a gradient of the loss function). Various loss functions may be used such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update the parameters over a number of training iterations.
[0227] In an embodiment, performing backwards propagation of errors may include performing truncated backpropagation through time. The model trainer 47 may perform a number of generalization techniques (e.g., weight decays, dropouts, etc.) to improve the generalization capability’ of a model being trained. In particular, the model trainer 47 may train the machine learned models 26, 46 based on a set of training data 48.
[0228] In an embodiment, if consent / authorization has been provided, the training examples within training data 48 may be provided by the first computing system 20 and / or second computing system 40. Thus, in such implementations, a model 26 provided to the first computing system 20 may be trained by the model trainer 47 in a manner to personalize the model 26.
[0229] The model trainer 47 may include computer logic utilized to provide desired functionality. The model trainer 47 may be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in an embodiment, the model trainer 47 may include program files stored on a storage device, loaded into a memory’, and executed by one or more processors. In other implementations, the model trainer 47 may include one or more sets of computerexecutable instructions that are stored in a tangible computer-readable storage medium such as RAM, hard disk, or optical or magnetic media. In some implementations, the model trainer 47 can be implemented in a cloud-computing environment.
[0230] The machine learned models described in this specification may have various types of input data and / or combinations thereof, representing data available to sensors and / or other systems. Input data can include, for example, latent encoding data (e.g., a latent space representation of an input, etc.), statistical data (e g., data computed and / or calculated from some other data source), sensor data (e.g., raw and / or processed data captured by a linescanner or other image capture device), or other types of data.
[0231] While the computing system 10 can include the computing system(s) 20 and / or 40, as discussed above in the current disclosure, it is not limited as such. In some examples, the computing system 10 can include any number of computing systems, such as the computing system(s) 20, 40, and / or a third computing system.
[0232] By way of a hypothetical example, any combination of large language models (LLMs) and / or visual learning models (VLMs) can exist as a third-party sendee. A system (e.g., the system 20) can include a user's PC / tablet. A system (e.g., the system 40) can include a cloud web-application. Another system (e.g., a system providing the third-party service) can host, and / or be utilized to implement, the LLM / VLM models (e g. a cloud computing platform utilized to manage, access, and / or develop applications and / or sendees provided to individuals, companies, and / or governments through a global infrastructure, etc.). The system (e.g., the system 40) with the cloud web-application can include all of the prompts (e.g., from prompt engineering). The LLM / VLM can exist elsewhere in the cloud, such as on yet another computing platform. The system 20, the system 40, the other system (e.g., providing the third-party7service), and / or one or more other systems, can be utilized to implement, as appropriate, any of the techniques as discussed above with respect to FIGS. 1-9.
[0233] FIG. 11 illustrates one example computing system that may be used to implement the present disclosure. Other computing systems may be used as well. For example, in an embodiment, the first computing system 20 may include the model trainer 47 and the training data 48. In such implementations, the models 26, 46 may be both trained and used locally at the first computing system 20. In some of such implementations, the first computing system 20 may implement the model trainer 47 to personalize the models 26.
[0234] This writen description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they include structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.
[0235] Each example is provided by way of explanation, rather than limitation of, the technology. In fact, it will be apparent to those skilled in the art that modifications and variations can be made in the present technology without departing from the scope or spirit of the claimed technology. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that the present disclosure covers such modifications and variations as come within the scope of the appended claims and their equivalents.
[0236] The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary ” is not necessarily to be construed as preferred or advantageous over other implementations. Additionally, unless specifically identified otherwise, all embodiments described herein should be considered exemplary.
[0237] The detailed description uses numerical and leter designations to refer to features in the drawings. Like or similar designations in the drawings and description have been used to refer to like or similar parts of the invention. As used herein, the terms “first”, “second”, and “third” may be used interchangeably to distinguish one component from another and are not intended to signify location or importance of the individual components.
Claims
WHAT IS CLAIMED IS:
1. An asset condition monitoring system, comprising:one or more processors;one or more databases that collectively store a plurality of machine-health diagnostic files and a plurality of asset identifiers (IDs) respectively associated with a plurality of machine assets; andone or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the asset condition monitoring system to perform operations, the operations comprising:obtaining a plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files;clustering the plurality7of machine-health diagnostic files into a plurality7of clusters based on the plurality of sets of metadata respectively associated with the plurality of machine-health diagnostic files; andassociating at least one of the plurality of clusters with one or more of the asset IDs associated with one or more of the plurality7of machine assets.
2. The asset condition monitoring system of claim 1. wherein:at least a subset of the plurality7of machine-health diagnostic files comprise a plurality of machine-health diagnostic images;each of the machine-health diagnostic images comprises a visible-light image, an acoustic image, or a thermal image; andeach of the machine-health diagnostic images depicts one or more of the machine assets or an area adjacent to the one or more of the machine assets.
3. The asset condition monitoring system of claim 2, wherein:the respective set of metadata associated each of the machine-health diagnostic images comprises a capture time associated with the machine-health diagnostic image; andclustering the plurality of machine-health diagnostic files comprises clustering the plurality of machine-health diagnostic images.
4. The asset condition monitoring system of claim 3, wherein clustering the plurality of machine-health diagnostic images comprises clustering the plurality’ of machine-health diagnostic images based on a user-defined hyperparameter of maximum time difference, the user-defined hyperparameter of maximum time difference specifying a maximum permissible time difference between capture times of any two machine-health diagnostic images contained in the same cluster.
5. The asset condition monitoring system of claim 2, wherein clustering the plurality of machine-health diagnostic files comprises:generating a plurality of image embeddings respectively for the plurality' of machine-health diagnostic images, wherein the respective image embedding for each machine-health diagnostic image comprises a numerical vector that represents visual features contained in the machine-health diagnostic image; andclustering the plurality7of machine-health diagnostic images based on the plurality of image embeddings.
6. The asset condition monitoring system of claim 1, wherein clustering the plurality7of machine-health diagnostic files comprises:generating a plurality of extracted text strings respectively for the plurality' of machine-health diagnostic files, wherein the respective extracted text strings for each machine-health diagnostic file comprises strings of text extracted from the machinehealth diagnostic file; andclustering the plurality of machine-health diagnostic files based on the plurality of extracted text strings.
7. The asset condition monitoring system of claim 6, wherein clustering the plurality’ of machine-health diagnostic files based on the plurality’ of extracted text strings comprises:determining a plurality of edit distances values between the plurality of extracted text strings: andclustering the plurality of machine-health diagnostic files based on the plurality7of edit distance values.
8. The asset condition monitoring system of claim 6, wherein clustering the plurality of machine-health diagnostic files based on the plurality of extracted text strings comprises:receiving a user input that specifies a particular text string;searching the extracted text strings for the plurality' of machine-health diagnostic files to identify any machine-health diagnostic files that contain the particular text string; andinserting into one of the plurality of clusters, any machine-health diagnostic files that were identified as containing the particular text string.
9. The asset condition monitoring system of claim 1, wherein clustering the plurality of machine-health diagnostic files comprises:generating, for each of the machine-health diagnostic files, multiple scores according to multiple different clustering criteria, the clustering criteria including asset locations and geospatial information associated with the plurality of machine assets;determining, for each of the machine-health diagnostic files, a score from the multiple scores; andclustering the plurality of machine-health diagnostic files based on the scores.
10. The asset condition monitoring system of claim 1, wherein clustering the plurality of machine-health diagnostic files comprises:providing a drop down menu that includes a number of clustering criteria; receiving a user input that selects one of the clustering criteria; and performing said clustering based on the selected one of the clustering criteria.
11. The asset condition monitoring system of claim 1, wherein the operations further comprise:providing a graphical user interface that graphically depicts the plurality of clusters of machine-health diagnostic files;receiving a user input that selects one of the clusters from the graphical user interface; andvisually presenting, within the graphical user interface, at least one of each individual machine-health diagnostic fde contained within the selected cluster, data associated with the machine-health diagnostic file, or content associated with the machine-health diagnostic file.
12. The asset condition monitoring system of claim 1, wherein associating the at least one of the plurality of clusters with the one or more of the asset IDs associated with the one or more of the plurality of machine assets comprises:providing a graphical user interface that enables a user to search for machine assets using one or more search strings;receiving one or more user inputs that define one or more search strings; identifying the one or more asset IDs based on the one or more search strings; andassociating the at least one of the plurality of clusters with the one or more of the asset IDs identified using the one or more search strings.
13. The asset condition monitoring system of claim 1, wherein associating the at least one of the plurality of clusters with the one or more of the asset IDs associated with the one or more of the plurality of machine assets comprises automatically associating the at least one of the plurality of clusters with the one or more of the asset IDs based on a comparison of the respective sets of metadata associated with the machine-health diagnostic files contained in the at least one of the plurality of clusters and metadata associated with the one or more of the plurality of machine assets.
14. A computer-implemented method to associate a plurality of asset-health diagnostic files and a plurality of asset identifiers (IDs) respectively associated with a plurality of assets, the computer-implemented method comprising:obtaining, by a computing system comprising one or more computing devices, a plurality of sets of metadata respectively associated with the plurality of asset-health diagnostic files;clustering, by the computing system, the plurality7of asset-health diagnostic files into a plurality7of clusters based on the plurality of sets of metadata respectively associated with the plurality of asset-health diagnostic files; andassociating, by the computing system, at least one of the plurality of clusters with one or more of the asset IDs associated with one or more of the plurality7of assets.
15. The computer-implemented method of claim 14, wherein:at least a subset of the plurality7of asset-health diagnostic files comprise a plurality7of asset-health diagnostic images;each of the asset-health diagnostic images comprising a visible-light image, an acoustic image, or a thermal image; andclustering, by7the computing system, the plurality of asset-health diagnostic files comprises clustering, by the computing system, the plurality of asset-health diagnostic images.
16. The computer-implemented method of claim 15, wherein clustering, by the computing system, the plurality7of asset-health diagnostic images comprises clustering, by the computing system, the plurality of asset-health diagnostic images based on a user-defined hyperparameter of maximum time difference, the user-defined hyperparameter of maximum time difference specifying a maximum permissible time difference between capture times of any two asset-health diagnostic images contained in the same cluster.
17. The computer-implemented method of claim 15, wherein clustering, by the computing system, the plurality of asset-health diagnostic files comprises:generating, by the computing system, a plurality7of image embeddings respectively for the plurality of asset-health diagnostic images, wherein the respective image embedding for each asset-health diagnostic image comprises a numerical vector that represents visual features contained in the asset-health diagnostic image; andclustering, by the computing system, the plurality7of asset-health diagnostic images based on the plurality7of image embeddings.
18. The computer-implemented method of claim 14, wherein clustering, by' the computing system, the plurality' of asset-health diagnostic files comprises:generating, by the computing system, a plurality of extracted text strings respectively for the plurality of asset-health diagnostic files, wherein the respective extracted text strings for each asset-health diagnostic file comprises strings of text extracted from the asset-health diagnostic file; andclustering, by the computing system, the plurality of asset-health diagnostic files based on the plurality of extracted text strings.
19. The computer-implemented method of claim 14, wherein clustering, by' the computing system, the plurality7of asset-health diagnostic files comprises:identifying, by the computing system, a search string associated with at least one of text, an image, a timestamp, or a location,wherein clustering, by the computing system, the plurality of asset-health diagnostic files comprises clustering, by the computing system, the plurality' of assethealth diagnostic files further based on the search string.
20. The computer-implemented method of claim 14, wherein clustering, by the computing system, the plurality of asset-health diagnostic files comprises:identifying, by the computing system, a plurality of locations associated with the plurality of assets, respectively;identifying at least two locations from among the plurality of locations that are separated from one another by less than a distance threshold; andclustering, by the computing system, at least two of the plurality7of assethealth diagnostic files associated with the at least two locations.
