System and method for determining thickness and / or diameter using computer vision

Abstract

A system and method for determining the dimensions of a component are disclosed. The method includes receiving one or more images, one or more videos, or a combination thereof, with respect to a caliper (100) having a plurality of markers (111, 113, 115). The one or more images, one or more videos, or a combination thereof are acquired by one or more image sensors (305) of a mobile device (301). The plurality of markers (111, 113, 115) includes markers respectively corresponding on each of a first measuring arm (101), a second measuring arm (103), and a pivot point of the caliper (100). The positions of the plurality of markers are determined by processing one or more images, one or more videos, or a combination thereof, with respect to the caliper (100) having the plurality of markers, via one or more processors. The distance between a first tip (107) of the first measuring arm (101) and a second tip (109) of the second measuring arm (103) is determined by processing the positions of the plurality of markers (111, 113, 115) via one or more processors.

Description

Technical Field

[0001] The present disclosure generally relates to systems and methods for determining thickness, and more particularly, to systems and methods for determining caliper measurements using computer vision.

Background Art

[0002] In many activities, it is necessary to accurately measure dimensions such as thickness or diameter, and benefits are received therefrom. For example, machine maintenance may include evaluating the wear amount of machine parts based on thickness measurements using, for example, calipers. In mobile work machines used at construction sites, since they can be exposed to very harsh conditions, some parts such as undercarriages may be subject to extreme wear such as scratches or abrasions. If worn parts are operated until they fail, in many cases, other parts that depend on the failed parts will also fail. If even one part fails, the operating cost of the entire machine will increase, so regular maintenance is important.

[0003] Typically, calipers include two measurement arms, and these measurement arms come into contact with both sides located on opposite sides in the physical structure of the part to be measured. Conventionally, the user measures the distance between the tips of the two measurement arms and manually records the measurement value, and thereby determines the degree of wear of the part, for example, by inputting the measurement value into the system. This process is generally time-consuming, inefficient, and prone to errors, especially when the user is operating under a tight schedule, there may not be enough time to obtain the necessary measurement values for the part. For example, it may be difficult and time-consuming to move back and forth between the calipers, the measurement device, and the recording instrument.

[0004] A method for measuring the size of an object using a laser sensor and a smartphone camera is disclosed in Korean Patent Application Publication No. KR20180056534A, published on May 29, 2018, for Hyoung et al. (hereinafter, the " '534 publication"). The method described in the '534 publication may be useful in some situations, but with a laser sensor and / or camera, it may be difficult to measure components with complex specifications or objects placed in difficult locations. Although high accuracy is required for such sensor-based measurements, a laser sensor and / or camera may not be suitable for accurately measuring components with complex designs, and due to being installed in inconvenient locations, misalignment of the sensor may occur. Furthermore, the method described in the '534 publication cannot solve the problems of being time-consuming, inefficient, and prone to errors, where the user has to move back and forth between a ruler, a measuring device, and a recording instrument to accurately measure components with complex designs, and it is installed in an inconvenient location.

[0005] The disclosed method and system can solve one or more of the above problems and / or other problems in the art. However, the scope of the present disclosure is defined by the appended claims and not by the ability to solve any particular problem. SUMMARY OF THE INVENTION

[0006] In one aspect, an exemplary embodiment of a system for determining the dimensions of a machine part may include a caliper, the caliper having a first measuring arm with a first tip and a second measuring arm connected to the first arm via a pivot point and having a second tip. The plurality of markers includes markers corresponding respectively on each of the first measuring arm, the second measuring arm, and the pivot point. A mobile device including one or more sensors may be configured to acquire an image or video, at least one memory may store instructions, and one or more processors are operably connected to the one or more sensors and to the at least one memory and, by executing the instructions, acquire, via the one or more sensors, one or more images, one or more videos, or a combination thereof, of a caliper having a plurality of markers; determine the positions of the plurality of markers by processing the one or more images, the one or more videos, or a combination thereof, of the caliper having the plurality of markers; and determine the distance between the first tip of the first measuring arm and the second tip of the second measuring arm by processing the positions of the plurality of markers.

[0007] In another aspect, an exemplary embodiment of a computer-implemented method for determining the dimensions of a machine part includes applying a caliper having a plurality of markers on a first measurement arm, a second measurement arm, and a pivot point to a part to be measured, obtaining, via one or more sensors of a mobile device, one or more images, one or more videos, or a combination thereof, of the caliper having the plurality of markers, determining the positions of the plurality of markers by processing, via one or more processors of the mobile device, the one or more images, one or more videos, or a combination thereof, of the caliper having the plurality of markers, and determining the distance between a first tip of the first measurement arm and a second tip of the second measurement arm by processing the positions of the plurality of markers via one or more processors of the mobile device.

[0008] In a further aspect, an exemplary embodiment of a method for determining the dimensions of a part includes receiving, with respect to a caliper having a plurality of markers, one or more images, one or more videos, or a combination thereof, where the one or more images, one or more videos, or a combination thereof are acquired by one or more image sensors of a mobile device and the plurality of markers include markers respectively corresponding to each of a first measurement arm, a second measurement arm, and a pivot point of the caliper, determining the positions of the plurality of markers by processing, via one or more processors, the one or more images, one or more videos, or a combination thereof, of the caliper having the plurality of markers, and determining the distance between a first tip of the first measurement arm and a second tip of the second measurement arm by processing the positions of the plurality of markers via one or more processors.

[0009] Other features and aspects of the present disclosure will become apparent from the following description and the accompanying drawings.

Brief Description of the Drawings

[0010] The accompanying drawings, which are incorporated herein and form a part of this specification, illustrate one or more embodiments and, together with the description, serve to explain the embodiments. The accompanying drawings are not necessarily drawn to scale. Further, any numerical values or dimensions in the accompanying drawings are for illustrative purposes only and may or may not represent actual values or dimensions, or preferred values or dimensions. In some cases, some or all of the selected features related to the basic features may not be illustrated to assist in the explanation and understanding. In some parts of the drawings, exemplary components or their dimensions are illustrated as being measured. However, it will be understood that other components and their dimensions not illustrated may also be measured. Any suitable components and their dimensions may be measured based on the techniques of this specification.

[0011]

Figure 1

Figures 2A - 2B

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

DETAILED DESCRIPTION OF THE INVENTION

[0012] Both the foregoing general description and the following detailed description are merely exemplary and explanatory and do not limit the claimed features. As used herein, the terms "comprises," "comprising," "has," "having," "includes," "including," or other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In the present disclosure, unless otherwise stated, relative terms, for example, terms such as "about," "substantially," and "approximately" are used to indicate possible variations of ±10% with respect to the stated value.

[0013] FIG. 1 is a schematic view of a caliper 100 according to an aspect of the present disclosure. Although the caliper 100 may include a pair of curved arms 101, 103, it will be understood that the arms 101, 103 may include any other shape as desired. The arms 101, 103 may be connected by a joint in the vicinity of the first end. In the embodiment illustrated in FIG. 1, the joint is a fastening member 105. However, other types of joints are also envisioned, such as an integral spring portion, a clip, a support member connected individually to each of the arms 101, 103, or any other suitable type of joint that defines a pivot point about which the arms 101, 103 can rotate. The fastening member 105 may pass through each hole in the arms 101, 103 and may define a pivot point. The arms 101, 103 may have second ends that are free ends. The second free ends of the arms 101, 103 may each include sharp tips 107, 109. In one example, the sharp tips 107, 109 may be configured to contact points located on opposite sides on the outer surface of the component, in which case the distance between the sharp tips 107, 109 may indicate an outer dimension of the component, such as a diameter. In another example, the sharp tips 107, 109 may contact a surface on a side located on the opposite side, in which case the distance between the sharp tips 107, 109 may indicate the thickness of the component. The caliper 100 may be used to measure an inner diameter, an outer diameter, a thickness, or any other suitable dimension, with respect to any suitable component in any suitable machine. It will be understood that any other variations regarding the caliper may be used, and in fact, any other known features regarding the caliper may be used to calculate the diameter or thickness of a machine part.

[0014] The caliper 100 may include a plurality of markers. For example, it may include marker 111 on the surface of arm 101, marker 113 on the surface of arm 103, and marker 115 on the surface of fastening member 105. Although the embodiment of FIG. 1 includes three markers 111, 113, and 115, any suitable number of markers may be used. As will be described in more detail below, at least three markers may be beneficial when using computer vision to determine the measured values of caliper 100. These markers may be formed or arranged at predetermined positions of caliper 100, but it will be understood that each marker may be formed at any other position of caliper 100. As will be described in more detail below, the positions of each marker on each of arms 101, 103 and at joint / fastening member 105 may further facilitate determining the measured values of caliper 100 using computer vision. In FIG. 1, the markers are shown as having a spherical shape, but these markers may be formed in any other shape as desired. The markers may be formed from rubber, plastic, or any other material known in the art. In some embodiments, the markers may be at least partially an integral member with respect to the caliper. For example, the markers may be etched, engraved, and / or embossed on caliper 100. In another embodiment, the markers may include stickers, logos, labels, inks, or any other surface markings that are suitable for being recognized by the image sensor of UE301.

[0015] In one example, these markers may include multiple colors. In some embodiments, the multiple colors may include colors having high contrast with each other, for example, black and white, or of the same kind. In one example, the marker may include a QR code, a data matrix code, a PDF417 barcode, an Aztec code, any other two-dimensional barcode, for example, the same code for each marker, or different codes for one or more markers. For example, markers 111 and 113 may be the same, and marker 115 may be different from them. As will be described in more detail below, markers having multiple high-contrast colors may further facilitate determining the measurement values of the calipers 100 using computer vision.

[0016] Figure 2A illustrates a scenario where a user measures the thickness of a machine part by using the caliper 100 according to an aspect of the present disclosure. In this example, although the undercarriage 201 of the track type machine 200 is being measured for part wear, it will be understood that any suitable dimension can be measured for any part in any other machine. These track type machines 200 can be exposed to harsh conditions that can cause extreme wear on the undercarriage 201. Unexpected part failures in the undercarriage 201 can cause various problems, for example, the maintenance operator may not be able to respond immediately, causing the machine to become inoperable until the repair is completed, or the stopped machine may be located in a place that is not easily accessible when performing the necessary maintenance, or the stopped machine may block the passage of other working machines, etc. Therefore, the parts of the undercarriage 201 require regular maintenance. As described above, the maintenance may include evaluating the amount of wear on the parts of the undercarriage 201 by the caliper 100. For example, the user 203 may measure the thickness of the parts of the undercarriage 201 by using the caliper 100. The relative positions of the arms 101, 103 may be adjusted until the tips 107, 109 contact points located on opposite sides on the surface of the parts of the undercarriage 201. The distance between the tips 107, 109 may indicate the thickness of the part. Figure 2B illustrates an enlarged view 205 of the use of the caliper 100 of Figure 2A when measuring the parts of the undercarriage 201. In this example, although the undercarriage of the track type machine is being measured, it will be understood that the caliper 100 can be used to measure the inner diameter, outer diameter, and thickness for any part in any other machine.

[0017] Referring to FIG. 3, a schematic diagram is shown of a system 300 for determining various quantities, such as the thickness of a component, component wear of machine 200, or the like, according to aspects of the present disclosure. System 300 includes user equipment (UE) 301a - 301n (collectively referred to as UE 301), an analysis platform 309, and a caliper 100. UE 301 includes or is associated with applications 303a - 303n (collectively referred to as applications 303) and sensors 305a - 305n (collectively referred to as sensors 305). In one example, analysis platform 309 has connectivity to machine 200, UE 301, and / or database 311 via a communication network 307, such as a wireless communication network. In some embodiments, machine 200 may not be connected to network 307.

[0018] UE 301 may include, but is not limited to, any type of mobile terminal, wireless terminal, or portable terminal. Examples of UE 301 include, but are not limited to, mobile phones, smartphones, wireless communication devices, webcams, laptops, personal digital assistants (PDAs), units, devices, multimedia computers, multimedia tablets, Internet nodes, communicators, personal communication system (PCS) devices, personal navigation devices, personal digital assistants (PDAs), digital cameras / camcorders, infotainment systems, or any combination including accessories and peripherals of these devices, or any combination of these. Additionally, UE 301 may facilitate various input means for receiving and generating information, including, but not limited to, touch - screen functionality, keypad data input, voice - based input mechanisms, and the like. Any known and future implementations related to UE 301 may also be applicable.

[0019] The application(s) 303 may include various applications, such as, but not limited to, a camera / imaging application, a content provisioning application, a network application, a multimedia application, a location-based application, a media player application, and the like. In one example, one of the applications 303 in the UE 301 may function as a client for the analytics platform 309 and interact with the analytics platform 309 via the communication network 307, e.g., via an application programming interface (API), to execute one or more functions of the analytics platform 309.

[0020] The sensor 305 includes an image sensor, such as a camera, configured to acquire image data. As an example, the sensor 305 may further include any other type of sensor. In one example, the sensor 305 may further include, for example, an orientation sensor reinforced by an altitude sensor and an acceleration sensor for determining the orientation of the UE 301, an inclinometer and a tilt meter for detecting the degree of inclination or declination of the UE 301, a depth sensor, an audio recorder for collecting audio data, a network detection sensor for detecting wireless signals or receivers related to different short-range communications (e.g., Bluetooth, Wi-Fi, Li-Fi, near-field communication (NFC), etc.), a global positioning sensor for collecting location data, a light sensor, and the like. Any known implementation and future implementation related to the sensor 305 may also be applicable.

[0021] The communication network 307 of system 300 may include one or more networks, for example, a data network, a wired network or a wireless network, a telephone network, or a combination thereof, etc. It is assumed that the data network can be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), public data network (e.g., the Internet), short-range wireless network, or any other suitable packet-switched network such as, for example, a proprietary cable, fiber optic network, or the like, such as a commercially-owned proprietary packet-switched network, or any combination thereof. Additionally, the wireless network may be, for example, a cellular network, and various technologies including 5G (fifth generation), 4G, 3G, 2G, long term evolution (LTE), enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., and also, for example, any other suitable wireless media such as worldwide interoperability for microwave access (WiMAX), code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (Wi-Fi), wireless local area network (WLAN), Bluetooth (registered trademark), Internet protocol (IP) data casting, satellite, mobile ad hoc network (MANET), and the like, or any combination thereof, may be adopted.

[0022] In one example, the analysis platform 309 may be a platform having a plurality of interconnected components. The analysis platform 309 may include one or more servers, intelligent networking devices, computing devices, components, and corresponding software for estimating component wear within a machine. Additionally, note that the analysis platform 309 may be a separate entity with respect to the system 300 or may be part of the UE 301. The analysis platform 309 may employ any known or currently under development methods, techniques, or processes for estimating component wear within a machine.

[0023] In one example, the analysis platform 309 may communicate with the control system 313 of the machine 200 via the communication network 307. The control system 313 may include servers, engine controllers, and sensors for monitoring the operation of various components of the machine 200. In particular, the control system 313 may be configured to sense the operating conditions of the machine 200 and may be configured to control the machine 200 by performing one or more estimations, calculations, modellings, and the like in response to the sensed operating conditions. The control system 313 may transmit information related to the operating status of various components of the machine 200 to the analysis platform 309 in real time.

[0024] In one example, the database 311 may be any type of database, such as relational, hierarchical, object-oriented, and / or the like, and in this case, the data is organized in any suitable manner, including data tables or lookup tables. In one example, the database 311 may store and manage multiple types of information, thereby providing means for assisting the content providing process and sharing process. The database 311 may include a machine learning-based training database having a predefined mapping that defines the relationship between various input parameters and output parameters based on various statistical methods. In one example, the training database may include a machine learning algorithm for learning the mapping between input parameters related to a video or image of a component, the estimated life of the component, the estimated time to complete wear value, and measured values such as thickness values, wear values, diameter information, etc. for various components. In one example, the training database is periodically updated and / or supplemented based on machine learning techniques.

[0025] FIG. 4 is a diagram regarding each component of the analysis platform 309 according to an exemplary embodiment. As used herein, terms such as "component" or "module" generally include hardware and / or software such that, for example, a processor or the like may be used to implement the related functions. By way of example, the analysis platform 309 includes one or more components for determining various quantities such as the thickness of components within a machine and component wear. It is contemplated that the functions of these components can be combined within one or more components or can be performed by other components having equivalent functions. In one example, the analysis platform 309 includes a data collection module 401, a calculation module 403, a data processing module 405, a user interface module 407, or any combination thereof.

[0026] In one example, the data collection module 401 may collect video or one or more images regarding the caliper 100 from the sensor 305 of the UE 301, such as an image sensor or a camera, for example, almost in real time or in real time. In another example, the data collection module 401 may collect related data, such as machine-specific information, measurement information of various parts of the machine, operating range, maintenance data of various parts of the machine, safety threshold levels, etc., using various data collection techniques, for example, in real time or almost in real time. For example, the data collection module 401 may collect related data related to various parts of the machine by accessing various databases or other information sources using a web crawling component. In another example, the data collection module 401 may include various software applications, such as a data mining application within an Extensible Markup Language (XML), that automatically search for and transmit related data regarding various parts of the machine. In one example, the data collection module 401 may analyze the data and organize it into a common format that can be easily processed by other modules and platforms.

[0027] In one example, the calculation module 403 may receive, in real time or substantially in real time, video or a plurality of images regarding the caliper 100 from the data collection module 401, and may identify the position of the marker by performing computer vision processing on the video or the plurality of images regarding the caliper 100. In another example, the calculation module 403 may receive previously acquired image data or video data. The calculation module 403 may calculate the distance between the tips of the caliper based on predetermined information regarding the caliper and based on the received data. For example, the calculation module 403 may, at least in part, be based on a predetermined distance between a plurality of markers on the caliper, based on the angle formed by the measurement arms of the caliper, based on a plurality of arcs formed between the predetermined distances, or based on a combination thereof, to determine the distance. In another example, the calculation module 403 may calculate the wear life of the component by tracking, in real time, the usage data of the component when the machine is operating, the maintenance data of the component, the distance calculated between the tips of the caliper, or a combination thereof. In another example, the calculation module 403 may calculate the wear rate of the component by comparing the distance calculated between the tips of the caliper with the calculated wear life of the component. In a further example, the calculation module 403 may indicate the operability of the component by determining a minimum thickness threshold, a safety threshold, a minimum wear rate threshold, or a combination thereof.

[0028] The data processing module 405 may process the data collected by the data collection module 401. In one example, the data processing module 405 may receive caliper-based measurements regarding the parts of the machine 200 from the calculation module 403, and determine the dimensional change of the part, for example, the wear rate of the part, by comparing the caliper-based measurement with the reference measurement stored in the database 311. The reference measurement may include, but is not limited to, original measurement value information regarding one or more parts, past measurement value information regarding one or more parts, minimum required measurement values regarding one or more parts, or combinations thereof. In another example, the data processing module 405 may determine whether the usage amount exceeds a threshold ratio of the estimated life of the part by processing the usage amount of the measured parts of the machine 200. In another example, the data processing module 405 may determine whether the state of the measured part of the machine 200 is below a safety threshold, below a predetermined operation threshold, or below a similar threshold by processing the maintenance data of the measured part of the machine 200.

[0029] In one example, the user interface module 407 may enable the display of a graphical user interface (GUI) within the UE 301. The user interface module 407 may employ various APIs or other function calls corresponding to applications on the UE 301, thereby enabling the display of graphic elements such as icons, menus, buttons, data input fields, etc. for generating user interface members. In one example, the user interface module 407 may include a data access interface configured to enable the user to access, configure, modify, store, and / or download information of the UE 301 or any other type of data device. For example, the user may change the operating parameters, operating range, or safety threshold levels related to one or more component configurations stored in the database 311. In another example, the user interface module 407 may include at least partially in the interface for guidance information to the user one or more annotations, text messages, audio messages, video messages, or combinations thereof. For example, the user interface module 407 may depict, by presenting audio / visual within the interface of the UE 301, for example, when the thickness of the engine measurement component does not meet the operating criteria or operating threshold, the measured thickness, metadata related to the measurement, and / or other data determined by the processing module 405.

[0030] The modules and components described above with respect to the analysis platform 309 may be implemented in hardware, firmware, software, or combinations thereof. Although illustrated as separate entities in FIG. 3, it is envisioned that the analysis platform 309 may be implemented to be directly operated by the respective corresponding UEs 301. Thus, the analysis platform 309 may directly generate signal inputs by the operating system of the UE 301. In another embodiment, one or more of the modules 401-407 may be implemented as, or in combination with, the analysis platform 309 for operation by the respective corresponding UEs. The various executions presented herein assume any configuration and model.

[0031] FIG. 5 is a diagram illustrating a user device for obtaining a video or image related to a measurement device for calculating a measurement value, e.g., a video or image related to the caliper 100 of FIG. 1, according to an aspect of the present disclosure. As shown in FIG. 5, the user 203 holds the camera of the UE 301 towards the caliper 100 used to measure parts of the machine 200. The analysis platform 309 may guide the user 203 over the process of obtaining a video or multiple images related to the caliper 100 by generating an audio / video notification within the user interface 501 of the UE 301. Such notifications may be based on various sensor-based measurement values, e.g., measurement values of an orientation sensor reinforced by an altitude sensor to determine the orientation of the UE 301, measurement values of an inclination sensor and an inclinometer to detect the degree of inclination or descent of the UE 301, etc. The analysis platform 309 may detect the position of each marker on the caliper 100 via the sensor 305 and may also obtain the known distances between those markers. The analysis platform 309 may also detect, via the sensor 305, the angle formed by the measuring arms of the caliper 100 and / or the measurement value of the caliper 100, e.g., the distance between the tips 107, 109.

[0032] FIG. 6 is a diagram related to an exemplary GUI for calculating the distance between the tips of the calipers 100 according to an aspect of the present disclosure. In one example, the calculation module 403 may implement one or more computer vision algorithms, machine learning algorithms, and / or deep learning algorithms to process and analyze the video and / or images acquired by the sensor 305 of the UE 301. The calculation module 403 may perform object identification by processing the acquired video and / or images to identify, for example, each marker on the calipers 100 to identify one or more objects. The processing may include using a computer vision algorithm or equivalent to recognize pixels corresponding to each object in the acquired video and / or images. The calculation module 403 may perform object classification by analyzing the visual content and classifying the identified objects into predefined categories, for example, to predefined positions regarding each marker. The calculation module 403 may also track their movement, for example, the movement of the calipers 100, by processing the video and / or image sequence, for example, in real time or near real time. The analysis platform 309 may perform calculations according to various formulas without any limitation. As shown in the user interface 601, the analysis platform 309 may acquire known distances 603, 605 from the markers 111, 113, 115. The analysis platform 309 may also acquire the angle 607 formed by the measurement arms 101, 103 at the pivot point. Thereafter, the analysis platform 309 may calculate the distances regarding the arcs 609, 611 based at least in part on the acquired distances and angles. For example, the three-dimensional configuration of the calipers 100 and / or the three-dimensional positions of the markers relative to each other may be determined by using the known distances 603 and 605, for example, in combination with the positions or relative positions of the markers determined via computer vision.By using such three-dimensional positions, other characteristics regarding the calipers 100 may be determined, such as the angles of the arms 101, 103, the distance between the tips 107, 109, the arcs between the markers 111, 113, or the like. Although FIG. 6 shows the arcs 609, 611, it will be understood that a plurality of arcs may be formed between the distances 603, 605 as desired. By utilizing the measurement of the arcs 609, 611, the distance between the tips 107, 109 may be calculated. In various embodiments, any suitable calculation, any suitable formula, or any suitable three-dimensional modeling or three-dimensional positioning may be used.

[0033] In one example, the analysis platform 309 may compare the distance between the tips 107, 109 with a reference measurement stored in the database 311, thereby determining, for example, the wear rate of the component. In another example, the analysis platform 309 may compare the wear rate of the component with an operating standard, a safety threshold level, a minimum thickness threshold, a threshold percentage of the estimated life of the component, or a combination thereof. The analysis platform 309 may determine that the component is not safe based at least in part on that comparison. To warn the user that the measured component of the machine 200 is not safe and needs to be replaced, an auditory / notification may be generated in the UE 301, for example, within the user interface 613, via the user interface module 407. In one example, the notification may include data describing information related to the measurement, such as the wear state of the component, for example, thickness reduction information, wear rate, amount of use of the component as a percentage of the total estimated life, estimated remaining operating life, whether the component should be removed from use, etc.

[0034] In one example, the analysis platform 309 may determine that the use of a component is approved, at least in part based on that comparison. For example, measurements regarding the component meet a reference measurement, a threshold requirement, or a combination thereof. As shown in FIG. 7, in order to notify the user that the measured component of the machine 200 is safe for use, an auditory / visual notification may be generated within the UE 301, for example within the user interface 701, via the user interface module 407. The notification may include data describing the maintenance status of the component, the current usage amount, the remaining rate of the estimated lifespan, and the like.

[0035] In various embodiments, one or more portions of the methods or techniques disclosed herein may be implemented, for example, within a chipset that includes a processor and a memory as shown in FIG. 8. FIG. 8 illustrates one implementation of a general computer system capable of executing the techniques presented herein. The computer system 800 may include a set of instructions executable to cause the computer system 800 to execute any one or more of the methods, systems, or computer-based functions disclosed herein. The computer system 800 may operate as a stand-alone device or may be connected, for example, using a network, to other computer systems or to peripheral devices.

[0036] In network deployment, computer system 800 may operate as a server, or as a client user computer in a server-client user network environment, or as a peer computer system in a peer-to-peer (i.e., distributed) network environment. Computer system 800 may also be implemented or incorporated as various devices such as, for example, a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a mobile device, a palmtop computer, a laptop computer, a desktop computer, a communication device, a wireless phone, a landline phone, a control system, a camera, a scanner, a facsimile machine, a personal creditworthiness device, a web appliance, a network router, a switch or bridge, or any other machine capable of executing a set of instructions (sequential or otherwise) specifying the actions to be performed by that machine. In a particular implementation, computer system 800 may be implemented using an electronic device that provides voice, video, or data communication. Further, although computer system 800 is illustrated as a single system, the term "system" is also interpreted to include any collection of systems or subsystems that individually or jointly execute a set of or multiple sets of instructions to perform one or more computer functions.

[0037] As shown in FIG. 8, computer system 800 may include a processor 802, which may include, for example, a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 802 may be configured as components in various systems. For example, the processor 802 may be part of a standard personal computer or a workstation. The processor 802 may be one or more general-purpose processors, digital signal processors, application-specific integrated circuits, field-programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other devices known now or developed later for analyzing and processing data. The processor 802 may implement software programs such as manually generated (i.e., programmed) code.

[0038] The computer system 800 may include a memory 804 that is communicable via a bus 808. The memory 804 may be a main memory, a static memory, or a dynamic memory. The memory 804 may include computer-readable storage media such as, but not limited to, various types of volatile and non-volatile storage media, including random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or magnetic disk, optical media, and the like. In one implementation, the memory 804 includes a cache or random access memory for the processor 802. In an alternative implementation, the memory 804 is separate from the processor 802, such as a cache memory of the processor, a system memory, or other memory. The memory 804 may be an external storage device or an external database for storing data. Examples include a hard drive, a compact disc (“CD”), a digital video disc (“DVD”), a memory card, a memory stick, a floppy disk, a universal serial bus (“USB”) memory device, or any other device that operates to store data. The memory 804 is operable to store instructions executable by the processor 802. Functions, acts, or tasks such as those illustrated in the drawings and described herein may be performed by the processor 802 executing instructions stored in the memory 804. The functions, acts, or tasks are independent of a particular type of instruction set, storage media, processor, or processing strategy and may be performed by software, hardware, integrated circuits, firmware, microcode, and the like, operating alone or in combination. Similarly, the processing strategy may include multiprocessing, multitasking, parallel processing, and the like.

[0039] As shown in the figure, computer system 800 may further include a display 810, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid state display, a cathode ray tube (CRT), a projector, a printer, or any other display device known currently or developed later for outputting the determined information, etc. The display 810 may function as an interface for the user to view the functions of the processor 802, specifically, as an interface for the software stored in the memory 804 or in the drive unit 806.

[0040] Additionally or alternatively, computer system 800 may include an input / output device 812 configured to enable the user to interact with any component of computer system 800. The input / output device 812 may be a numeric keypad, a keyboard, a cursor control device such as a mouse, a joystick, a touch screen display, a remote control, or any other device operable to interact with computer system 800.

[0041] Computer system 800 may also or alternatively include a drive unit 806 implemented as a disk drive or an optical drive. The drive unit 806 may include a computer-readable medium 822, such as software may be incorporated therein, so that one or more instruction sets 824 can be incorporated therein. Further, the instructions 824 may embody one or more of the methods or logics described herein. When executed by computer system 800, the instructions 824 may exist wholly or partially inside the memory 804 and / or inside the processor 802. The memory 804 and the processor 802 may also include a computer-readable medium as described above.

[0042] In some systems, the computer-readable medium 822 includes instructions 824 that respond to propagated signals so that a device connected to the network 870 can communicate voice, video, audio, images, or any other data over the network 870, or receives and executes such instructions 824. Further, the instructions 824 may be transmitted and received over the network 870 via a communication port or communication interface 820 and / or using the bus 808. The communication port or communication interface 820 may be part of the processor 802 or a separate component. The communication port or communication interface 820 may be created in software or physically connected in hardware. The communication port or communication interface 820 may be configured to connect to the network 870, to an external medium, to the display 810, or to any other component within the computer system 800, or a combination thereof. The connection to the network 870 may be a physical connection such as a wired Ethernet connection or may be established wirelessly as described below. Similarly, additional connections to other components of the computer system 800 may be physical or may be established wirelessly. The network 870 may alternatively be directly connected to the bus 808.

[0043] Although computer-readable medium 822 is shown as a single medium, the term "computer-readable medium" may include a single medium or a plurality of media, for example, a centralized or distributed database, and / or related caches and servers storing one or more instruction sets, etc. The term "computer-readable medium" may also be capable of storing, encoding, or carrying an instruction set for execution by a processor, or any medium that enables a computer system to execute any one or more of the methods or operations disclosed herein. Computer-readable medium 822 may be non-transitory and may be tangible.

[0044] Computer-readable medium 822 may include solid-state memory, for example, a memory card or other package containing one or more non-volatile read-only memories. Computer-readable medium 822 may be random access memory or other volatile rewritable memory. Additionally or alternatively, computer-readable medium 822 may include a magnetic-optical medium or an optical medium, such as a disk, tape, or other storage device for capturing a carrier signal, such as a signal communicated via a transmission medium. A digital file attachment to an email or other self-contained information archive or set of archives may be considered a distribution medium forming a tangible storage medium. Accordingly, the present disclosure is considered to include any one or more of computer-readable media, i.e., distribution media, in which data or instructions can be stored internally, and other equivalents and successor media.

[0045] In an alternative implementation, one or more of the methods described herein may be implemented by constructing dedicated hardware implementations such as application specific integrated circuits, programmable logic arrays, and other hardware devices. Applications that may include the apparatus and systems in various implementations may broadly include various electronic and computer systems. One or more implementations described herein may use two or more specific interconnected hardware modules or hardware devices that are communicable with related control signals and data signals across and through the modules, or as part of an application specific integrated circuit, to implement the functionality. Accordingly, the system encompasses software, firmware, and hardware implementations.

[0046] The computer system 800 may be connected to a network 870. The network 870 may define one or more networks, including a wired network or a wireless network. The wireless network may be a cellular phone network, 802.11, 802.16, 802.20, or a WiMAX network. Further, such networks may include a public network such as the Internet, a private network such as an intranet, or a combination thereof, and may utilize various networking protocols that are currently available or will be developed later, including but not limited to TCP / IP-based networking protocols. The network 870 may include a wide area network (WAN) such as the Internet, a local area network (LAN), a campus area network, a metropolitan area network, a direct connection such as via a Universal Serial Bus (USB) port, or any other network capable of enabling data communication. The network 870 may be configured to couple one computing device to another computing device to enable data communication between the devices. The network 870 may generally be capable of employing any form of machine-readable medium to communicate information from one device to another. The network 870 may include a communication method capable of moving information between computing devices. The network 870 may be divided into a plurality of sub-networks. The sub-network may enable access to all other components connected thereto, or the sub-network may restrict access between components. The network 870 may be regarded as a public network connection or a private network connection and may include, for example, a virtual private network, an encryption mechanism or other security mechanism employed on the public Internet, or the like.

[0047] It will be understood that each step in the described method may, in one embodiment, be performed by a suitable processor (or processors) in a processing (i.e., computer) system that executes instructions (computer-readable code) stored in storage. Also, it will be understood that the present disclosure is not limited to any particular implementation or programming technique, and that the present disclosure may be implemented using any suitable technique for implementing the functions described herein. The present disclosure is not limited to any particular programming language or operating system.

[0048] In the foregoing description of exemplary embodiments of the invention, it will be understood that various features of the invention may be grouped together in a single embodiment, drawing, or description thereof for purposes of simplifying the disclosure and aiding in the understanding of one or more of the various inventive aspects. However, this disclosure method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, the inventive aspects are less than all of the features in a single one of the above-described disclosed embodiments. Thus, the claims that follow the detailed description are hereby expressly incorporated herein into this detailed description, and each claim stands on its own as an individual embodiment of the invention.

Industrial Applicability

[0049] The disclosed method for measuring the thickness and / or wear of a component may be used with any suitable machine that is subject to various forms of wear and / or damage. The user may measure the thickness and / or diameter of a machine component by using a marked caliper. The analysis platform 309 may identify the markers and their positions and calculate the distance between the tips in the measuring arms of the caliper by processing a video or one or more images regarding the caliper in real time or near real time via computer vision techniques. The method according to the present disclosure for determining the thickness, diameter, and / or characteristics of a component related to the measurement may find particular applicability with machines operating in industries including construction, mining, agriculture, etc. Non-limiting examples of machines may include commercial machines, such as trucks, cranes, construction vehicles, mining vehicles, wheel loaders, backhoes, motor graders, tractor-type tractors, hydraulic excavators, tractor-type loaders, material handling equipment, agricultural machinery, and other types of mobile machines, etc.

[0050] FIG. 9 is a flowchart regarding a process for determining the distance between the tips in the measuring arms of a caliper according to an aspect of the present disclosure. In various embodiments, the analysis platform 309 and / or any of the modules 401-407 may perform one or more parts of the process 900 and may be implemented, for example, within a chipset including a processor and a memory as shown in FIG. 8. Thus, the analysis platform 309 and / or any of the modules 401-407 may provide means for achieving various parts of the process 900 and means for achieving embodiments of other processes described herein in combination with other components of the system 300. Although the process 900 is illustrated and described as a sequence of multiple steps, it is envisioned that various embodiments of the process 900 may be performed in any order or in any combination and need not include all of the steps illustrated.

[0051] In step 901, the analysis platform 309 may acquire a plurality of images of the caliper having a plurality of markers via one or more sensors. For example, the tips 107 and 109 of the caliper 100 may contact the surfaces on both sides located on the opposite side in the parts of the machine 200. The visual sensor of the UE 301, such as a camera, may acquire a video or a plurality of images regarding the caliper 100. The UE 301 may transmit the acquired video or plurality of images to the analysis platform 309 via the communication network 307, for example, in real time or substantially in real time. In some embodiments, the analysis platform 309 may be at least partially implemented on the UE 301. In some embodiments, the analysis platform 309 may communicate with the UE 301 by, for example, the network 870 or the like.

[0052] In step 903, the analysis platform 309 may determine the positions of the plurality of markers by processing one or more images of the caliper having a plurality of markers via one or more processors. For example, the analysis platform 309 may process and analyze the plurality of images received regarding the caliper 100 by implementing various computer vision algorithms, machine learning algorithms, and / or deep learning algorithms, so as to recognize the pixels corresponding to the visible features of the image, for example, the pixels corresponding to the positions of the markers of the caliper 100.

[0053] In step 905, the analysis platform 309 may determine the distance between the first tip of the first measurement arm and the second tip of the second measurement arm by processing the positions of the plurality of markers via one or more processors. The analysis platform 309 may implement various formulas for calculating the distance between the tips 107, 109 based at least in part on the known distance between the markers, the angle formed by the measurement arms, the plurality of arcs located between the known distances, or a combination thereof (as will be described in detail in FIGS. 3-7).

[0054] In one example, the analysis platform 309 may determine the operating characteristics of the component, such as the wear rate of the component, by comparing the distance between the first tip and the second tip with a reference measurement value. The analysis platform 309 may compare the calculated distance between the tips 107, 109 with a pre-determined operating criterion, safety threshold, minimum thickness threshold, minimum wear rate threshold, or a combination thereof. For example, the analysis platform 309 may determine that the wear rate is low when it is determined that the distance between the first tip and the second tip meets a pre-determined operating criterion. The analysis platform 309 may also determine that the wear rate is high when it is determined that the distance between the first tip and the second tip is below a pre-determined operating criterion.

[0055] In one example, the analysis platform 309 may generate a notification within the user interface of the device, at least in part based on the determined measurements and / or at least in part based on the determined operating characteristics for the component. In one example, the analysis platform 309 may warn the user to replace the component by generating a notification within the user interface of the UE301, e.g., in real time or substantially in real time. In another example, the analysis platform 309 may notify the user that the component is safe for use by generating a notification within the user interface of the UE301. The notification may include measurement information, metadata related to the measurement information, wear rate, the ratio of the usage amount to the total estimated life of the component, etc. (as described in detail in FIGS. 6 and 7).

[0056] Thus, the system 300 may assist in preventing productivity losses in the case where the component is not repaired / replaced or where the repair / replacement of the component is too early, by providing an accurate wear rate estimate for the component. One or more embodiments of the present disclosure may promote the efficient use of components by determining safety requirements and / or maintenance requirements, thereby assisting in preventing potential damage to other components of the machine that depends on the component. Thereby, the user experience can be improved and costs such as inspection costs and repair and maintenance costs can be reduced.

[0057] It will be apparent to those skilled in the art that various modifications and variations can be made to the present disclosure without departing from the scope of the disclosure. Other embodiments of the system will be apparent to those skilled in the art in view of the specification and practice of the system disclosed herein. The specification and examples are intended to be considered only as examples, and the true scope of the disclosure is indicated by the following claims and their equivalents.

Claims

1. A system for determining dimensions of parts of a machine (200), comprising: a caliper (100), a first measuring arm (101) having a first tip (107), a second measuring arm (103) connected to the first arm (101) via a pivot point and having a second tip (109), a plurality of markers (111, 113, 115), including markers respectively corresponding to each of the first measuring arm (101), the second measuring arm (103), and the pivot point, a mobile device (301), one or more sensors (305) configured to acquire images or videos, at least one memory storing instructions, one or more processors operatively connected to the one or more sensors (305) and the at least one memory, which, by executing the instructions, acquire, via the one or more sensors (305), one or more images, one or more videos, or a combination thereof, of the caliper (100) having the plurality of markers (111, 113, 115), determine positions of the plurality of markers (111, 113, 115) by processing the one or more images, the one or more videos, or a combination thereof, of the caliper (100) having the plurality of markers (111, 113, 115), and determine a distance between the first tip (107) of the first measuring arm (101) and the second tip (109) of the second measuring arm (103) by processing the positions of the plurality of markers (111, 113, 115),

2. Determining the distance between the first tip (107) and the second tip (109) comprises: determining based on a pre-specified distance between the positions of the plurality of markers (111, 113, 115), an angle (607) formed by the first measurement arm (101) and the second measurement arm (103) at the pivot point, and one or more arcs (609, 611) formed between the pre-specified distances, the system according to claim 1, further comprising.

3. The first measurement arm (101) and the second measurement arm (103) form the angle (607) and the plurality of arcs (609, 611) by rotating around the pivot point, the system according to claim 2.

4. The operation is further comprising comparing, via the one or more processors, the distance between the first tip (107) and the second tip (109) with one or more reference measurement values, wherein the one or more reference measurement values include past measurement data, past maintenance data, past usage data, or combinations thereof related to the component, the system according to any one of claims 1 to 3.

5. The operation is determining whether the distance between the first tip (107) and the second tip (109) meets a pre-determined safety threshold, minimum thickness threshold, minimum wear rate threshold, or a combination thereof; in response to this determination, in response to the distance between the first tip (107) and the second tip (109) meeting a pre-determined safety threshold, minimum thickness threshold, minimum wear rate threshold, or a combination thereof, generating a first notification within the user interface (701) of the mobile device (301) indicating that the component is operable, or in response to the distance between the first tip (107) and the second tip (109) not meeting a pre-determined safety threshold, minimum thickness threshold, minimum wear rate threshold, or a combination thereof, selectively generating a second notification within the user interface (613) of the mobile device (301) indicating that the component is not operable, the system according to claim 4, further comprising.

6. The operation is performed in real time or substantially in real time, the system according to claim 5.

7. With respect to the caliper (100), obtaining the one or more images, the one or more videos, or a combination thereof, further includes, via the one or more processors, generating a presentation regarding the caliper (100) within a user interface (501) of the mobile device (301), the presentation including one or more notifications at least partially based on an orientation sensor, an inclinometer, inclinometer data, or a combination thereof, for aligning the mobile device (301), the system according to any one of claims 1-6.

8. The first tip (107) is configured to contact a first surface of the component, the second tip (109) is configured to contact a second surface of the component, and the first surface and the second surface are located on surfaces located on opposite sides of the component, the system according to any one of claims 1-7.

9. The operation is tracking the usage data of the component in real time or substantially in real time, determining an estimated service life of the component by comparing the usage data with the distance between the first tip (107) and the second tip (109), and further including, when it is determined that the usage data exceeds a threshold percentage of the estimated service life of the component, generating a notification within a user interface (613) of the mobile device (301), the system according to any one of claims 1-8.

10. The distance between the first tip (107) and the second tip (109) indicates the thickness of the component, the system according to any one of claims 1-9.

Images