System and method for determining thickness and / or diameter using computer vision

Abstract

Systems and methods for determining dimensions of a component are disclosed. The method includes receiving one or more images, one or more videos, or a combination thereof, of a caliper (100) having a plurality of indicia (111, 113, 115). The one or more images, the one or more videos, or a combination thereof are captured by one or more image sensors (305) of the mobile device (301). The plurality of indicia (111, 113, 115) includes respective indicia on each of the first measurement arm (101), the second measurement arm (103), and the pivot point of the caliper (100). One or more images, one or more videos, or a combination thereof of a caliper (100) having a plurality of indicia (111, 113, 115) are processed via one or more processors to determine the position of the plurality of indicia. The positions of the plurality of markers (111, 113, 115) are processed via one or more processors to determine a distance between a first tip (107) of the first measurement arm (101) and a second tip (109) of the second measurement arm (103).

Description

Technical Field

[0001] The present disclosure relates generally to systems and methods for determining thickness, and more particularly to systems and methods for determining caliper measurements using computer vision. Background Art

[0002] Many activities require or benefit from accurate measurement of dimensions such as thickness or diameter. For example, maintenance of a machine may include assessing the amount of wear on a component of the machine based on thickness measurements, such as by using a caliper. Mobile work machines used on construction sites may be subject to extremely harsh conditions, resulting in extreme wear, such as scrapes and abrasions, on several components such as the chassis. Operating a worn component until it fails often results in the failure of other components that depend on the failed component. Failure of a single component may result in higher operating expenses for the overall operation of the machine, so regular maintenance is important.

[0003] Typically, a caliper includes two measuring arms that are brought into contact with opposing surfaces of the physical structure of the component to be measured. Conventionally, a user measures the distance between the tips of the two measuring arms and manually records the measurement, for example by entering the measurement into a system to determine the percentage of wear on the component. This process is often slow, inefficient, and prone to error, especially when the user's schedule is tight and may not have enough time to capture the required measurements of the component. For example, handling a caliper, a measuring device, and a recording device simultaneously may be difficult and / or time consuming.

[0004] A method for measuring the size of an object by using a laser sensor and a smartphone camera is disclosed in Korean Patent Application Publication No. KR20180056534A (“the '534 Publication”), published on May 29, 2018, to Hyoung et al. Although the method described in the '534 Publication may be useful in some cases, the laser sensor and / or camera may have difficulty measuring parts with complex specifications and positioned in challenging locations. Such sensor-based measurements require high precision, however, the laser sensor and / or camera may not be suitable for accurately measuring parts with complex designs and located in inconvenient locations that cause misalignment with the sensor. In addition, the method described in the '534 Publication may not solve the slow, inefficient, and error-prone problem that a user must simultaneously handle a caliper, a measuring device, and a recording instrument to accurately measure parts with complex designs and located in inconvenient locations.

[0005] The disclosed methods and systems may solve one or more of the problems set forth above and / or other problems in the art. However, the scope of the present disclosure is defined by the appended claims, rather than by the ability to solve any particular problem. Summary of the invention

[0006] In one aspect, an exemplary embodiment of a system for determining the size of a component of a machine may include a caliper having a first measuring arm and a second measuring arm, the first measuring arm having a first tip, the second measuring arm connected to the first arm via a pivot point and having a second tip. A plurality of markings include respective markings on each of the first measuring arm, the second measuring arm, and the pivot point. A mobile device includes one or more sensors that may be configured to capture images or videos; at least one memory may store instructions; and one or more processors operably connected to the one or more sensors and the at least one memory may be configured to execute the instructions to perform operations, the operations including: capturing one or more images, one or more videos, or a combination thereof of the caliper having the plurality of markings via the one or more sensors; processing one or more images, one or more videos, or a combination thereof of the caliper having the plurality of markings to determine the locations of the plurality of markings; and processing the locations of the plurality of markings to determine the distance between the first tip of the first measuring arm and the second tip of the second measuring arm.

[0007] On the other hand, an exemplary embodiment of a computer-implemented method for determining the dimensions of a component of a machine may include: applying a caliper having multiple markings on a first measuring arm, a second measuring arm, and a pivot point to the component to be measured; capturing one or more images, one or more videos, or a combination thereof of the caliper having the multiple markings via one or more sensors of a mobile device; processing one or more images, one or more videos, or a combination thereof of the caliper having the multiple markings via one or more processors of the mobile device to determine the positions of the multiple markings; and processing the positions of the multiple markings via one or more processors of the mobile device to determine the distance between the first tip of the first measuring arm and the second tip of the second measuring arm.

[0008] On the other hand, an exemplary embodiment of a method for determining the size of a component may include: receiving one or more images, one or more videos, or a combination thereof of a caliper having a plurality of markings, the one or more images, the one or more videos, or a combination thereof being captured by one or more image sensors of a mobile device, the plurality of markings including corresponding markings on each of a first measuring arm, a second measuring arm, and a pivot point of the caliper; processing the one or more images, the one or more videos, or a combination thereof of the caliper having the plurality of markings via one or more processors to determine the positions of the plurality of markings; and processing the positions of the plurality of markings via the one or more processors to determine the distance between a first tip of the first measuring arm and a second tip of the second measuring arm.

[0009] Other features and aspects of the present disclosure will become apparent from the following description and accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more embodiments and, together with the description, explain the embodiments. The drawings are not necessarily drawn to scale. In addition, any values ​​or dimensions in the drawings are for illustrative purposes only and may or may not represent actual or preferred values ​​or dimensions. Where applicable, some or all selected features may not be described to help describe and understand the basic features. In some drawings, the dimensions of exemplary parts or parts are shown as measured. However, it should be understood that the dimensions of other parts or parts not shown may also be measured. Any suitable parts or dimensions thereof may be measured according to the technology herein.

[0011] Figure 1 is a schematic diagram of a caliper according to aspects of the present disclosure.

[0012] Figure 2A A user according to aspects of the present disclosure uses Figure 1 A scene in which a caliper 100 measures the thickness of a component of a machine 200 .

[0013] Figure 2B An enlarged view of measuring a component using a caliper 100 is shown in accordance with aspects of the present disclosure.

[0014] Figure 3 is a diagram of a system for component wear estimation in accordance with aspects of the present disclosure.

[0015] Figure 4 is a diagram of components of an analysis platform 309 in accordance with aspects of the present disclosure.

[0016] Figure 5 is a diagram showing a user device capturing video or images of a measurement device, for example via an image sensor, in real-time or near real-time to calculate measurement values ​​according to aspects of the present disclosure.

[0017] Figure 6 is a diagram of an exemplary model for calculating the distance between tips of a measurement device according to aspects of the present disclosure.

[0018] Figure 7 is a diagram of a user interface according to aspects of the present disclosure, in which a user is informed of the operability of a component of a machine based on measurement information.

[0019] Figure 8 is a flow chart of a process for determining the distance between the tips of the measuring arms of the caliper 100 in accordance with aspects of the present disclosure.

[0020] Fig. 9An embodiment of a general computer system is shown that can perform the techniques presented herein. DETAILED DESCRIPTION

[0021] Both the foregoing general description and the following detailed description are exemplary and illustrative only and do not limit the claimed features. As used herein, the terms "comprise," "comprising," "has," "having," "include," "including," or other variations thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. In this disclosure, unless otherwise indicated, relative terms, such as, for example, "about," "substantially," and "approximately," are used to indicate a possible variation of ±10% of the stated value.

[0022] Figure 1 is a schematic diagram of a caliper 100 according to various aspects of the present disclosure. The caliper 100 may include a pair of curved arms 101 and 103, however, it should be understood that the arms 101 and 103 may include any other shape as desired. The arms 101 and 103 may be connected by a joint near a first end. Figure 1 In the illustrated embodiment, the joint is a fastener 105. However, other types of joints are also contemplated, such as an integral spring portion, a clip, a support member separately connected to each of the arms 101 and 103, or any other suitable type of joint defining a pivot point about which the arms 101 and 103 can rotate. The fastener 105 can pass through a hole in the arms 101 and 103, respectively, and can define a pivot point. The arms 101 and 103 can have a second free end. The second free ends of the arms 101 and 103 can include sharp tips 107 and 109, respectively. In one example, the sharp tips 107 and 109 can be configured to contact diametrically opposed points on an outer surface of the component, whereby the distance between the sharp tips 107 and 109 can indicate an outer dimension of the component, such as a diameter. In another example, the sharp tips 107 and 109 can contact surfaces on opposite sides, and the distance between the sharp tips 107 and 109 can indicate the thickness of the component. The caliper 100 can be used to measure the inner diameter, outer diameter, thickness or any other suitable dimension of any suitable component of any suitable machine. It should be understood that any other variation of the caliper can be used, and indeed any other known features of the caliper can be used to calculate the diameter or thickness of a component of a machine.

[0023] The caliper 100 may include a plurality of markings, for example, markings 111, 113, and 115 on the surfaces of the arm 101, the arm 103, and the fastener 105, respectively. Figure 1The embodiment in includes three markings 111, 113, and 115, but any suitable number of markings may be used. As discussed in further detail below, at least three markings may be beneficial for determining the measurement of caliper 100 using computer vision. These markings may be formed on or located at predetermined locations of caliper 100, however, it should be understood that the markings may be formed at any other location of caliper 100. As discussed in further detail below, the location of the markings on each of arms 101 and 103 and at joint / fastener 105, respectively, may further facilitate the determination of the measurement of caliper 100 using computer vision. Figure 1 In the embodiment, the markings are shown as having a spherical shape, however, the markings may be formed in any other shape as desired. The markings may be formed of rubber, plastic, or any other material known in the art. In some embodiments, the markings may be at least partially integrated with the caliper, for example, the markings may be etched, engraved, and / or stamped on the caliper 100. In another embodiment, the markings may include stickers, logos, labels, inks, or any other surface markings suitable for recognition by the image sensor of the UE 301.

[0024] In one example, the markings may include multiple colors. In some embodiments, the multiple colors may include colors that have high contrast to each other, such as black and white, etc. In one example, the markings may include a QR code, a data matrix code, a PDF417 barcode, an Aztec code, any other two-dimensional barcode, for example, the same code for each marking or different codes for one or more markings. For example, markings 111 and 113 may be the same, and marking 115 may be different from them. As discussed in further detail below, markings with multiple high contrast colors may further facilitate the use of computer vision to determine the measurement of the caliper 100.

[0025] Figure 2AA scenario is shown in which a user according to aspects of the present disclosure measures the thickness of a component of a machine by using a caliper 100. In this example, the wear of a component of a chassis 201 of a crawler machine 200 is measured, however, it should be understood that any suitable dimension of any component of any other machine can be measured. These crawler machines 200 may be subjected to harsh conditions, which may cause extreme wear on the chassis 201. Any unexpected component failure of the chassis 201 may cause various problems, for example, the maintenance operator may not be available at all times, and the machine may not be able to operate before repair, the stopped machine may be in a position that is not easily accessible to perform the required maintenance, the stopped machine may block the passage of other working machines, etc. Therefore, the components of the chassis 201 require regular maintenance. As previously discussed, maintenance may include assessing the amount of wear on the components of the chassis 201 by the caliper 100. For example, the user 203 can use the caliper 100 to measure the thickness of the component of the chassis 201. The relative positions of the arms 101 and 103 can be adjusted until the tips 107 and 109 contact opposite points on the surface of the component of the chassis 201. The distance between tips 107 and 109 may indicate the thickness of the component. Figure 2B Describes the use Figure 2A 205 of an enlarged view of a caliper 100 measuring a component of a chassis 201. In this example, the chassis of a track-type machine is measured, however, it should be understood that the caliper 100 can be used to measure the inner diameter, outer diameter, and thickness of any component of any other machine.

[0026] refer to Figure 3 , shows a schematic diagram of a system 300 for determining various quantities (e.g., thickness of a component, component wear), etc. of a machine 200 according to various aspects of the present disclosure. The system 300 includes user equipment (UE) 301a-301n (collectively referred to as UE301), an analysis platform 309, and a caliper 100. The UE 301 includes or is associated with applications 303a-303n (collectively referred to as applications 303) and sensors 305a-305n (collectively referred to as sensors 305). In one example, the analysis platform 309 has connectivity with the machine 200, the UE 301, and / or a database 311 via a communication network 307 (e.g., a wireless communication network). In some embodiments, the machine 200 may not be connected to the network 307.

[0027] UE 301 may include, but is not limited to, any type of mobile terminal, wireless terminal, or portable terminal. Examples of UE 301 may include, but are not limited to, mobile handheld devices, smart phones, wireless communication devices, webcams, laptop computers, personal digital assistants (PDAs), units, devices, multimedia computers, multimedia tablet computers, Internet nodes, communicators, personal communication systems (PCS) devices, personal navigation devices, personal digital assistants (PDAs), digital cameras / camcorders, infotainment systems, or any combination thereof, including accessories and peripherals of these devices, or any combination thereof. In addition, UE 301 may facilitate various input methods for receiving and generating information, including, but not limited to, touch screen capabilities, keyboard data input, voice-based input mechanisms, etc. Any known and future implementations of UE 301 may also be applicable.

[0028] Application(s) 303 may include various applications, such as, but not limited to, camera / imaging applications, content provision applications, network applications, multimedia applications, location-based applications, media player applications, etc. In one example, one of applications 303 at UE 301 may act as a client of analysis platform 309 and may interact with analysis platform 309 over communication network 307, for example, via an application programming interface (API) to perform one or more functions of analysis platform 309.

[0029] The sensor 305 includes an image sensor, such as a camera configured to capture image data. For example, the sensor 305 may also include any other type of sensor. In one example, the sensor 305 may also include, for example, an orientation sensor enhanced with an altitude sensor and an acceleration sensor to determine the orientation of the UE 301, an inclination sensor and an inclinometer for detecting the tilt or tilt degree of the UE 301, a depth sensor, an audio recorder for collecting audio data, a network detection sensor for detecting wireless signals or a receiver for different short-range communications (e.g., Bluetooth, Wi-Fi, Li-Fi, near field communication (NFC), etc.), a global positioning sensor for collecting location data, a light sensor, etc. Any known and future implementations of the sensor 305 may also be applicable.

[0030] The communication network 307 of the system 300 may include one or more networks, such as a data network, a wired or wireless network, a telephone network, or any combination thereof. It is contemplated that the data network may be any local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a public data network (e.g., the Internet), a short-range wireless network, or any other suitable packet-switched network, such as a commercially owned private packet-switched network, such as a dedicated cable or fiber optic network, or the like, or any combination thereof. In addition, the wireless network may be, for example, a cellular network, and may employ various technologies, including 5G (5th Generation), 4G, 3G, 2G, Long Term Evolution (LTE), Enhanced Data Rates for Global Evolution (EDGE), General Packet Radio Service (GPRS), Global System for Mobile Communications (GSM), Internet Protocol Multimedia Subsystem (IMS), Universal Mobile Telecommunications System (UMTS), etc., as well as any other suitable wireless medium, such as Worldwide Interoperability for Microwave Access (WiMAX), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Wireless Fidelity (Wi-Fi), Wireless LAN (WLAN), Internet Protocol (IP) data broadcasting, satellite, mobile ad hoc network (MANET), etc., or any combination thereof.

[0031] In one example, the analysis platform 309 may be a platform with multiple interconnected components. The analysis platform 309 may include one or more servers, intelligent network devices, computing devices, components, and corresponding software for estimating component wear in a machine. In addition, it should be noted that the analysis platform 309 may be a separate entity of the system 300 or a part of the UE 301. The analysis platform 309 may employ any known or still developing method, technology, or process for estimating component wear in a machine.

[0032] In one example, analysis platform 309 may communicate with control system 313 of machine 200 via communication network 307. Control system 313 may include servers, engine controllers, and sensors that monitor the operation of various components of machine 200. In particular, control system 313 may be configured to sense the operating state of machine 200, and in response to the sensed operating state, perform one or more estimations, calculations, modeling, etc. to control machine 200. Control system 313 may transmit information about the operating state of various components of machine 200 to analysis platform 309 in real time.

[0033] In one example, database 311 can be any type of database, such as relational, hierarchical, object-oriented, etc., where data is organized in any suitable manner, including as a data table or a lookup table. In one example, database 311 can store and manage multiple types of information, which can provide means for assisting the content provision and sharing process. Database 311 can include a machine learning-based training database having a predefined mapping that defines the relationship between various input parameters and output parameters based on various statistical methods. In one example, the training database can include a machine learning algorithm that learns the mapping between input parameters related to videos or images of components, estimated life of components, and estimated time to full wear values ​​and measured values ​​of various components (e.g., thickness values, wear values, diameter information, etc.). In one example, the training database is regularly updated and / or supplemented based on a machine learning method.

[0034] Figure 4 is a diagram of components of analysis platform 309 according to an example embodiment. As used herein, terms such as "component" or "module" generally encompass hardware and / or software, such as a processor, that can be used to implement the associated functionality. For example, analysis platform 309 includes one or more components for determining various quantities in a machine (e.g., thickness of a component, wear of a component), etc. It is contemplated that the functions of these components may be combined in one or more components or performed by other components with equivalent functionality. In one example, analysis platform 309 includes data collection module 401, calculation module 403, data processing module 405, and user interface module 407, or any combination thereof.

[0035] In one example, the data collection module 401 can collect a video or one or more images of the caliper 100 from a sensor 305 (e.g., an image sensor, a camera, etc.) of the UE 301, for example, in near real time or real time. In another example, the data collection module 401 can collect relevant data, such as machine-specific information, measurement information of various components of the machine, operating ranges, maintenance data of various components of the machine, safety threshold levels, etc., through various data collection techniques, such as in real time or near real time. For example, the data collection module 401 can use a web crawler component to access various databases or other information sources to collect relevant data related to various components of the machine. In another example, the data collection module 401 can include various software applications, such as data mining applications in an extended meta-language (XML), which automatically search and return relevant data about various components of the machine. In one example, the data collection module 401 can parse the data and arrange the data into a common format that can be easily processed by other modules and platforms.

[0036] In one example, the computing module 403 may receive a video or multiple images of the caliper 100 from the data collection module 401 in real time or near real time, and may perform computer vision processing on the video or multiple images of the caliper 100 to identify the location of the mark. In another example, the computing module 403 may receive previously captured image or video data. The computing module 403 may calculate the distance between the tips of the caliper based on predetermined information about the caliper and the received data. For example, the computing module 403 may determine the distance based at least in part on a predetermined distance between multiple marks on the caliper, an angle formed by a measuring arm of the caliper, a plurality of arcs formed between predetermined distances, or a combination thereof. In another example, the computing module 403 may calculate the wear life of the component by tracking the usage data of the component during the operation of the machine in real time, the maintenance data of the component, the calculated distance between the tips of the caliper, or a combination thereof. In another example, the computing module 403 may calculate the wear percentage of the component by comparing the calculated distance between the tips of the caliper and the calculated wear life of the component. In yet another example, the computing module 403 can determine a minimum thickness threshold, a safety threshold, a minimum wear percentage threshold, or a combination thereof to indicate the operability of the component.

[0037] The data processing module 405 can process the data collected by the data collection module 401. In one example, the data processing module 405 can receive caliper-based measurements of components of the machine 200 from the calculation module 403, and can compare the caliper-based measurements with reference measurements stored in the database 311 to determine the dimensional change of the component, such as the wear percentage of the component. The reference measurements may include, but are not limited to, original measurement information of one or more components, past measurement information of one or more components, minimum required measurements of one or more components, or a combination thereof. In another example, the data processing module 405 can process the usage of the measured components of the machine 200 to determine whether the usage exceeds a threshold percentage of the estimated life of the component. In another example, the data processing module 405 can process maintenance data of the measured components of the machine 200 to determine whether the status of the measured components is below a safety threshold, below a predetermined operating threshold, etc.

[0038] In one example, the user interface module 407 can implement the presentation of a graphical user interface (GUI) in the UE 301. The user interface module 407 can use various APIs or other function calls corresponding to the application on the UE 301 to enable the display of graphical primitives, such as icons, menus, buttons, data input fields, etc., to generate user interface elements. In one example, the user interface module 407 may include a data access interface configured to allow a user to access, configure, modify, store and / or download information to the UE 301 or any other type of data device. For example, the user can modify the operating parameters, operating ranges or safety threshold levels associated with one or more component configurations stored in the database 311. In another example, the user interface module 407 can interface the guidance information with the user to include at least in part one or more annotations, text messages, audio messages, video messages or a combination thereof. For example, the user interface module 407 can present audio / visual in the interface of the UE 301 to depict, for example, the measured thickness, metadata associated with the measurement and / or other data determined by the processing module 405, such as when the thickness of the measured component of the engine does not meet the operating standard or threshold.

[0039] The modules and components of the analysis platform 309 presented above may be implemented in hardware, firmware, software, or a combination thereof. Figure 3 309, but it is contemplated that the analysis platform 309 may be implemented to be directly operated by the respective UE 301. Thus, the analysis platform 309 may generate direct signal inputs with the aid of the operating system of the UE 301. In another embodiment, one or more of the modules 401-407 may be implemented to be operated by the respective UE or a combination thereof as the analysis platform 309. The various executions presented herein encompass any and all arrangements and modes.

[0040] Figure 5 is a diagram depicting a capture measurement device (e.g. Figure 1 A video or image of the caliper 100) is provided to a user device for calculating the measurement value. Figure 5, the user 203 holds the camera of the UE 301 toward the caliper 100, which has been used to measure a part of the machine 200. The analysis platform 309 can generate an audio / video notification in the user interface 501 of the UE 301 to guide the user 203 through the process of capturing a video or multiple images of the caliper 100. Such notifications can be based on various sensor-based measurements, such as an orientation sensor enhanced with an altitude sensor to determine the orientation of the UE 301, a tilt sensor and an inclinometer to detect the tilt or decline of the UE 301, etc. The analysis platform 309, through the sensor 305, can detect the location of the markers on the caliper 100 and obtain the known distances between these markers. The analysis platform 309, through the sensor 305, can also detect the angle formed by the measuring arm of the caliper 100 and / or the measurement value of the caliper 100, such as the distance between the tips 107 and 109.

[0041] Figure 6 is a diagram of an exemplary GUI for calculating the distance between the tips of the caliper 100 according to various aspects of the present disclosure. In one example, the computing module 403 can implement one or more computer vision algorithms, machine learning algorithms, and / or deep learning algorithms to process and analyze the video and / or images captured by the sensor 305 of the UE 301. The computing module 403 can perform object recognition by processing the captured video and / or images to identify one or more objects, such as markers on the caliper 100. The processing can include, for example, using a computer vision algorithm or an equivalent algorithm to identify pixels corresponding to objects in the captured video and / or images. The computing module 403 can perform object classification by parsing the visual content and classifying the identified objects into predefined categories, such as predetermined locations of markers. The computing module 403 can also process the video and / or image sequence, for example, in real time or near real time to track its movement, such as the movement of the caliper 100. The analysis platform 309 can perform calculations according to various equations without any limitation. As depicted in user interface 601, analysis platform 309 may obtain known distances 603 and 605 from markers 111, 113, and 115. Analysis platform 309 may also obtain an angle 607 formed by measuring arms 101 and 103 at a pivot point. Analysis platform 309 may then calculate distances of arcs 609 and 611 based at least in part on the obtained distances and angles. For example, known distances 603 and 605 may be used, for example, in conjunction with the positions or relative positions of the markers determined via computer vision, to determine a three-dimensional configuration of caliper 100 and / or the three-dimensional positions of the markers relative to each other. Such three-dimensional positions may be used to determine other aspects of caliper 100, such as the angle of arms 101 and 103, the distance between tips 107 and 109, the arc between markers 111 and 113, and the like. Figure 6Arcs 609 and 611 are shown, however, it should be understood that multiple arcs may be formed as desired between distances 603 and 605. The measurements of arcs 609 and 611 may be used to calculate the distance between tips 107 and 109. In various embodiments, any suitable calculation, equation, or three-dimensional modeling or positioning may be used.

[0042] In one example, the analysis platform 309 may compare the distance between the tips 107 and 109 with reference measurements stored in the database 311, for example, to determine a percentage of wear of the component. In another example, the analysis platform 309 may compare the percentage of wear of the component with an operating standard, a safety threshold level, a minimum thickness threshold, a threshold percentage of an estimated life of the component, or a combination thereof. The analysis platform 309 may determine that the component is unsafe based at least in part on the comparison. An audio / visual notification may be generated in the UE 301 (e.g., the user interface 613) via the user interface module 407 to alert the user that the measured component of the machine 200 is unsafe and needs to be replaced. In one example, the notification may include data describing information related to the measurement (e.g., the wear state of the component, such as thickness reduction information), the percentage of wear, the usage of the component as a percentage of its total estimated life, an estimate of the remaining useful life, whether the component should be stopped from use, etc.

[0043] In one example, the analysis platform 309 can determine that the component is approved for use based at least in part on the comparison. For example, the measurement value of the component meets the reference measurement value, the threshold requirement, or a combination thereof. Figure 7 As depicted in , an audio / visual notification can be generated in UE 301 (e.g., user interface 701) via user interface module 407 to remind the user that the measured component of machine 200 is safe to use. The notification may include data describing the maintenance status of the component, current usage, estimated percentage of remaining life, etc.

[0044] In various embodiments, one or more portions of the methods or techniques disclosed herein may be included, for example, in Figure 8 The processor and memory shown in are implemented in a chipset. Figure 8 An embodiment of a general computer system that can perform the techniques presented herein is shown. The computer system 800 may include a set of instructions that can be executed to cause the computer system 800 to perform any one or more of the methods, systems, or computer-based functions disclosed herein. The computer system 800 can operate as a stand-alone device, or can be connected to other computer systems or peripheral devices, for example using a network.

[0045] In a networked deployment, the computer system 800 can operate in the capacity of a server in a server-client user network environment or as a client user computer, or in a peer-to-peer (or distributed) network environment as a peer computer system. The computer system 800 can also be implemented as or incorporated into various devices, such as personal computers (PCs), tablet PCs, personal digital assistants (PDAs), mobile devices, palmtop computers, laptop computers, desktop computers, communication devices, wireless phones, fixed phones, control systems, cameras, scanners, fax machines, personal trusted devices, network devices, network routers, switches or bridges, or any other machine capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the machine. In a specific embodiment, the computer system 800 can be implemented using an electronic device that provides voice, video or data communication. In addition, although the computer system 800 is shown as a single system, the term "system" should also be considered to include any collection of systems or subsystems that execute one or more sets of instructions to perform one or more computer functions, either individually or together.

[0046] like Figure 8 As shown in , computer system 800 may include a processor 802, such as a central processing unit (CPU), a graphics processing unit (GPU), or both. Processor 802 may be a component in a variety of systems. For example, processor 802 may be part of a standard personal computer or workstation. Processor 802 may be one or more general-purpose processors, digital signal processors, application-specific integrated circuits, field programmable gate arrays, servers, networks, digital circuits, analog circuits, combinations thereof, or other now known or later developed devices for analyzing and processing data. Processor 802 may implement a software program, such as a manually generated (i.e., programmed) code.

[0047] The computer system 800 may include a memory 804 that can communicate via a bus 808. The memory 804 may be a main memory, a static memory, or a dynamic memory. The memory 804 may include, but is not limited to, computer-readable storage media, such as various types of volatile and non-volatile storage media, including but not limited to random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, tape or disk, optical media, etc. In one embodiment, the memory 804 includes a cache or random access memory for the processor 802. In an alternative embodiment, the memory 804 is separated from the processor 802, such as a cache memory, system memory, or other memory of the processor. The memory 804 may be an external storage device or database for storing data. Examples include a hard drive, a compact disk ("CD"), a digital video disk ("DVD"), a memory card, a memory stick, a floppy disk, a universal serial bus ("USB") memory device, or any other device operable to store data. The memory 804 is operable to store instructions that can be executed by the processor 802. The functions, actions, or tasks shown in the figures or described herein may be performed by the processor 802 executing instructions stored in the memory 804. The functions, actions, or tasks are independent of a particular type of instruction set, storage medium, processor, or processing strategy, and may be performed by software, hardware, integrated circuits, firmware, microcode, etc. operating alone or in combination. Similarly, processing strategies may include multi-processing, multi-tasking, parallel processing, etc.

[0048] As shown, the computer system 800 may also include a display 810, such as a liquid crystal display (LCD), an organic light emitting diode (OLED), a flat panel display, a solid state display, a cathode ray tube (CRT), a projector, a printer, or other display devices known now or developed later for outputting determined information. The display 810 may be used as an interface for a user to view the functions of the processor 802, or specifically as an interface with software stored in the memory 804 or the drive unit 806.

[0049] Additionally or alternatively, the computer system 800 may include an input / output device 812 configured to allow a user to interact with any component of the computer system 800. The input / output device 812 may be a numeric keypad, a keyboard or a cursor control device (e.g., a mouse or joystick), a touch screen display, a remote control, or any other device operable to interact with the computer system 800.

[0050] The computer system 800 may also or alternatively include a drive unit 806 implemented as a disk or optical drive. The drive unit 806 may include a computer-readable medium 822, in which one or more sets of instructions 824, such as software, may be embedded. In addition, the instructions 824 may include one or more of the methods or logics described herein. During execution by the computer system 800, the instructions 824 may reside in whole or in part within the memory 804 and / or within the processor 802. The memory 804 and the processor 802 may also include computer-readable media as discussed above.

[0051] In some systems, the computer-readable medium 822 includes instructions 824, or receives and executes instructions 824 in response to a propagated signal, so that a device connected to the network 870 can transmit voice, video, audio, image, or any other data through the network 870. In addition, instructions 824 can be transmitted or received through the network 870 via a communication port or interface 820 and / or using a bus 808. The communication port or interface 820 can be part of the processor 802, or it can be a separate component. The communication port or interface 820 can be created in software, or it can be a physical connection in hardware. The communication port or interface 820 can be configured to connect to the network 870, external media, display 810, or any other component or combination thereof in the computer system 800. The connection to the network 870 can be a physical connection, such as a wired Ethernet connection, or it can be established wirelessly as discussed below. Similarly, additional connections to other components of the computer system 800 can be physical connections or can be established wirelessly. The network 870 can be directly connected to the bus 808 instead.

[0052] Although the computer-readable medium 822 is shown as a single medium, the term "computer-readable medium" may include a single medium or multiple media, such as a centralized or distributed database, and / or associated caches and servers that store one or more sets of instructions. The term "computer-readable medium" may also include any medium that can store, encode, or carry a set of instructions to be executed by a processor or cause a computer system to perform any one or more of the methods or operations disclosed herein. The computer-readable medium 822 may be non-transitory and may be tangible.

[0053] The computer readable medium 822 may include solid-state memory, such as a memory card or other package that accommodates one or more non-volatile read-only memories. The computer readable medium 822 may be a random access memory or other volatile rewritable memory. Additionally or alternatively, the computer readable medium 822 may include magneto-optical or optical media, such as a disk or tape or other storage device to capture carrier signals, such as signals transmitted through a transmission medium. Digital file attachments to e-mails or other self-contained information archives or archives may be considered to be distribution media as tangible storage media. Therefore, the present disclosure is considered to include any one or more of computer-readable media or distribution media and other equivalents and successor media in which data or instructions may be stored.

[0054] In alternative embodiments, dedicated hardware implementations (e.g., application specific integrated circuits, programmable logic arrays, and other hardware devices) may be configured to implement one or more of the methods described herein. Applications of the devices and systems that may include various implementations may broadly include various electronic and computer systems. One or more embodiments described herein may use two or more specific interconnected hardware modules or devices (which have associated control and data signals that may be transmitted between and through the modules) to implement the functions, or may be implemented as part of an application specific integrated circuit. Therefore, the present system includes software, firmware, and hardware implementations.

[0055] The computer system 800 can be connected to a network 870. The network 870 can define one or more networks including wired or wireless networks. The wireless network can be a cellular telephone network, an 802.11, 802.16, 802.20, or WiMAX network. In addition, such a network can include a public network (e.g., the Internet), a private network (e.g., an intranet), or a combination thereof, and can utilize various networking protocols that are now available or developed later, including but not limited to TCP / IP-based networking protocols. The network 870 can include a wide area network (WAN) (e.g., the Internet), a local area network (LAN), a campus network, a metropolitan area network, a direct connection such as through a universal serial bus (USB) port, or any other network that can allow data communication. The network 870 can be configured to connect one computing device to another computing device to enable data communication between the devices. The network 870 can generally be enabled to use any form of machine-readable media to transmit information from one device to another. The network 870 can include a communication method by which information can be propagated between computing devices. The network 870 can be divided into subnetworks. A subnetwork may allow access to all other components connected thereto, or a subnetwork may restrict access between components. Network 870 may be considered a public or private network connection, and may include, for example, a virtual private network or encryption or other security mechanisms used over the public Internet.

[0056] It will be appreciated that, in one embodiment, the steps of the discussed method are performed by an appropriate processor (or processors) of a processing (i.e., computer) system executing instructions (computer readable code) stored in a memory. It will also be appreciated that the present disclosure is not limited to any particular implementation or programming technique, and that the present disclosure may be implemented using any suitable technique for implementing the functionality described herein. The present disclosure is not limited to any particular programming language or operating system.

[0057] It should be understood that in the above description of exemplary embodiments of the present invention, for the purpose of simplifying the present disclosure and aiding in the understanding of one or more of the various inventive aspects, the various features of the present invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting the intention that the claimed invention requires more features than those explicitly recited in each claim. Rather, as reflected in the following claims, each inventive aspect consists in less than all the features of a single aforementioned disclosed embodiment. Therefore, the claims following the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim existing independently as a separate embodiment of the present invention.

[0058] Industrial Applicability

[0059] The disclosed method for determining the thickness and / or wear of a component can be used on any suitable machine that is subject to various forms of wear and / or damage. A user can use a caliper with a mark to measure the thickness and / or diameter of a component of a machine. The analysis platform 309 can process a video or one or more images of the caliper through computer vision technology in real time or near real time to identify the mark and its location, and calculate the distance between the tips of the measuring arms of the caliper. The disclosed method for determining the thickness, diameter and / or characteristics of a component associated with a measurement value can be particularly suitable for machines operating in industries including construction, mining, agriculture, etc. Non-limiting examples of machines may include commercial machines such as trucks, cranes, earth moving vehicles, mining vehicles, wheel loaders, backhoes, motor graders, crawler tractors, hydraulic excavators, crawler loaders, material handling equipment, agricultural equipment, and other types of movable machines.

[0060] Fig. 9 900 is a flow chart of a process for determining the distance between the tips of the measuring arms of a caliper according to aspects of the present disclosure. In various embodiments, analysis platform 309 and / or any of modules 401-407 may perform one or more portions of process 900 and may include, for example, Figure 8900. Thus, analysis platform 309 and / or any of modules 401-407 may provide means for completing portions of process 900, as well as means for implementing embodiments of other processes described herein in conjunction with other components of system 300. Although process 900 is shown and described as a sequence of steps, it is contemplated that various embodiments of process 900 may be performed in any order or combination, and need not include all of the steps shown.

[0061] In step 901, the analysis platform 309 may capture multiple images of a caliper with multiple markings via one or more sensors. For example, the tips 107 and 109 of the caliper 100 may contact surfaces on opposite sides of a component of the machine 200. A visual sensor of the UE 301, such as a camera, may capture a video or multiple images of the caliper 100. The UE 301 may transmit the captured video or multiple images to the analysis platform 309 via the communication network 307, for example, in real time or near real time. In some embodiments, the analysis platform 309 may be implemented at least in part on the UE 301. In some embodiments, the analysis platform 309 may communicate with the UE 301, for example, via a network 870 or the like.

[0062] In step 903, the analysis platform 309 may process one or more images of the caliper having the plurality of markings via one or more processors to determine the locations of the plurality of markings. For example, the analysis platform 309 may implement various computer vision algorithms, machine learning algorithms, and / or deep learning algorithms to process and analyze the received plurality of images of the caliper 100 to identify pixels of the images corresponding to visible features of the caliper 100 (e.g., the locations of the markings).

[0063] In step 905, the analysis platform 309, via one or more processors, can process the positions of the plurality of markers to determine a distance between the first tip of the first measuring arm and the second tip of the second measuring arm. The analysis platform 309, via one or more processors, can implement various equations to calculate the distance between the tips 107 and 109 based at least in part on the known distances between the markers, the angle formed by the measuring arms, the plurality of arcs between the known distances, or a combination thereof (e.g., as Figure 3-7 Detailed discussion in ).

[0064] In one example, the analysis platform 309 can determine an operational characteristic of a component, such as a percentage of wear of the component, by comparing the distance between the first tip and the second tip to a reference measurement. The analysis platform 309 can compare the calculated distance between the tips 107 and 109 to a predetermined operational criterion, a safety threshold, a minimum thickness threshold, a minimum wear percentage threshold, or a combination thereof. For example, the analysis platform 309 can determine a low wear percentage when it is determined that the distance between the first tip and the second tip meets the predetermined operational criterion. The analysis platform 309 can also determine a high wear percentage when it is determined that the distance between the first tip and the second tip is below the predetermined operational criterion.

[0065] In one example, the analysis platform 309 can generate a notification in a user interface of the device based at least in part on the determined measurement of the component and / or the determined operating characteristics of the component. In one example, the analysis platform 309 can generate a notification in a user interface of the UE 301 to remind a user to replace the component, for example, in real time or near real time. In another example, the analysis platform 309 can generate a notification in a user interface of the UE 301 to remind a user that the component is safe to use. The notification can include measurement information, metadata associated with the measurement information, a percentage of wear, usage of the component as a percentage of its total estimated life, etc. (e.g., Figure 6 and Figure 7 Detailed discussion in ).

[0066] In this way, the system 300 can help prevent productivity loss when a component is not repaired / replaced or is repaired / replaced prematurely by providing an accurate wear percentage estimate of the component. One or more embodiments of the present disclosure can promote efficient use of components by determining safety and / or maintenance requirements, thereby helping to prevent potential damage to other parts of the machine that rely on the component. This can result in an improved user experience and reduced costs, such as inspection costs, repair and maintenance costs, etc.

[0067] It will be apparent to those skilled in the art that various modifications and changes may be made to the disclosed system without departing from the scope of the present disclosure. Other embodiments of the system will be apparent to those skilled in the art through careful consideration of the description and practice of the system disclosed herein. The specification and examples are intended to be considered as exemplary only, with the true scope of the present disclosure being indicated by the following claims and their equivalents.

Claims

1. A system for determining the size of a component of a machine (200), comprising: A caliper (100), the caliper comprising: A first measuring arm (101), the first measuring arm having a first tip (107); a second measuring arm (103) connected to the first arm (101) via a pivot point and having a second tip (109); and a plurality of markings (111, 113, 115), the plurality of markings comprising respective markings on each of the first measuring arm (101), the second measuring arm (103) and the pivot point; and A mobile device (301), the mobile device comprising: one or more sensors (305) configured to capture images or video; at least one memory storing instructions; and One or more processors operably connected to the one or more sensors (305) and the at least one memory and configured to execute the instructions to perform operations comprising: capturing, via the one or more sensors (305), one or more images, one or more videos, or a combination thereof, of the caliper (100) having the plurality of markings (111, 113, 115); processing one or more images, one or more videos, or a combination thereof of the caliper (100) having the plurality of markers (111, 113, 115) to determine the positions of the plurality of markers (111, 113, 115); and The positions of the plurality of markings (111, 113, 115) are processed to determine the distance between the first tip (107) of the first measuring arm (101) and the second tip (109) of the second measuring arm (103).

2. The system of claim 1, wherein determining the distance between the first tip (107) and the second tip (109) further comprises: Based on a predefined distance between the positions of the plurality of markings (111, 113, 115), an angle (607) formed by the first measuring arm (101) and the second measuring arm (103) at the pivot point and one or more arcs (609, 611) formed between the predefined distances are determined.

3. The system according to claim 2, wherein the first measuring arm (101) and the second measuring arm (103) rotate around the pivot point to form the angle (607) and the plurality of arcs (609, 611).

4. The system of any preceding claim, wherein the operations further comprise: The distance between the first tip (107) and the second tip (109) is compared with one or more reference measurements via the one or more processors, wherein the one or more reference measurements include past measurement data, past maintenance data, past usage data, or a combination thereof, related to the component.

5. The system of claim 4, wherein the operations further comprise: Determining whether the distance between the first tip (107) and the second tip (109) satisfies a predetermined safety threshold, a minimum thickness threshold, a minimum wear percentage threshold, or a combination thereof; as well as In response to the determination, optionally: In response to the distance between the first tip (107) and the second tip (109) satisfying the predetermined safety threshold, the minimum thickness threshold, the minimum wear percentage threshold, or a combination thereof, generating a first notification in a user interface (701) of the mobile device (301) indicating that the component is in an operable state; or In response to the distance between the first tip (107) and the second tip (109) not satisfying the predetermined safety threshold, the minimum thickness threshold, the minimum wear percentage threshold, or a combination thereof, a second notification is generated in the user interface (613) of the mobile device (301) indicating that the component is not in an operable state.

6. The system of claim 5, wherein the operations are performed in real time or near real time.

7. The system of any preceding claim, wherein capturing one or more images, one or more videos, or a combination thereof of the caliper (100) further comprises: Generating, via the one or more processors, a presentation of the caliper (100) in a user interface (501) of the mobile device (301), wherein the presentation includes one or more notifications of aligning the mobile device (301) based at least in part on an orientation sensor, a tilt sensor, an inclinometer data, or a combination thereof.

8. A system according to any preceding claim, wherein the first tip (107) is configured to contact a first surface of the component and the second tip (109) is configured to contact a second surface of the component, and wherein the first surface and the second surface are on opposite sides of the component.

9. The system of any preceding claim, wherein the operations further comprise: Tracking usage data of the component in real time or near real time; comparing the usage data to a distance between the first tip (107) and the second tip (109) to determine an estimated useful life of the component; as well as Upon determining that the usage data exceeds a threshold percentage of the estimated useful life of the component, a notification is generated in a user interface (613) of the mobile device (301).

10. The system according to any preceding claim, wherein the distance between the first tip (107) and the second tip (109) is indicative of the thickness of the component.

Images