Intelligent shaft connector identification method based on augmented reality interface

Abstract

The present invention provides an intelligent shaft connector identification method based on an augmented reality interface, comprising the following steps: providing a first interface to allow an operator to select real-time images of a flange from different perspectives, and establishing a virtual flange surface based on multiple real-time images; providing a second interface to allow the operator to select three circumferential points corresponding to the flange to establish a virtual flange model with a virtual outer diameter; providing a third interface to allow the operator to adjust the virtual hole diameter of the virtual flange model from the third interface; providing a fourth interface to allow the operator to adjust the virtual thickness of the virtual flange model; providing a fifth interface to allow the operator to input the number of bolts; and searching a database based on the virtual outer diameter, virtual hole diameter and virtual thickness of the virtual flange model to obtain a query result. The present invention can be combined with the augmented reality interface technology to automatically search the database to find the most likely flange specifications, eliminating the steps of disassembling and measuring the flange.

Description

Technical Field

[0001] The present invention relates to an intelligent shaft connector identification method, and more particularly to an intelligent shaft connector identification method based on an augmented reality interface. Background Art

[0002] Shaft connectors, such as flanges (or flanges), refer to components used to connect valves and fluid pipes in fluid conveying equipment. They are commonly found in semiconductor, chemical, and electronics manufacturing sites, as well as hospitals and water and gas for domestic use. The standard specifications of flanges vary from country to country. The main standard specifications can be divided into Japan (JIS), the European Union, the United States (ANSI), Australia (AS), and international standards (ISO). Flanges of different specifications and standards are not interchangeable. Therefore, when replacing valves and repairing equipment, the specifications of the flange must be determined before subsequent adjustments, replacements, and testing and resumption of work can be performed.

[0003] Old flanges may be rusted and worn, so that the specifications on the surface cannot be identified, or when there are no detailed records of specifications, maintenance personnel must first remove the flange from the pipeline and use measuring tools to determine its size and shape before selecting the correct component specifications for cutting and replacing it after the components arrive. If the flange is not removed for measurement, the dimensional parameters may not be obtained correctly because part of the flange area is obscured. The current operating method will cause the entire pipeline to be shut down twice, the first time to measure the flange specifications, and the second time for the actual replacement operation. For industries with uninterrupted fluid demand, such as technology plants and hospitals, each shutdown will cause a considerable degree of loss and even harm to public safety.

[0004] Patent No. TWI651661 discloses a computer program product for identifying flange specifications. The computer program product is used to load a computer program to execute data reading instructions and data analysis instructions. The data reading instructions enable the computer to read the flange data, and the data analysis instructions enable the computer to compare the flange data with the flange specification standard reference data to obtain flange specification analysis result data, wherein the flange data includes first to third flange size data, which are specific data about flange specifications. Although TWI651661 can identify flange specifications, the user needs to manually measure and input data during actual operation, and the fluency of use is not high. Although TWI651661 also mentions that the depth map in computer vision technology can be used for processing, the details are not disclosed. Therefore, the known technology still has considerable room for improvement to improve the fluency of use. Summary of the invention

[0005] Therefore, one object of the present invention is to provide an intelligent shaft connector identification method based on an augmented reality interface, so that on-site operators can use a handheld electronic device to identify the specifications of flanges installed in horizontal or vertical pipelines without dismantling equipment or pipelines, so as to facilitate maintenance and replacement.

[0006] To achieve the above-mentioned purpose, the present invention provides an intelligent shaft connector recognition method based on an augmented reality interface, which is applied to an electronic device. The recognition method includes: an environmental depth establishment step, providing a first interface with a real-time image representing a flange, so that an operator can select multiple real-time images of different perspectives of the flange to be processed, and obtain multiple sets of plane feature point information based on the multiple real-time images, so as to establish a virtual flange surface based on the multiple sets of plane feature point information; a circumferential point selection step, providing a superimposed virtual flange surface and the real-time image on a second interface, allowing the operator to select three circumferential points corresponding to the flange from the second interface, so as to obtain a virtual outer diameter based on the three circumferential points, so as to establish a virtual flange surface. A virtual flange model having an outer diameter and a virtual hole diameter; a hole diameter adjustment step, providing the superimposed virtual flange model and the real-time image on a third interface, allowing the operator to adjust the virtual hole diameter of the virtual flange model from the third interface; a flange thickness adjustment step, providing the superimposed virtual flange model and the real-time image on a fourth interface, allowing the operator to adjust a virtual thickness of the virtual flange model from the fourth interface; a bolt quantity input step, providing a fifth interface for the operator to input a bolt quantity; and a query step, searching a database according to the virtual outer diameter, virtual hole diameter and virtual thickness of the virtual flange model to obtain one or more query results of a physical flange matching the virtual flange model.

[0007] Through the above-mentioned embodiments, the augmented reality interface technology can be combined to provide auxiliary information after image processing, and cooperate with human spatial judgment ability to assist in quickly obtaining the three-dimensional dimensions including the outer diameter of the flange, the circular diameter of the flange hole, and the thickness of the flange, and automatically search the database to find the most likely flange specifications, eliminating the steps of disassembling and measuring the flange.

[0008] In order to make the above contents of the present invention more clearly understood, preferred embodiments are specifically cited below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A schematic diagram showing an application scenario of the identification method according to a preferred embodiment of the present invention.

[0010] Figure 2 A flow chart showing an identification method according to a preferred embodiment of the present invention.

[0011] Figures 3 to 14 show Figure 2 Schematic diagram of multiple interfaces provided by the identification method. Fig.15 show Figure 2 The query results of the identification method.

[0012] Reference numerals:

[0013] B1, B2: Bolt

[0014] P1, P2, P3: circumference points

[0015] S1: Environmental depth establishment steps

[0016] S2: Circumference point selection steps

[0017] S3: Hole diameter adjustment steps

[0018] S4: Flange thickness adjustment steps

[0019] S5: Bolt quantity input step

[0020] S6: Query step

[0021] S7: Archiving step

[0022] V1: Large Cylinder

[0023] V2: Small cylinder

[0024] VD1: Virtual outer diameter

[0025] VD2: Virtual hole diameter

[0026] VFM: Virtual Flange Model

[0027] VFS: Virtual Flange Face

[0028] VT: Virtual Thickness

[0029] 10: First interface

[0030] 11: First center mark

[0031] 15: Turntable

[0032] 16: Execute key

[0033] 20: Second interface

[0034] 21: Second center mark

[0035] 30: The third interface

[0036] 40: The fourth interface

[0037] 50: The fifth interface

[0038] 60: Display

[0039] 70: Camera

[0040] 80:CPU

[0041] 90: Database

[0042] 100: Electronic devices

[0043] 200: Flange

[0044] 201: Top

[0045] 210: Instant Image DETAILED DESCRIPTION

[0046] The present invention mainly utilizes the Augmented Reality (AR) interactive interface to identify the specifications of flanges without dismantling equipment and pipelines, thereby eliminating the need for the first shutdown and reducing the cost losses caused by the shutdown. Taking semiconductor chip factories as an example, water resources are extremely important to their production quality. In the manufacturing process alone, pure water must be used for chip pickling, organic cleaning, post-yellow light development cleaning, and mechanical grinding cleaning. If the production capacity performance is affected due to water outage, additional cost losses will be incurred for both semiconductor manufacturers and their downstream suppliers.

[0047] The technology of machine vision-based size recognition has been developed for many years, but most of them use a fixed perspective to analyze a single image. Considering that flanges are usually installed on walls, underground or ceilings, setting up tripods and cameras and calibrating them is not a practical method. In addition, the mutual shielding between flanges and pipelines or other devices makes it difficult to obtain images. Therefore, it is necessary to provide mobile measurement technology that is not restricted by the field and is easy to use for operations that require high mobility.

[0048] AR interactive interface technology provides the following advantages. First, there are fewer pre-works. Users only need a few seconds to identify the environment through intelligent applications to establish quite accurate spatial depth information. In addition, there is no need to set up cameras or perform calibration operations, which greatly shortens the operation time. Furthermore, flanges installed in actual fields are usually fixed on pipelines or equipment, and some parts will be blocked and shielded, making it impossible to effectively identify the size using traditional machine vision. In this case, the operation of the pipeline equipment must be stopped, the flange must be removed, and its specifications can only be determined after measuring its important dimensions. In contrast, the present invention can quickly identify the flange specifications without disassembling the flange through the prompt of intelligent virtual information, in conjunction with the results of user spatial cognition. According to previous studies, there are five parameters required for traditional identification of flange specifications, namely: outer diameter, hole diameter, thickness, number of bolts, and bolt hole diameter. The present invention only needs four necessary parameters (outer diameter, hole diameter, thickness, and number of screws) to identify the flange specifications through database analysis. The hole circle diameter represents the diameter of the hole circle on which the multiple bolt holes of the flange are located.

[0049] In actual applications, flanges appear in pairs, each at the end of a pipeline or valve. Several screws or bolts are used to fix the two flanges to achieve the effect of connecting two pipelines or a pipeline and a valve. The direction of the pipeline is usually divided into two types: "installed parallel to the ground" and "installed perpendicular to the ground". Regardless of the direction, the flange will be installed perpendicular to the pipeline so that the pipeline can pass through the central circular hole of the flange. Therefore, the technology provided by the present invention can also meet the application of the two pipeline directions.

[0050] Figure 1 A schematic diagram showing an application scenario of the identification method according to a preferred embodiment of the present invention. Figure 2 A flow chart showing a recognition method according to a preferred embodiment of the present invention is shown. Figure 1 and Figure 2As shown, the intelligent shaft connector identification method based on the augmented reality interface can be applied to an electronic device 100, such as a mobile phone, a tablet computer or a wearable device. The electronic device 100 at least includes a central processing unit (CPU) 80 and a display screen 60 electrically connected to the CPU 80, a camera 70 and a database 90. Of course, the database can also be located in the cloud, so that the electronic device 100 can connect to the cloud through the network for query. The electronic device 100 can install and execute the application program and then execute the identification method. The identification method includes an environmental depth establishment step S1, a circumferential point selection step S2, a hole diameter adjustment step S3, a flange thickness adjustment step S4, a bolt quantity input step S5 and a query step S6. Of course, in order to establish a wider database, the identification method can also include an archiving step S7.

[0051] In the environment depth establishment step S1, if Figure 3 As shown, the camera 70 is started to perform real-time photography, and a first interface 10 having a real-time image 210 representing a flange 200 is provided on the display screen 60. Therefore, an operator can select a plurality of real-time images 210 of different viewing angles of the flange 200 to be processed through the first interface 10, for example Figures 4 to 6 Therefore, the CPU 80 can obtain multiple sets of plane feature point information based on the multiple sets of plane feature point information to establish a virtual flange surface VFS based on the multiple sets of plane feature point information, such as Figure 7 shown.

[0052] In a non-limiting example, Figure 3As shown, the upper half of the first interface 10 provides a real-time image 210 and a first center mark 11, and the middle part of the first interface 10 also provides a horizontal pipeline option and a vertical pipeline option for the operator to click or check (in this case, the vertical pipeline is checked). The lower half of the first interface 10 also provides information such as the outer diameter, hole diameter, thickness, etc., and also provides a setting key, a reset key, a dial 15 and an execution key 16. The first interface 10 can provide information (such as a cross mark indicating placement at the center) to guide the operator to keep the first center mark 11 at a position between the two bolts B1 and B2, or the electronic device 100 can guide the operator through a speaker (not shown) to perform any of the above or the following operations. In addition, the first interface 10 can also guide the operator to surround the flange 200 with the electronic device 100 in three directions (such as the small icon displayed on the execution key 16) to obtain the multiple real-time images 210 of different viewing angles. The above three-way surround includes surround upward, surround rightward and surround leftward. Alternatively, the first center mark 11 of the first interface 10 can be used to guide the operator to align the first center mark 11 with a top surface 201 of the flange 200 when the electronic device 100 is three-dimensionally wrapped around the flange 200, so as to obtain the multiple real-time images 210 of different viewing angles. The virtual flange surface VFS is substantially located on the same plane as the top surface 201.

[0053] Therefore, the operator can operate the lens of the camera 70 of the electronic device 100 to stably keep the first center mark 11 in the area between the two bolts B1 and B2, and move the device around in all directions to collect plane feature point information from different perspectives (see Figures 4 to 6 ). This step is to enable the electronic device 100 to detect the real environment, collect relevant depth information, and thereby regress the plane position in space. After the plane feature point information is collected, the operator presses the execution key 16 of the first interface 10, and the virtual flange surface VFS ( Figure 7 ).

[0054] In the circumferential point selection step S2, the camera 70 is continuously activated, and the CPU 80 provides a superimposed virtual flange surface VFS and a real-time image 210 on a second interface 20 on the display screen 60, such as Figure 8 As shown, the operator is allowed to select three circumferential points P1, P2, and P3 ( Figure 8 , Fig. 9 and Fig.10 ). Thus, the CPU 80 can obtain a virtual outer diameter VD1 according to the three circumferential points P1, P2, and P3 to establish a virtual flange model VFM having a virtual outer diameter VD1 and a virtual hole diameter VD2, such as Fig.11As shown. Since the mathematical logic of determining a circle (defined by the center and diameter) by taking three points in space and on a plane is the same, it is just extended to the third dimension, and those skilled in the art can easily implement it, so it will not be described in detail here. Fig.11 In the embodiment, the virtual flange model VFM provided includes a large cylinder V1 and a small cylinder V2 which are overlapped with each other, the large cylinder V1 has a virtual outer diameter VD1, and the small cylinder V2 has a virtual hole diameter VD2.

[0055] In a non-limiting example, the second interface 20 has a second center mark 21 for guiding the operator to align the electronic device 100 with the second center mark 21 at three circumferential points P1, P2, P3 of the flange 200, wherein the three circumferential points P1, P2, P3 and the virtual flange surface VFS are located in the same plane.

[0056] Therefore, the operator uses the second center mark 21 in the second interface 20 as the alignment basis and presses the execution key 16 to select. Three different points can be selected on the edge of the flange on the virtual flange surface VFS and the real-time image 210 in order to determine the outer diameter parameters projected by the virtual flange model. The outer diameter obtained here is equal to 305.4 mm.

[0057] In the hole diameter adjustment step S3, the superimposed virtual flange model VFM and the real-time image 210 are provided on a third interface 30, such as Fig.11 As shown, the operator is allowed to adjust the virtual hole diameter VD2 of the virtual flange model VFM from the third interface 30, and the virtual flange model VFM can be selectively updated in real time according to the operator's adjustment, such as Fig.12 For example, the user can adjust the virtual hole diameter VD2 to 260.6 mm through the dial 15 to match the obtained real-time image 210, and confirm the adjustment result through the execution key 16. It is worth noting that when adjusting the hole diameter, the circumference of the bottom surface of the small cylinder V2 and the center point of the bolt / bolt hole can be made to fit together to determine the hole diameter parameter of the virtual flange model VFM projection.

[0058] In the flange thickness adjustment step S4, the operator can judge from the angle close to the side, and use the CPU 80, the camera 70 and the display screen 60 to provide a superimposed virtual flange model VFM and a real-time image 210 on a fourth interface 40, so that the operator can adjust a virtual thickness VT of the virtual flange model VFM from the fourth interface 40, and the virtual flange model VFM can be selectively updated in real time according to the operator's adjustment, such as Fig.13For example, the operator can adjust the thickness to 24.1 mm through the dial 15 to match the obtained real-time image 210, and confirm the adjustment result through the execution key 16 to determine the thickness parameter of the virtual flange model VFM projection. It is worth noting that the first interface 10, the second interface 20, the third interface 30 and the fourth interface 40 are all augmented reality interfaces.

[0059] In the bolt quantity input step S5, a fifth interface 50 is provided for the operator to input a bolt quantity, such as Fig.14 For example, the user selects the number of bolts as 12 by sliding on the fifth interface 50. It is worth noting that steps S4 and S5 can be moved to Figure 2 For example, steps S4 and S5 may be performed at the beginning, and the order of steps S4 and S5 is not particularly limited.

[0060] In the query step S6, after selecting the number of bolts as 12, the execution key 16 is pressed to query, and the CPU 80 searches the database 90 according to the virtual outer diameter VD1, the virtual hole diameter VD2 and the virtual thickness VT of the virtual flange model VFM to obtain one or more query results of the physical flange matching the virtual flange model VFM, such as Fig.15 As shown, multiple query results are ranked according to the matching scores.

[0061] In the archiving step S7 , the CPU 80 executes control to store one or more of the real-time images 210 of the flange 200 and the one or more query results in the database 90 , so that an additional reference data can be obtained during the next search.

[0062] It is worth noting that although the above example uses a mobile phone as the electronic device 100, a wearable device can also be used. The wearable device can track the operator's eye or head movements to allow the operator to perform selections and adjustments, and the same effect can be achieved.

[0063] The intelligent shaft connector identification method based on the augmented reality interface of the above embodiment can be combined with the augmented reality interface technology to provide auxiliary information after image processing, and cooperate with human spatial judgment ability to assist in quickly obtaining the three-dimensional dimensions including the outer diameter of the flange, the hole diameter of the flange (or the diameter of the pitch circle), and the thickness of the flange, and automatically search the database to find the most likely flange specifications, eliminating the steps of disassembling and measuring the flange. The present invention only uses the four most identifiable parameters of the flange, namely the outer diameter, hole diameter, thickness, and number of bolts, and can automatically determine the corresponding specifications from the database.

[0064] The specific embodiments proposed in the detailed description of the preferred embodiments are only used to facilitate the explanation of the technical content of the present invention, rather than narrowly limiting the present invention to the above embodiments. Various changes and implementations made without exceeding the spirit of the present invention and the scope of the patent application are all within the scope of the present invention.

Claims

1. An intelligent shaft connector identification method based on augmented reality interface, characterized in that: Applied in an electronic device, the identification method comprises: an environmental depth establishment step, providing a first interface having a real-time image representing a flange, so that an operator can select a plurality of real-time images of the flange to be processed at different viewing angles, obtain a plurality of sets of plane feature point information according to the plurality of real-time images, and establish a virtual flange surface according to the plurality of sets of plane feature point information; a circumferential point selection step, providing the superimposed virtual flange surface and the real-time image on a second interface, allowing the operator to select three circumferential points corresponding to the flange from the second interface, so as to obtain a virtual outer diameter based on the three circumferential points located in the same plane as the virtual flange surface, so as to establish a virtual flange model having the virtual outer diameter and a virtual hole diameter; a hole diameter adjustment step, providing the superimposed virtual flange model and the real-time image on a third interface, allowing the operator to adjust the virtual hole diameter of the virtual flange model from the third interface; a flange thickness adjustment step, providing the superimposed virtual flange model and the real-time image on a fourth interface, allowing the operator to adjust a virtual thickness of the virtual flange model from the fourth interface; a bolt quantity input step, providing a fifth interface for the operator to input a bolt quantity; and A query step is to search a database according to the virtual outer diameter, the virtual hole diameter and the virtual thickness of the virtual flange model to obtain one or more query results of the physical flange matching the virtual flange model.

2. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: Also includes: An archiving step stores one or more of the multiple real-time images of the flange and the one or more query results in the database, wherein in the hole diameter adjustment step and the flange thickness adjustment step, the virtual flange model is updated in real time based on the adjustment of the operator.

3. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: The first interface instructs the operator to move the electronic device around the flange in three directions to obtain the multiple real-time images at different viewing angles.

4. The intelligent shaft connector identification method based on augmented reality interface according to claim 3, characterized in that: The first interface has a first center mark for guiding the operator to align the first center mark with a top surface of the flange when the electronic device is rotated around the flange in three directions, so as to obtain the multiple real-time images of different viewing angles.

5. The intelligent shaft connector identification method based on augmented reality interface according to claim 4, characterized in that: The first interface guides the operator to keep the first center mark at a position between the two bolts.

6. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: The second interface has a second center mark to guide the operator to align the electronic device with the second center mark on the three circumferential points of the flange.

7. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: The first interface provides an option for a horizontal pipeline and an option for a vertical pipeline.

8. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: The multiple query results are ranked according to the matching scores.

9. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: On the third interface, the virtual flange model includes a large cylinder and a small cylinder superimposed on each other, the large cylinder has the virtual outer diameter, and the small cylinder has the virtual hole diameter.

10. The intelligent shaft connector identification method based on augmented reality interface according to claim 1, characterized in that: The electronic device tracks the operator's eye or head movements to allow the operator to perform selection and adjustment.

Images