Information processing device and information processing method

Abstract

[Problem] To provide an information processing device capable of generating an isometric diagram without preparing a three-dimensional model of a plant in advance. [Solution] This information processing device 4 is a device for supporting the management of a plant including piping and piping members, attached to the piping, as components. The information processing device 4 comprises: a data acquisition unit 400 that acquires point group data 11 obtained by measuring a plant with a three-dimensional measurement device; an object data processing unit 401 that specifies three-dimensional positions of piping from the point group data 11 and generates piping objects 14A including piping position information indicating the three-dimensional positions of the piping; and a drawing generation unit 402 that specifies connection relationships between three-dimensional positions of piping nodes, indicating curving points or bending points of the piping, and piping line segments, connecting the piping nodes, on the basis of the piping position information included in the piping objects 14A, and generates an isometric drawing 15B of the piping by connecting the piping nodes with drawing line segments according to the connection relationships.

Description

Information Processing Apparatus and Information Processing Method

[0001] The present invention relates to an information processing apparatus and an information processing method.

[0002] For various purposes such as plant construction and maintenance, a three-dimensional model capable of reproducing the three-dimensional shape of each component of a plant is used. For example, Patent Document 1 discloses a system that grasps the position information of each facility from a photographed image of a plant facility and displays a virtual site in which a three-dimensional model drawing of each facility is superimposed on the photographed image based on the grasped position information.

[0003] International Publication No. 2023 / 132555

[0004] The 3D modeling database used in the system disclosed in Patent Document 1 is a three-dimensional model created by 3D CAD at the time of plant design. Therefore, a three-dimensional model is required as a premise of the system. However, for example, in an existing plant, a three-dimensional model may not have been created at the time of design, or even if a three-dimensional model has been created, the renovation work of the plant may not be reflected in the three-dimensional model, resulting in a situation where it does not match the actual site of the plant. In such a situation, it has been difficult to generate accurate drawings even when trying to generate various drawings such as isometric drawings from the three-dimensional model of the plant.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide an information processing apparatus and an information processing method that enable the generation of an isometric drawing without preparing a three-dimensional model of a plant in advance.

[0006] To achieve the above objective, an information processing device according to one aspect of the present invention is an information processing device that supports the management of a plant including piping and piping members attached to the piping as constituent elements, comprising: a data acquisition unit that acquires point cloud data measured by a three-dimensional measuring device of the plant; an object data processing unit that identifies the three-dimensional position of the piping from the point cloud data and generates a piping object that includes piping position information indicating the three-dimensional position of the piping; and a drawing generation unit that, based on the piping position information included in the piping object, identifies the three-dimensional position of piping nodes indicating bending points or inflection points of the piping and the connection relationship of piping line segments connecting the piping nodes, and generates an isometric drawing of the piping by connecting the piping nodes with drawing line segments according to the connection relationship.

[0007] According to one aspect of the present invention, an information processing device can generate an isometric diagram without having to prepare a three-dimensional model of the plant in advance.

[0008] Other issues, configurations, and effects will be clarified in the embodiments for carrying out the invention described later.

[0009] This is an overall diagram showing an example of the plant management support system 1 and plant 10. This is a block diagram showing an example of the information processing device 4. This is a data configuration diagram showing an example of the plant management database 410. This is a functional explanation diagram showing an example of the information processing device 4. This is a hardware configuration diagram showing an example of the computer 900 that constitutes each device. This is a flowchart showing an example of the operation of the plant management support system 1. This is a flowchart showing an example of the operation of the piping object 14B generation process (step S30). This is a first schematic diagram showing an example of the piping object 14B generation process (step S30). This is a second schematic diagram showing an example of the piping object 14B generation process (step S30). This is a flowchart showing an example of the operation of the piping member object 14C generation process (step S40). This is a schematic diagram showing an example of the piping member object 14C generation process (step S40). This is a flowchart showing an example of the operation of the isometric figure 15B generation process (step S50). This is a schematic diagram showing an example of the isometric figure 15B generation process (step S50). This is a diagram showing an example of the isometric figure 15B.

[0010] Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. In the following, the scope necessary for explaining how to achieve the objectives of the present invention will be schematically shown, and the scope necessary for explaining the relevant parts of the present invention will be mainly explained, with any parts that are omitted from explanation being based on prior art.

[0011] (Configuration of Plant Management Support System 1) Figure 1 is an overall diagram showing an example of Plant Management Support System 1 and Plant 10. Plant Management Support System 1 functions as a system to support the management of Plant 10. Plant 10 is any plant, such as a natural gas plant, an oil refinery, a chemical processing plant, a power plant, or a steelmaking plant, and is not limited to these examples.

[0012] Plant 10 is composed of multiple components, each component performing a predetermined process. Plant 10 includes, for example, various devices 100 for processing any fluid such as gas, liquid, or fluid powder; piping 101 connecting the devices 100 to form fluid pathways; piping members 102 attached to the piping 101; various instruments (not shown) consisting of flow sensors, pressure sensors, temperature sensors, etc.; and various controllers (not shown) consisting of pumps, compressors, controllers, etc. The devices 100 include, but are not limited to, towers, tanks, and heat exchangers for performing reactions, distillation, extraction, absorption, washing, temperature control, etc. The piping 101 is arranged three-dimensionally within the site of Plant 10. The piping members 102 include, but are not limited to, valves for controlling the flow rate and pressure of the fluid flowing through the piping 101, flanges, supports, etc. for connecting and supporting the piping 101.

[0013] The plant management support system 1 comprises, as its main components, a three-dimensional measuring device 2, a two-dimensional imaging device 3, an information processing device 4 that supports the management of the plant 10, and a terminal device 5 used by the manager of the plant 10. Each of the devices 2 to 5 is, for example, composed of a general-purpose or dedicated computer (see Figure 5 below) and is connected to a wired or wireless network 6, enabling the mutual transmission and reception of various types of data. The number of each of the devices 2 to 5 and the connection configuration of the network 6 are not limited to the example in Figure 1 and may be changed as appropriate.

[0014] The three-dimensional measuring device 2 is a device for measuring the three-dimensional shape of the plant 10. The three-dimensional measuring device 2 is composed of, for example, a distance measuring sensor using a laser or ultrasound, a stereo camera, etc. The three-dimensional measuring device 2 measures the three-dimensional shape of the plant 10 and outputs point cloud data 11 as the measurement result. The point cloud data 11 is sent to the information processing device 4 via the network 6 or a recording medium, etc.

[0015] The two-dimensional imaging device 3 is a device that captures two-dimensional images of the plant 10. The two-dimensional imaging device 3 is composed of, for example, a panoramic camera or a 360-degree camera having an image sensor, and is equipped with a group of sensors such as a positioning sensor, tilt sensor, and direction sensor that can receive positioning signals such as GPS or GNSS. The two-dimensional imaging device 3 captures the plant 10 under predetermined shooting conditions (shooting location, shooting direction, shooting angle of view, etc.) and outputs two-dimensional shooting data 12 as the result of the shooting. At that time, the two-dimensional imaging device 3 outputs shooting area data 13 indicating the shooting area of ​​the plant 10 when the two-dimensional shooting data 12 was captured, based on the positioning result of the shooting location based on the positioning sensor, the detection result of the shooting direction based on the tilt sensor and direction sensor, and the shooting angle of view determined by the field of view of the two-dimensional imaging device 3. The two-dimensional shooting data 12 and the shooting area data 13 are sent to the information processing device 4 via the network 6 or a recording medium, etc.

[0016] The three-dimensional measuring device 2 and the two-dimensional imaging device 3 can be attached to, for example, a manned vehicle, an unmanned vehicle, a drone, or other flying object, to perform three-dimensional shape measurements and two-dimensional image captures at various locations in the plant 10. Alternatively, the administrator may use the three-dimensional measuring device 2 and the two-dimensional imaging device 3 to perform three-dimensional shape measurements and two-dimensional image captures.

[0017] The information processing device 4 is composed of, for example, a server-type computer or a cloud-type computer. The information processing device 4 manages various data related to the plant 10 using a plant management database 410, and generates drawing data 15 such as three-dimensional model data 14 and isometric drawings, and provides them to the terminal device 5.

[0018] The plant management database 410 stores point cloud data 11 obtained when the plant 10 is measured by the three-dimensional measuring device 2, two-dimensional image data 12 obtained when the plant 10 is photographed by the two-dimensional imaging device 3, and image area data 13, etc. It also stores three-dimensional model data 14 and drawing data 15 generated based on the point cloud data 11, two-dimensional image data 12, and image area data 13. The plant management database 410 registers and stores data 11 to 15 related to multiple plants 10 that are to be managed, and the details of the data structure will be described later.

[0019] The terminal device 5 is composed of, for example, a stationary computer or a portable computer. The terminal device 5 has programs such as applications and browsers installed, accepts various input operations, and outputs various information via a display screen and sound. By sending and receiving various data with the information processing device 4, the terminal device 5 supports the management of the plant 10 by, for example, displaying the contents of the plant management database 410 on the display screen, accepting various input operations on the display screen to register new data in the plant management database 410, or modifying registered data.

[0020] (Configuration of Information Processing Device 4) Figure 2 is a block diagram showing an example of an information processing device 4. The information processing device 4 comprises a control unit 40 composed of a processor, a storage unit 41 composed of an HDD, SSD, memory, etc., a communication unit 42 which is a communication interface with the network 6, an input unit 43 composed of a keyboard, mouse, etc., and a display unit 44 composed of a display, etc. Note that the input unit 43 and the display unit 44 may be omitted.

[0021] The storage unit 41 stores the plant management database 410 and the information processing program 411, as well as the operating system, other programs, various data, etc.

[0022] Figure 3 is a data configuration diagram showing an example of a plant management database 410. The plant management database 410 is a database for storing data 11 to 15 related to each plant 10 (in the example of Figure 3, plants A, B, ..., N) for each plant 10.

[0023] The point cloud data 11 is data recorded as a point cloud, which is a collection of measurement points that are the measurement results when various parts of the plant 10 are measured by the three-dimensional measuring device 2. The point cloud data 11 is managed as data that records the point cloud of the entire plant 10 by combining the point clouds that are the measurement results when the plant 10 is measured at various locations in the plant 10, while aligning them.

[0024] The two-dimensional image data 12 and the image area data 13 are data that records the two-dimensional images and image areas when various parts of the plant 10 are photographed by the two-dimensional image capture device 3. The image area is specified, for example, by the shooting location, shooting direction, and shooting angle of view. The two-dimensional image data 12 and the image area data 13 are the results of photographing the plant 10 at various locations in the plant 10, and are managed for each image area.

[0025] The three-dimensional model data 14 is composed of equipment objects 14A, piping objects 14B, piping member objects 14C, instrument objects 14D, and controller objects 14E, which correspond to the components of the plant 10: equipment 100, piping 101, piping members 102, instruments, and controllers, respectively. Each object 14A to 14E includes attributes such as identification information, type information, and location information.

[0026] Furthermore, the three-dimensional model data 14 is data that can display the three-dimensional shape of each object 14A to 14E at any viewpoint, scale, transparency, and display color. The three-dimensional model data 14 is generated based on the point cloud data 11, two-dimensional image data 12, and image area data 13, and is edited through editing operations by the administrator. The three-dimensional model data 14 can use any data format; for example, it may be in PLY format, XML format, or a combination of multiple data formats as appropriate.

[0027] The drawing data 15 is data that records various drawings generated based on each object 14A to 14E included in the three-dimensional model data 14. The drawing data 15 includes, for example, an orthographic projection drawing 15A, an isometric drawing 15B, etc. The drawing data 15 can be in any data format, similar to the three-dimensional model data 14. For example, it may be in CAD format, raster format, or a combination of multiple data formats as appropriate.

[0028] Each of the data 11 to 15 is referenced by the terminal device 5, and editing operations such as adding, deleting, and modifying each of the data 11 to 15 are performed on the display screen of the terminal device 5. The data structure of each of the data 11 to 15 is not limited to the example above and may be changed as appropriate, some of the above data may be omitted, or other data may be added. In addition, part or all of the plant management database 410 may be stored in an external device (which may be multiple) or any storage medium that can be connected to the network 6, in which case the information processing device 4 can access the external device or storage medium via the network 6 and the communication unit 42.

[0029] As shown in Figure 2, the control unit 40 functions as a data acquisition unit 400, an object data processing unit 401, a drawing generation unit 402, and a display information generation unit 403 by executing an information processing program 411 stored in the storage unit 41. Each of the units 400 to 403 of the control unit 40 transmits display information to the terminal device 5 to display various display screens, and accepts various input operations through these display screens, thereby functioning as a user interface with the administrator who manages the plant 10.

[0030] Figure 4 is a functional diagram illustrating an example of an information processing device 4.

[0031] The data acquisition unit 400 acquires point cloud data 11 obtained by measuring the plant 10 with the three-dimensional measuring device 2. The data acquisition unit 400 also acquires two-dimensional imaging data 12 obtained by imaging the plant 10 with the two-dimensional imaging device 3, and imaging area data 13 indicating the imaging area of ​​the plant 10 where the two-dimensional imaging data 12 was taken. For example, based on input operations from the terminal device 5, the data acquisition unit 400 refers to the plant management database 410 to acquire point cloud data 11 of the entire plant 10, as well as two-dimensional imaging data 12 and imaging area data 13 for each imaging area of ​​the plant 10.

[0032] The object data processing unit 401 identifies the three-dimensional position of the pipe 101 from the point cloud data 11 acquired by the data acquisition unit 400, and generates a pipe object 14B that includes pipe position information indicating the three-dimensional position of the pipe 101. The generated pipe object 14B is registered in the plant management database 410 as part of the data of the three-dimensional model data 14.

[0033] Furthermore, the object data processing unit 401 recognizes the two-dimensional position of the piping member 102 from the two-dimensional imaging data 12 acquired by the data acquisition unit 400, and identifies the three-dimensional position of the piping member 102 by illuminating the point cloud data 11 via the imaging area data 13 acquired by the data acquisition unit 400 with the two-dimensional position of the piping member 102, thereby generating a piping member object 14C that includes piping member position information indicating the three-dimensional position of the piping member 102. The generated piping member object 14C is registered in the plant management database 410 as part of the data of the three-dimensional model data 14.

[0034] The drawing generation unit 402 generates an isometric drawing 15B (see Figure 14 described later) of the pipe 101 and pipe member 102 based on the pipe position information contained in the pipe object 14B and the pipe member position information contained in the pipe member object 14C. The isometric drawing 15B is registered in the plant management database 410 as part of the drawing data 15.

[0035] For example, the drawing generation unit 402 identifies the three-dimensional positions of pipe nodes indicating bending or inflection points of pipe 101 and the connection relationships of pipe line segments connecting pipe nodes based on the pipe position information contained in the pipe object 14B, and identifies the three-dimensional positions of pipe member nodes indicating installation points of pipe member 102 and the installation relationships of pipe member nodes with respect to pipe line segments based on the pipe member position information contained in the pipe member object 14C. The drawing generation unit 402 then generates an isometric drawing 15B by connecting the pipe nodes with drawing line segments according to the connection relationships and representing the pipe member nodes with drawing symbols according to their installation relationships. In this case, the drawing generation unit 402 may generate an isometric drawing 15B by correcting the inclination of the pipe segments when connecting the pipe nodes according to the connection relationship of the pipe segments in predetermined angle increments, rotating the three-dimensional positions of the pipe nodes and pipe member nodes based on the corrected angles, projecting the rotated pipe nodes and pipe member nodes onto a predetermined two-dimensional plane, connecting the projected pipe nodes with drawing segments according to the connection relationship of the pipe segments, and representing the projected pipe member nodes with drawing symbols according to the installation relationship of the pipe member nodes.

[0036] The drawing generation unit 402 may generate an isometric drawing 15B without referring to the piping member object 14C and omitting the piping member 102. In this case, the drawing generation unit 402 identifies the three-dimensional positions of the piping nodes indicating the bending or inflection points of the piping 101 and the connection relationships of the piping line segments connecting the piping nodes, based on the piping position information included in the piping object 14B. The drawing generation unit 402 then generates the isometric drawing 15B by connecting the piping nodes with drawing line segments according to the connection relationships. At this time, the drawing generation unit 402 may correct the inclination of the piping line segments when the piping nodes are connected according to the connection relationships of the piping line segments in predetermined angle increments, rotate the three-dimensional positions of the piping nodes based on the corrected angles, project the rotated piping nodes onto a predetermined two-dimensional plane, and generate the isometric drawing 15B by connecting the projected piping nodes with drawing line segments according to the connection relationships of the piping line segments.

[0037] (Hardware configuration of each device) Figure 5 is a hardware configuration diagram showing an example of the computer 900 that constitutes each device. Each device 2 to 5 in the plant management support system 1 is composed of a general-purpose or dedicated computer 900.

[0038] As shown in Figure 5, the computer 900 comprises, as its main components, a bus 910, a processor 912, a memory 914, an input device 916, an output device 917, a display device 918, a storage device 920, a communication interface unit 922, an external device interface unit 924, an I / O device interface unit 926, and a media input / output unit 928. Note that the above components may be omitted as appropriate depending on the intended use of the computer 900.

[0039] The processor 912 consists of one or more arithmetic processing units (CPU (Central Processing Unit), MPU (Micro-Processing Unit), DSP (Digital Signal Processor), GPU (Graphics Processing Unit), NPU (Neural Processing Unit), etc.) and operates as a control unit that oversees the entire computer 900. The memory 914 stores various data and programs 930 and consists of volatile memory (DRAM, SRAM, etc.) that functions as main memory, and non-volatile memory (ROM), flash memory, etc.

[0040] The input device 916 consists of, for example, a keyboard, mouse, numeric keypad, or electronic pen, and functions as an input unit. The output device 917 consists of, for example, a sound (voice) output device or a vibration device, and functions as an output unit. The display device 918 consists of, for example, a liquid crystal display, an organic EL display, electronic paper, or a projector, and functions as an output unit. The input device 916 and the display device 918 may be configured as an integrated unit, such as a touch panel display. The storage device 920 consists of, for example, an HDD or SSD, and functions as a storage unit. The storage device 920 stores various data necessary for the execution of the operating system and the program 930.

[0041] The communication I / F unit 922 is connected by wire or wireless to a network 940 such as the Internet or an intranet (which may be the same as network 6 in Figure 1) and functions as a communication unit that sends and receives data with other computers according to a predetermined communication standard. The external device I / F unit 924 is connected by wire or wireless to external devices 950 such as a camera, printer, scanner, or reader / writer and functions as a communication unit that sends and receives data with the external devices 950 according to a predetermined communication standard. The I / O device I / F unit 926 is connected to I / O devices 960 such as various sensors and actuators and functions as a communication unit that sends and receives various signals and data with the I / O devices 960, for example, detection signals from sensors and control signals to actuators. The media input / output unit 928 consists of, for example, a drive device such as a DVD drive or CD drive, a memory card slot, and a USB connector, and reads and writes data to media (non-temporary storage media) 970 such as DVDs, CDs, memory cards, and USB memory.

[0042] In the computer 900 having the above configuration, the processor 912 calls and executes the program 930 stored in the storage device 920 in the memory 914, and controls each part of the computer 900 via the bus 910. Note that the program 930 may be stored in the memory 914 instead of the storage device 920. The program 930 may be recorded in the medium 970 in an installable file format or an executable file format, and may be provided to the computer 900 via the media input / output unit 928. The program 930 may be provided to the computer 900 by downloading it via the network 940 through the communication I / F unit 922. Further, the various functions realized by the computer 900 when the processor 912 executes the program 930 may be realized by hardware such as, for example, an FPGA (Field-Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), or the like.

[0043] The computer 900 is constituted by, for example, a stationary computer or a portable computer, and is an electronic device in any form. The computer 900 may be a client-type computer, a server-type computer, a cloud-type computer, or, for example, an embedded computer called a control panel, a controller (including a microcomputer, a programmable logic controller, a sequencer), or the like.

[0044] (Operation of the plant management support system 1) Hereinafter, a series of operations by the plant management support system 1 will be described. The series of operations are executed by the cooperation of each part of the information processing apparatus 4 (each step of the information processing method executed by the information processing program 411) and the terminal device 5.

[0045] FIG. 6 is a flowchart showing an example of the operation of the plant management support system 1. Hereinafter, it will be described assuming that the point group data 11, the two-dimensional imaging data 12, and the imaging area data 13 of the plant 10 to be processed are registered in the plant management database 410.

[0046] In step S10, when the terminal device 5 receives, for example, as an input operation by the administrator on the display screen, a selection instruction for the plant 10 to be processed and a generation instruction for the isometric view 15B, it transmits instruction information based on the input operation to the information processing device 4.

[0047] In step S20, when the data acquisition unit 400 of the information processing device 4 receives the instruction information from the terminal device 5, it refers to the plant management database 410 and acquires the point cloud data 11, two-dimensional photographed data 12, and photographed area data 13 of the plant 10 to be processed.

[0048] In step S30, the object data processing unit 401 performs a pipe object generation process of specifying the three-dimensional position of the pipe 101 based on the point cloud data 11 acquired in step S20 and generating the pipe object 14B, and registers the generated pipe object 14B in the plant management database 410.

[0049] FIG. 7 is a flowchart showing an example of the operation of the pipe object generation process (step S30). FIG. 8 is a first schematic diagram showing an example of the pipe object generation process (step S30). FIG. 9 is a second schematic diagram showing an example of the pipe object generation process (step S30).

[0050] In step S300, based on the point cloud data 11 acquired in step S20, as shown in FIGS. 8 and 9, a plurality of pipe cross-section candidates 1115 (1115A to 1115G) regarded as the cross-section of the pipe 101 are extracted.

[0051] For example, as shown in Figure 8, in the point cloud data 11, a set of measurement points 110 that are within a predetermined range for a specific measurement point 110 is created as a reference point group 1110. Then, two arbitrary candidate points 1111A and 1111B are selected from the reference point group 1110, and inverse normals 1112A and 1112B are drawn from the two candidate points 1111A and 1111B, respectively. The distance of the nearest tangent segment 1114 connecting the nearest tangential points 1113A and 1113B, where the two inverse normals 1112A and 1112B are closest, is determined as the nearest tangential distance L1. Next, if the nearest contact distance L1 is less than or equal to a predetermined reference distance, a candidate pipe cross section 1115 is extracted based on the center point 1116, which is the midpoint of the candidate points 1111A, 1111B and the nearest contacts 1113A, 1113B at that time. For the candidate pipe cross section 1115, the position coordinates Oc of the center point 1116, the radius Or, and the normal vector Onv are obtained.

[0052] As described above, multiple reference point groups 1110 are created from the point cloud data 11, and pipe cross-section candidates 1115 are extracted from each reference point group 1110, thereby obtaining the position coordinate Oc, radius Or, and normal vector Onv for each pipe cross-section candidate 1115.

[0053] In step S310, a center line 1117 is generated for each pipe cross-section candidate 1115, passing through the center point 1116 of the pipe cross-section candidate 1115 and extending in the direction normal to the pipe cross-section candidate 1115. Figure 9 shows the case where a center line 1117 is generated for each of the six pipe cross-section candidates 1115A to 1115G.

[0054] In step S320, as shown in Figure 9, the three-dimensional position of the pipe 101 is determined by connecting centerlines 1117 that satisfy predetermined adjacency conditions. At that time, for example, if the positional relationship between the centerline 1117 of one pipe cross-section candidate 1115 and the centerline 1117 of the other pipe cross-section candidate 1115 is such that the distance between the centerlines 1117 is less than or equal to a predetermined reference distance, and the angle between the centerlines 1117 is less than or equal to a predetermined reference angle, it is determined that they are connected in a straight line. Then, with one pipe cross-section candidate 1115 as the first pipe end and the other pipe cross-section candidate 1115 as the second pipe end, the three-dimensional position of the pipe 101 is determined by assuming that a straight pipe 101 is installed between the first pipe end and the second pipe end. Figure 9 illustrates a case where the three-dimensional position of the pipe 101 is determined to be such that the first pipe 101A, which is determined to be connected in a straight line with four pipe cross-section candidates 1115A to 1115D, and the second pipe 101B, which is determined to be connected in a straight line with three pipe cross-section candidates 1115E to 1115F, intersect.

[0055] In step S330, a pipe object 14B is generated based on the result of identifying the three-dimensional position of the pipe 101. In this case, for example, if it is determined that a straight pipe 101 is installed between a first pipe end (first pipe cross-section candidate 1115) and a second pipe end (second pipe cross-section candidate 1115), the pipe position information of the pipe object 14B is set as follows: first pipe end coordinates = position coordinates Oc of the first pipe cross-section candidate 1115, second pipe end coordinates = position coordinates Oc of the second pipe cross-section candidate 1115, pipe azimuth angle = azimuth angle formed by the line segment connecting the position coordinates Oc of the first pipe cross-section candidate 1115 and the position coordinates Oc of the second pipe cross-section candidate 1115, pipe radius = average value of the radius Or of the first pipe cross-section candidate 1115 and the radius Or of the second pipe cross-section candidate 1115.

[0056] In the example shown in Figure 9, the pipe position information for the pipe object 14B corresponding to the first pipe 101A is set as follows: first pipe end coordinates = position coordinates Oc of pipe cross-section candidate 1115A, second pipe end coordinates = position coordinates Oc of pipe cross-section candidate 1115D, pipe azimuth angle = azimuth angle formed by the line segment connecting the position coordinates Oc of pipe cross-section candidate 1115A and the position coordinates Oc of pipe cross-section candidate 1115D, and pipe radius = average value of the radius Or of pipe cross-section candidate 1115A and the radius Or of pipe cross-section candidate 1115D. Furthermore, the piping position information of the piping object 14B corresponding to the second piping 101B is set as follows: first piping end coordinates = position coordinates Oc of piping cross-section candidate 1115E, second piping end coordinates = position coordinates Oc of piping cross-section candidate 1115G, piping azimuth angle = azimuth angle formed by the line segment connecting the position coordinates Oc of piping cross-section candidate 1115E and the position coordinates Oc of piping cross-section candidate 1115G, and piping radius = average value of the radius Or of piping cross-section candidate 1115E and the radius Or of piping cross-section candidate 1115G.

[0057] Returning to the flowchart in Figure 6, in step S40, the object data processing unit 401 determines the three-dimensional position of the piping member 102 based on the point cloud data 11, two-dimensional imaging data 12, and imaging area data 13 acquired in step S20, performs a piping member object generation process to generate a piping member object 14C, and registers the generated piping member object 14C in the plant management database 410.

[0058] Figure 10 is a flowchart showing an example of the operation of the process (step S40) for generating the piping member object 14C. Figure 11 is a schematic diagram showing an example of the process (step S40) for generating the piping member object 14C.

[0059] In step S400, the two-dimensional position of the piping member 102 is recognized from the two-dimensional image data 12 acquired in step S20. For example, a pre-trained model can be used to recognize the piping member 102. The training model is created by machine learning using training data in which the type of piping member 102 and the existence region indicating the two-dimensional position of the piping member 102 are annotated on the existing two-dimensional image data 12. When using the training model, the type of piping member 102 and the detection region 120 indicating the two-dimensional position of the piping member 102 are output by inputting the two-dimensional image data 12 into the training model. Note that a separate training model may be prepared for each type of piping member 102.

[0060] In step S410, the two-dimensional position (detection region 120) of the piping member 102 is illuminated onto the point cloud data 11 acquired in step S20 via the imaging region data 13 acquired in step S20. For example, as shown in Figure 11, based on the two-dimensional imaging data 12, an illumination line 121 is generated that passes through the two-dimensional position of the piping member 102 (detection point 120a on the frame line of the detection region 120) and extends from the imaging position 130 of the two-dimensional imaging data 12.

[0061] In step S420, the three-dimensional position of the piping member 102 is determined based on the proximity measurement points 1120 that satisfy predetermined proximity conditions with the irradiation line 121, from among the measurement points 110 included in the point cloud data 11. For example, if a proximity measurement point 1120 is identified for each detection point 120a, the three-dimensional position of the piping member 102 is determined by assuming that the piping member 102 is installed at the center point 1121 inside each proximity measurement point 1120. Figure 11 illustrates the case where four proximity measurement points 1120 are identified, and the piping member 102 is determined to be installed at the center point 1121 of these four proximity measurement points 1120.

[0062] In step S430, a piping member object 14C is generated based on the result of determining the three-dimensional position of the piping member 102. In this case, for example, if it is determined that the piping member 102 is installed inside a plurality of proximity measurement points 1120, the piping member position information of the piping member object 14C is set to piping member coordinates = position coordinates Oc of the center point 1121, as shown in Figure 11.

[0063] Returning to the flowchart in Figure 6, in step S50, the drawing generation unit 402 performs a drawing generation process to generate an isometric drawing 15B of the pipe 101 and pipe member 102 based on the pipe object 14B generated in step S30 and the pipe member object 14C generated in step S40, and registers the generated isometric drawing 15B in the plant management database 410.

[0064] Figure 12 is a flowchart showing an example of the operation of the isometric figure generation process (step S50). Figures 13A and 13B are schematic diagrams showing an example of the isometric figure generation process (step S50).

[0065] In step S500, based on the pipe position information contained in the pipe object 14B generated in step S30, the three-dimensional positions of pipe nodes 140 (140a to 140d) indicating the bending or inflection points of pipe 101 and the connection relationships of pipe segments 142 (142a to 142c) connecting the pipe nodes 140 are identified. For example, based on the pipe end coordinates, two pipes 101 whose pipe ends are connected are identified, and the pipe azimuth angle of one pipe 101 is compared with the pipe azimuth angle of the other pipe 101. If it is determined that the pipes are curved or inflection points, the midpoint of the pipe end coordinates of the two pipes 101 is identified as pipe node 140. Then, the connection relationships of the pipe segments 142 are identified so that they are represented in a graph where pipe nodes 140 (black rectangles in Figures 13A and 13B) are nodes and pipe segments 142 (solid lines in Figures 13A and 13B) are links.

[0066] In step S501, based on the piping member position information included in the piping member object 14C generated in step S40, the three-dimensional positions of the piping member nodes 141 (141a, 141b) indicating the installation points of the piping member 102 and the installation relationship of the piping member nodes 141 with respect to the piping line segments 142 are identified. For example, assuming that the piping member nodes 141 (black circles in Figures 13A and 13B) are located in the piping member coordinates of the piping member 102, the installation relationship of the piping member nodes 141 is identified by identifying the piping line segments 142 that overlap with or are close to the piping member nodes 141.

[0067] In step S510, based on the connection relationships of the piping line segments 142 identified in step S500, the starting piping node 140 and the piping node 140 adjacent to it are selected as a pair of piping nodes 140 to be processed. Figure 13A shows the case where piping nodes 140a and 140b are selected as a pair of piping nodes 140 to be processed. Figure 13B shows the case where piping nodes 140b and 140c are identified as the next pair of piping nodes 140 to be processed in step S540, which will be described later.

[0068] In step S520, based on the pair of pipe nodes 140 to be processed, the pipe segment 143 to be processed is identified by connecting the starting pipe node 140 and the opposite pipe node 140. Figure 13A shows the case where pipe segment 142a is identified as the pipe segment 143 to be processed. Figure 13B shows the case where pipe segment 142b is identified as the pipe segment 143 to be processed.

[0069] In step S521, the pipe segment 143 to be processed is projected onto the X-Y plane with the starting pipe node 140 as the origin, and a correction process is performed to correct the inclination of the X-Y plane segment 143a projected onto the X-Y plane in predetermined angular increments. In the correction process, for example, if the inclination is 0° to 7°, the correction is "0°", if the inclination is "8° to 22°", the correction is "15°", and if the inclination is 23° to 37°, the correction is "30°", and so on, in increments of 15°. At that time, the vector indicating the correction angle when the inclination of the X-Y plane segment 143a is corrected is called the X-Y plane correction vector 144a, and the corrected X-Y plane segment is called the X-Y plane corrected segment 143b.

[0070] In step S522, the X-Y plane correction line segment 143b is projected onto the Y-Z plane with the starting pipe node 140 as the origin. At that time, the inclination of the Y-Z plane line segment 143c projected onto the Y-Z plane is corrected in predetermined angle increments, similar to step S521. In this case, the vector indicating the correction angle when the inclination of the Y-Z plane line segment 143c is corrected is called the Y-Z plane correction vector 144b, and the corrected Y-Z plane line segment is called the Y-Z plane correction line segment 143d.

[0071] In step S523, the Y-Z plane correction line segment 143d is projected onto the Z-X plane with the starting pipe node 140 as the origin, and the inclination of the Z-X plane line segment 143e projected onto the Z-X plane is corrected in predetermined angle increments, similar to step S521. At this time, the vector indicating the correction angle when the inclination of the Z-X plane line segment 143e is corrected is called the Z-X plane correction vector 144c, and the corrected Z-X plane line segment is called the Z-X plane correction line segment 143f.

[0072] In step S530, the combined correction vector 145 is calculated by combining the X-Y plane correction vector 144a, the Y-Z plane correction vector 144b, and the Z-X plane correction vector 144c.

[0073] In step S531, using the starting pipe node 140 as the origin, the composite correction vector 145 is used to rotate and transform the three-dimensional positions of each pipe node 140 following the starting pipe node 140, and the three-dimensional positions of each pipe member node 141 installed on each pipe line segment 142 following the starting pipe node 140, thereby updating the three-dimensional positions of each pipe node 140 and each pipe member node 141. Figure 13A illustrates the case where the pipe nodes 140b to 140d following pipe node 140a and the pipe member nodes 141a and 141b are rotated and transformed using the composite correction vector 145 with pipe node 140a as the origin. Figure 13B illustrates the case where the pipe nodes 140c and 140d following pipe node 140b, and the pipe member nodes 141a and 141b are rotated and transformed by the composite correction vector 145, with pipe node 140b as the origin.

[0074] In step S540, the opposite piping node 140 and the piping node 140 adjacent to that piping node 140 are selected as the pair of piping nodes 140 to be processed next. Figure 13B illustrates the case where piping nodes 140b and 140c have been identified as the pair of piping nodes 140 to be processed next, as described above.

[0075] If, in step S550, a pair of pipe nodes 140 to be processed next exists (step S550: Yes), the process returns to step S520 and repeats each of the steps from step S520 onward. On the other hand, if, in step S550, a pair of pipe nodes 140 to be processed next does not exist (step S550: No), the process proceeds to step S560.

[0076] In step S560, the three-dimensional positions of each pipe node 140 and each pipe member node 141, which were updated in step S531, are projected onto a predetermined two-dimensional plane. At this time, the three-dimensional positions of each pipe node 140 and each pipe member node 141 may be further rotated and transformed before being projected onto the two-dimensional plane so that the plant north (PN) is aligned with a predetermined direction on the drawing.

[0077] In step S561, the projection positions 150 of each pipe node 140 projected in step S560 are connected by drawing line segments 152 according to the connection relationships of the pipe line segments 142, and the projection positions 151 of each pipe member node 141 projected in step S560 are represented by drawing symbols 153 according to the installation relationships of the pipe member nodes 141, thereby generating an isometric drawing 15B (see Figure 14 described later).

[0078] Figure 14 shows an example of an isometric figure 15B. The isometric figure 15B is represented, for example, by drawing line segments 152 connecting projection positions 150 corresponding to pipe nodes 140, and drawing symbols 153 indicated at projection positions 151 corresponding to pipe member nodes 141. The drawing symbols 153 are used, for example, when recognizing the two-dimensional position of the pipe member 102 from the two-dimensional imaging data 12 in step S400, by also recognizing the type of pipe member 102, and using symbols corresponding to the type of pipe member 102. Although the isometric figure 15B in Figure 14 shows a part of the plant 10, an isometric figure 15B of the entire plant 10 may be generated, or an isometric figure 15B of a range specified by the administrator may be generated.

[0079] Returning to the flowchart in Figure 6, in step S60, the display information generation unit 403 generates display information for displaying the isometric figure 15B generated in step S50 and transmits it to the terminal device 5. In step S70, when the terminal device 5 receives the display information from the information processing device 4, it displays the isometric figure 15B based on that display information.

[0080] As a result of the above series of processes, the isometric drawing 15B generated by the information processing device 4 is registered in the plant management database 410 and presented to the administrator. In the above series of processes, step S20 corresponds to the data acquisition process, steps S30 and S40 to the object data processing process, step S50 to the drawing generation process, and step S60 to the display information generation process.

[0081] As described above, according to the information processing device 4 and information processing method of this embodiment, the object data processing unit 401 generates a piping object 14B from the point cloud data 11 that includes piping position information indicating the three-dimensional position of the piping 101, and the drawing generation unit 402 generates an isometric drawing 15B based on the piping position information included in the piping object 14B. This makes it possible to generate an isometric drawing 15B of the piping 101 from the point cloud data 11 without having to prepare a three-dimensional model of the plant 10 in advance.

[0082] Furthermore, the object data processing unit 401 recognizes the two-dimensional position of the piping member 102 from the two-dimensional image data 12, and projects the two-dimensional position of the piping member 102 onto the point cloud data 11 via the image area data 13, thereby generating a piping member object 14C that includes piping member position information indicating the three-dimensional position of the piping member 102. The drawing generation unit 402 then generates an isometric drawing 15B based on the piping position information contained in the piping object 14B and the piping member position information contained in the piping member object 14C. This makes it possible to generate an isometric drawing 15B of the piping 101 and piping member 102 from the point cloud data 11, two-dimensional image data 12, and image area data 13 without having to prepare a three-dimensional model of the plant 10 in advance.

[0083] (Other Embodiments) The present invention is not limited to the embodiments described above, and can be implemented with various modifications without departing from the spirit of the invention. All of these modifications are included in the technical concept of the present invention.

[0084] In the above embodiment, the functions of each part of the information processing device 4 were described as being realized by a single device, but the functions of each part may be distributed among multiple devices to be realized by multiple devices. Alternatively, the control unit of the terminal device 5 may function as the information processing device 4 by executing the information processing program 411.

[0085] In the above embodiment, the case in which the plant management support system 1 operates according to the flowchart shown in Figure 6 has been described, but the execution order of each step may be changed as appropriate, or some steps may be omitted. For example, if an isometric drawing 15B of only the piping 101 is to be generated in step S50, only the point cloud data 11 may be acquired in step S20, and step S40 may be omitted. Also, if other types of drawing data 15 such as an orthographic projection drawing 15A are to be generated, a step of generating other types of drawing data 15 based on the piping object 14B and the piping member object 14C may be performed instead of or in addition to step S50. Furthermore, in the example in Figure 6, the case in which steps S20 to S50 are performed after step S10 has been described, but steps S20 to S40 may be performed in advance before step S10, and then step S50 may be performed after step S10, so that in step S50, an isometric drawing 15B or other types of drawing data 15 are generated based on the previously generated piping object 14B and the piping member object 14C.

[0086] 1...Plant management support system, 2...Three-dimensional measuring device, 3...Two-dimensional imaging device, 4...Information processing device, 5...Terminal device, 6...Network, 10...Plant, 40...Control unit, 41...Storage unit, 42...Communication unit, 43...Input unit, 44...Display unit, 100...Equipment, 101...Piping, 102...Piping components, 400...Data acquisition unit, 401...Object data processing unit, 402...Drawing generation unit, 403...Display information generation unit, 410...Plant management database, 411...Information processing program

Claims

1. An information processing device for supporting the management of a plant which includes piping and piping members attached to the piping as components, comprising: a data acquisition unit that acquires point cloud data measured by a three-dimensional measuring device of the plant; an object data processing unit that identifies the three-dimensional position of the piping from the point cloud data and generates a piping object which includes piping position information indicating the three-dimensional position of the piping; and a drawing generation unit that, based on the piping position information included in the piping object, identifies the three-dimensional position of piping nodes which indicate bending points or inflection points of the piping, and the connection relationship of piping line segments which connect the piping nodes, and generates an isometric drawing of the piping by connecting the piping nodes with drawing line segments according to the connection relationship.

2. The information processing apparatus according to claim 1, wherein the object data processing unit extracts a plurality of pipe cross-section candidates that are considered to be cross-sections of the pipe based on the point cloud data, generates a center line for each pipe cross-section candidate that passes through the center point of the pipe cross-section candidate and extends in the direction of the normal of the pipe cross-section candidate, and identifies the three-dimensional position of the pipe by connecting the center lines that satisfy predetermined adjacency conditions.

3. The information processing apparatus according to claim 1 or 2, wherein the drawing generation unit corrects the inclination of the pipe line segments when the pipe nodes are connected according to the connection relationship in predetermined angle increments, rotates the three-dimensional position of the pipe nodes based on the corrected angle, projects the rotated pipe nodes onto a predetermined two-dimensional plane, and connects the projected pipe nodes with drawing line segments according to the connection relationship to generate the isometric drawing.

4. The data acquisition unit acquires two-dimensional image data obtained by photographing the plant with a two-dimensional imaging device and image area data indicating the area of ​​the plant where the two-dimensional image data was taken. The object data processing unit recognizes the two-dimensional position of the piping member from the two-dimensional image data and identifies the three-dimensional position of the piping member by illuminating the point cloud data with the two-dimensional position of the piping member via the image area data. The object data processing unit generates a piping member object that includes piping member position information indicating the three-dimensional position of the piping member. The drawing generation unit identifies the three-dimensional position of the piping node indicating the bending point or inflection point of the piping and the connection relationship of the piping line segments connecting the piping nodes, based on the piping position information included in the piping object. The drawing generation unit identifies the three-dimensional position of the piping member node indicating the installation point of the piping member and the installation relationship of the piping member node with respect to the piping line segments, based on the piping position information included in the piping member object. The information processing device according to claim 1, which generates an isometric drawing of the piping and the piping members by connecting the piping nodes with line segments in accordance with the connection relationship and representing the piping member nodes with symbols in accordance with the installation relationship.

5. The information processing apparatus according to claim 4, wherein the object data processing unit generates an irradiation line that passes through the two-dimensional position of the piping member and extends from the shooting position of the two-dimensional imaging data based on the two-dimensional imaging data, and identifies the three-dimensional position of the piping member based on the measurement points among the measurement points included in the point cloud data that satisfy a predetermined proximity condition with the irradiation line.

6. The information processing apparatus according to claim 4 or 5, wherein the drawing generation unit corrects the inclination of the pipe line segments when the pipe nodes are connected according to the connection relationship in predetermined angle increments, rotates the three-dimensional positions of the pipe nodes and pipe member nodes based on the corrected angle, projects the rotated pipe nodes and pipe member nodes onto a predetermined two-dimensional plane, connects the projected pipe nodes with drawing line segments according to the connection relationship, and represents the projected pipe member nodes with drawing symbols according to the installation relationship to generate the isometric drawing.

7. An information processing method for supporting the management of a plant including piping and piping members attached to the piping as components, comprising: a data acquisition step of acquiring point cloud data measured by a three-dimensional measuring device of the plant; an object data processing step of identifying the three-dimensional position of the piping from the point cloud data and generating a piping object that includes piping position information indicating the three-dimensional position of the piping; and a drawing generation step of identifying the three-dimensional position of piping nodes indicating bending points or inflection points of the piping and the connection relationship of piping line segments connecting the piping nodes, and generating an isometric drawing of the piping by connecting the piping nodes with drawing line segments according to the connection relationship, based on the piping position information included in the piping object.

Images