Information processing device, display system, data generation method and program

Abstract

To provide an information processing device capable of displaying a component included in 3D data in an optimum viewpoint.SOLUTION: An information processing device comprises: a recognition unit which recognizes at least one piece of component data included in a predetermined area, from 3D data indicating the predetermined area; a setting unit which sets a bounding box including the component data within 3D space displaying the 3D data; and a determination unit which determines a position of a viewpoint when displaying the component data on a display unit, on the basis of the size of a surface constituting the bounding box.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a display system, a data generation method, and a program. [Background technology]

[0002] In recent years, LiDAR (Light Detection and Ranging) has been used to measure structures or monitor specific areas. LiDAR scans with a laser beam to perform 3D (three-dimensional) scans. The 3D data generated by LiDAR is displayed on a computer screen using a specific application.

[0003] Patent Document 1 discloses the configuration of an image processing device that displays a region to be observed from among multiple regions included in a medical image at an optimal viewpoint and line of sight direction. The image processing device in Patent Document 1 determines the line of sight direction, etc. of the selected region using a table in which the line of sight direction, rotation axis, and viewpoint are predetermined. The line of sight direction, etc. defined in the table are determined based on anatomical knowledge. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-011564 Summary of the Invention [Problem to be solved by the invention]

[0005] The image processing device of Patent Document 1 determines the line of sight, etc. of a selected part using a table in which line of sight, etc. are predetermined. However, unlike the structure of the human body, in the absence of prior knowledge about the layout of parts, etc., within the facility to be monitored, the image processing device of Patent Document 1 has a problem in that it cannot determine the optimal viewpoint and line of sight, etc., of parts, etc., included in the 3D data.

[0006] An object of the present disclosure is to provide an information processing device, a display system, a data generation method, and a program that are capable of displaying parts included in 3D data from an optimal viewpoint. [Means for solving the problem]

[0007] The information processing device according to the present disclosure includes a recognition unit that recognizes at least one piece of component data included in a predetermined area from 3D data indicating the area, a setting unit that sets a bounding box that contains the component data within a 3D space in which the 3D data is displayed, and a determination unit that determines the position of a viewpoint when displaying the component data on a display unit based on the size of the faces that make up the bounding box.

[0008] The display system according to the present disclosure includes an information processing device having a display device that displays display data, a recognition unit that recognizes at least one piece of part data included in a predetermined area from 3D data showing the predetermined area, a generation unit that generates the display data when the part data is observed from a viewpoint of any position in a 3D space in which the 3D data is displayed, and a determination unit that determines a position at which to observe the part data based on a display area of ​​the part data when the display data is displayed on the display device.

[0009] The data generation method according to the present disclosure recognizes at least one piece of component data included in a specified area from 3D data showing the specified area, generates display data for when the component data is observed from a viewpoint at any position within the 3D space in which the 3D data is displayed, and determines the position for observing the component data based on the display area of ​​the component data when the display data is displayed on a display unit.

[0010] The program according to the present disclosure causes a computer to recognize at least one piece of component data included in a specified area from 3D data showing the specified area, generate display data for when the component data is observed from a viewpoint of any position within the 3D space in which the 3D data is displayed, and determine a position for observing the component data based on the display area of ​​the component data when the display data is displayed on a display unit. [Effects of the Invention]

[0011] The present invention makes it possible to provide an information processing device, a display system, a data generation method, and a program that are capable of displaying parts included in 3D data from an optimal viewpoint. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 shows an example of the configuration of an information processing device. [Figure 2] FIG. 2 shows the flow of a data generation method executed by an information processing device. [Figure 3] FIG. 3 shows an example of the configuration of an information processing device. [Figure 4] FIG. 4 shows a state in which a part and a bounding box containing the part exist in the XYZ space. [Figure 5] FIG. 5 shows an image displayed on a display device. [Figure 6] FIG. 6 shows the flow of a parts list generation process executed in an information processing device. [Figure 7] FIG. 7 shows the flow of the part data display process executed in the information processing device. [Figure 8] FIG. 8 is a block diagram illustrating an example of the configuration of an information processing device. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Embodiment 1) 1 shows an example of the configuration of an information processing device 10. The information processing device 10 may be a computer device that operates when a processor executes a program stored in a memory.

[0014] The information processing device 10 has a recognition unit 11, a setting unit 12, and a determination unit 13. The recognition unit 11, the setting unit 12, and the determination unit 13 may be software or modules that are executed by a processor executing a program stored in a memory, or may be hardware such as a circuit or a chip.

[0015] The recognition unit 11 may be used as a means for recognizing data, the generation unit 12 may be used as a means for generating data, and the determination unit 13 may be used as a means for determining processing related to data.

[0016] The recognition unit 11 recognizes at least one piece of part data included in a predetermined area from 3D (dimensional) data indicating the predetermined area. The predetermined area may be an area including a facility to be monitored or observed. The predetermined area may be an enclosed space such as inside a building, or may be an area including a space outside the building.

[0017] The 3D data may be, for example, point cloud data. Point cloud data is a collection of points having three-dimensional information. The point cloud data may be generated using a sensor. The sensor may be, for example, a sensor that measures distance or an imaging device. The sensor that measures distance may be, for example, a sensor that measures the distance to an object using LiDAR (Light Detection and Ranging). Alternatively, the point cloud data may be generated by matching feature points of multiple image data obtained by photographing the same object from multiple locations. The generation of point cloud data using multiple image data may be performed using, for example, SfM (Structure from Motion). The image data may be frozen by an imaging device used as a sensor.

[0018] The recognition unit 11 may acquire point cloud data from a sensor via a network, or may generate point cloud data using data acquired from a sensor via a network.

[0019] Recognizing part data may mean recognizing each part included in the 3D data. Recognizing part data may mean identifying the type, size, shape, name, etc. of the part. Recognizing part data may also mean dividing the part data from the 3D data. Recognizing part data may also mean classifying each point that makes up the point cloud data into a specific part. Alternatively, recognizing part data may mean classifying pixels that make up an image displaying the 3D data into a specific part. The part data may have three-dimensional information.

[0020] The 3D data may be data showing the shape of a facility such as a substation. The components may be devices, objects, etc. provided within the facility. For example, the components provided within a substation may be disconnectors, circuit breakers, transformers, electric wires, etc. Furthermore, the components may be devices, objects, etc. that constitute the disconnectors, circuit breakers, transformers, etc. For example, the components may be cables, etc.

[0021] The setting unit 12 sets a bounding box that contains the part data in a 3D space that represents the 3D data.

[0022] The bounding box that contains the part data is set to cover the exterior of the part in 3D space, meaning that the entire part is contained within the space formed by the bounding box.

[0023] A bounding box may be composed of multiple faces. The multiple faces may be, for example, three or more faces. Specifically, the bounding box may be in the shape of a rectangular parallelepiped. Furthermore, the faces that make up the bounding box may be triangular, rectangular, or curved. The curved face may be, for example, circular, elliptical, or the like.

[0024] Setting a bounding box may mean setting the size and position of the bounding box in 3D space. The size of the bounding box may mean determining the size of its faces, its volume, etc. The position of the bounding box may mean determining the coordinates of the vertices of the bounding box, which is, for example, a cube.

[0025] The determination unit 13 determines the position of the viewpoint when displaying the part data on the display unit, based on the size of the faces that make up the bounding box.

[0026] The display unit may be a display screen of a display device such as a monitor. The component data may have three-dimensional information. The component data having three-dimensional information is displayed on the display unit in different shapes depending on the position of the viewpoint from which the component data is observed or viewed. In other words, the external characteristics of the component data displayed on the display unit differ depending on the position of the viewpoint from which the component data is observed.

[0027] For example, the determination unit 13 may determine the gaze position so that at least one of the multiple surfaces constituting the bounding box, which has an area larger than a predetermined area, is displayed in its entirety on the display unit. Alternatively, the determination unit 13 may determine the gaze position so that the largest of the multiple surfaces constituting the bounding box is displayed in its entirety on the display unit.

[0028] Next, the flow of processing executed by the information processing device 10 will be described. Fig. 2 shows the flow of a data generation method executed by the information processing device 10. First, the recognition unit 11 recognizes at least one piece of component data included in a predetermined area from 3D data indicating the predetermined area (S11). Next, the setting unit 12 sets a bounding box that encompasses the component data in the 3D space where the 3D data is displayed (S12). Next, the determination unit 13 determines the position of the viewpoint when displaying the component data on the display unit based on the size of the faces that make up the bounding box (S13).

[0029] As described above, the information processing device 10 determines the position of the viewpoint when displaying the component data on the display unit based on the size of the faces that make up the bounding box that contains the component data. This makes it possible to determine the position of the viewpoint based on the size of the bounding box that contains the component data even if the layout, etc., of the components included in the 3D data is not known in advance.

[0030] (Embodiment 2) 3 shows an example configuration of an information processing device 20 according to the present disclosure. The information processing device 20 corresponds to the information processing device 10 of FIG. 1. The information processing device 20 includes a 3D data division unit 21, a list generation unit 22, a selection unit 23, a setting unit 24, a camera position determination unit 25, and a display control unit 26. The components constituting the information processing device 20 may be software or modules that perform processing by a processor executing a program stored in a memory. Alternatively, the components constituting the information processing device 20 may be hardware such as a circuit or a chip.

[0031] The 3D data dividing unit 21 may be used as a means for dividing 3D data. The 3D data dividing unit 21 corresponds to the recognition unit 11 of the information processing device 10. The list generating unit 22 may be used as a means for generating list information. The selecting unit 23 may be used as a means for selecting information or accepting selection of information. The setting unit 24 may be used as a means for setting information. The setting unit 24 corresponds to the setting unit 12 of the information processing device 10. The camera position determining unit 25 may be used as a means for determining a camera position. The camera position determining unit 25 corresponds to the determination unit 13 of the information processing device 10. The display control unit 26 may be used as a means for controlling data to be displayed on the display device 30 or for displaying data on the display device 30.

[0032] The display device 30 may be a display device or the like. As shown in FIG. 3, the display device 30 may be a device different from the information processing device 20. The display device 30 may be connected to the information processing device 20 via a cable or the like, or may communicate with the information processing device 20 via a network. Alternatively, the display device 30 may be used as a device integrated with the information processing device 20. In other words, the display device 30 may be a component constituting the information processing device 20. The display device 30 displays parts and the like included in a 3D space. The 3D space may be represented using, for example, a coordinate system defined in software or the like that displays 3D data on the display device 30.

[0033] The 3D data division unit 21 receives 3D data. The 3D data may be, for example, point cloud data that indicates the shapes of facilities and the like within a predetermined area and that is generated by a sensor using LiDAR. The 3D data division unit 21 may receive the 3D data from the sensor via a network. Alternatively, if the sensor is mounted on the information processing device 20, the 3D data division unit 21 receives the 3D data generated by the sensor within the information processing device 20.

[0034] The 3D data division unit 21 performs semantic segmentation on the 3D data. Semantic segmentation may involve assigning labels indicating the types or names of objects, parts, etc. present in the 3D data to data elements, which are the smallest units of data. Data elements to which labels have been assigned are referred to as labeled data. In semantic segmentation, for example, a trained model may be used that has been trained using data of objects, parts, etc. and labels of the objects, parts, etc. as training data.

[0035] The 3D data division unit 21 inputs 3D data into the trained model, and outputs labeled data elements that constitute objects or parts included in the 3D data. In other words, the trained model divides or recognizes the objects or parts included in the 3D data. Furthermore, the trained model assigns labels to the data elements that constitute the divided objects, etc. The labeled data elements may be, for example, each pixel that constitutes the image data or each point that constitutes the point cloud data.

[0036] The 3D data division unit 21 may perform semantic segmentation using a method such as a convolutional neural network (CNN) or a recurrent neural network (RNN). The 3D data division unit 21 may also perform semantic segmentation using various methods other than CNN and RNN. CNN is a neural network having multiple layers. Specifically, CNN has a convolutional layer, a pooling layer, and a connection layer. CNN performs object recognition of objects included in an image by extracting features from the image.

[0037] RNN is a method suitable for recognizing data containing continuous information such as time series or text. RNN is a neural network that stores the output of a certain layer as past data and recurrently connects past data.

[0038] The list generation unit 22 generates a parts list. The parts list is a list of parts divided or recognized from the 3D data by the 3D data division unit 21. The parts list includes the names of each part. In other words, the list generation unit 22 generates a list including the names of parts divided from the 3D data by the 3D data division unit 21 and assigned labels. For example, if the 3D data represents a substation, the parts list may include names of disconnectors, circuit breakers, insulators, cables, cross arms, etc. Furthermore, the parts list may associate the size of each part, its position in 3D space, and the like with each name. The display control unit 26 displays the parts list on the display device 30. The display control unit 26 may display the 3D data and the parts list on the display device 30 so that the parts list is overlaid on 3D data representing a predetermined area. Alternatively, the display control unit 26 may cause the display device 30 to display the 3D data and the parts list so as to provide a display area on the display device 30 that displays 3D data indicating a specified area and a display area on the display device 30 that displays the parts list.

[0039] The selection unit 23 receives input of a part desired to be displayed on the display device 30 from, for example, an operator or user of the information processing device 20. For example, the user selects a part desired to be displayed on the display device 30 from a parts list displayed on the display device 30. The user selects an arbitrary part from the parts list using an input device such as a mouse, keyboard, or touch panel. The selection unit 23 outputs identification information of the part selected by the user to the setting unit 24.

[0040] The setting unit 24 receives identification information of the part selected by the user from the selection unit 23. The setting unit 24 sets a bounding box that encompasses the part indicated by the identification information. Specifically, the setting unit 24 includes the part data indicated by the identification information within the space formed by the bounding box. The setting unit 24 may set a bounding box of a predetermined shape, or may change the shape of the bounding box to match the shape of the part. For example, the setting unit 24 may set a bounding box of a shape that minimizes the difference between the volume of the space formed by the bounding box and the volume of the part. It is assumed that information regarding the size, position, etc. of the part has been identified by the 3D data division unit 21. In this case, the setting unit 24 receives information regarding the size, position, etc. of the part from the 3D data division unit 21.

[0041] The camera position determination unit 25 determines the position of a virtual camera (virtual camera) that captures an image of a component in 3D space. The display control unit 26 displays an image captured using the virtual camera on the display device 30. The image captured using the virtual camera may be an image generated when an image is captured using a virtual camera at a predetermined position. Determining the position of the virtual camera may also be determining the viewpoint and line of sight direction when the component is displayed on the display device 30. The angle of view of the virtual camera, etc., is assumed to be predetermined. In other words, the image or image displayed on the display device 30 is determined by the camera position determination unit 25 determining the position and direction of the virtual camera.

[0042] The camera position determination unit 25 determines the position of the virtual camera based on the size of the faces that make up the bounding box. It is assumed that the bounding box has multiple faces. The camera position determination unit 25 may determine, as the position of the virtual camera, a position that can capture the face with the largest area among the multiple faces that make up the bounding box.

[0043] For example, the camera position determination unit 25 may determine, as the position of the virtual camera, a position moved a predetermined distance from the center of the widest surface in a direction perpendicular to the surface. FIG. 4 shows a state in which a part 41 and a bounding box 42 including the part 41 exist in XYZ space. In FIG. 4, it is assumed that the widest surface of the bounding box exists in a position parallel to the XZ plane. Here, if the bounding box has multiple widest surfaces, an arbitrary plane 43 may be selected from the multiple widest surfaces.

[0044] The camera position determination unit 25 may determine, as the position of the virtual camera 44, a position that is a predetermined distance away from the center of the plane 43 in a direction parallel to the Y axis. In this case, the shooting direction of the camera may be a direction parallel to the Y axis. The predetermined distance may be, for example, a distance to a position of the virtual camera 44 where the entire plane 43 is included in an image captured by the virtual camera 44, depending on the angle of view of the virtual camera 44. Furthermore, the predetermined distance may be a distance to a position of the virtual camera 44 where the area of ​​the plane 43 occupies a predetermined percentage of the image captured by the virtual camera 44. Furthermore, the position of the virtual camera 44 is not limited to a position that is a predetermined distance away from the center of the plane 43 in a direction parallel to the Y axis. For example, the position of the virtual camera 44 may be a position that is a predetermined distance away from an arbitrary position in the plane 43 in an arbitrary direction. The position and direction in the plane 43 used to determine the position of the virtual camera 44 may be predetermined or may be specified by the user via an input device.

[0045] Alternatively, the camera position determination unit 25 may determine, as the position of the virtual camera 44, a position at which the plane 43, which is the widest plane of the bounding box, and a plane adjacent to the plane 43, can be photographed. For example, the camera position determination unit 25 may determine, as the position of the virtual camera 44, a position at which the plane 43 and one plane adjacent to the plane 43 can be photographed, or may determine, as the position of the virtual camera 44, a position at which the plane 43 and two or more planes adjacent to the plane 43 can be photographed. In this case, the area occupied by the plane 43 in the image photographed by the virtual camera 44 may be larger than the area occupied by the other planes.

[0046] Furthermore, the camera position determination unit 25 may determine the position of the virtual camera 44 so as to focus on a specific position of the part selected by the user. Fig. 5 shows an image 50 displayed on the display device 30. The image 50 includes an arm 51 because the arm 51 has been selected by the user. Here, the camera position determination unit 25 may determine, as the position of the virtual camera 44, a position at which an image can be generated in which the tip 52 of the arm 51 is located at the center.

[0047] In other words, the camera position determination unit 25 may determine, as the position of the virtual camera 44, a position at which an image 50 can be generated in which the coordinates indicating the position of the tip 52 of the arm 51 overlap with the coordinates of the center of the image 50.

[0048] Which part of the selected part is to be focused on, and further, where in the image 50 the focused position is to be displayed, may be specified by the user or may be determined in advance.

[0049] Furthermore, when the user selects a part different from the part displayed on display device 30, camera position determination unit 25 changes the position of the virtual camera to a position where data of the newly selected part is photographed. Changing the position of the virtual camera may be rephrased as moving the position of the virtual camera or correcting the position of the virtual camera.

[0050] The display control unit 26 causes the display device 30 to display an image of the part 41 taken from the position of the virtual camera determined by the camera position determination unit 25.

[0051] 6 shows the flow of parts list generation processing executed in the information processing device 20. First, the 3D data division unit 21 performs semantic segmentation on the 3D data (S21). Next, the list generation unit 22 generates a parts list including parts divided from the 3D data in the semantic segmentation (S22).

[0052] Next, the display control unit 26 displays the parts list on the display device 30 (S23).

[0053] 7 shows the flow of the part data display process executed by the information processing device 20. First, the selection unit 23 accepts the selection of one part from among the parts included in the parts list (S31). Next, the setting unit 24 sets a bounding box that encompasses the selected part (S32). Next, the camera position determination unit 25 determines, as the virtual camera position, a position that can capture the widest plane of the multiple faces that make up the bounding box (S33). Next, the display control unit 26 displays an image captured from the virtual camera position on the display device 30 (S34).

[0054] As described above, the information processing device 20 displays on the display device 30 the part 41 that appears on the widest plane out of the multiple faces that make up the bounding box that contains the part data. This allows a user who has selected the part 41 to observe the part 41 that appears largest among images of the part 41 taken from various viewpoints. The image in which the part 41 appears largest is likely to show many or large characteristic parts of the part 41.

[0055] 8 is a block diagram showing a configuration example of an information processing device 10 and an information processing device 20 (hereinafter referred to as the information processing device 10, etc.). Referring to FIG. 8, the information processing device 10, etc. includes a network interface 1201, a processor 1202, and a memory 1203. The network interface 1201 may be used to communicate with a network node. The network interface 1201 may include, for example, a network interface card (NIC) conforming to the IEEE 802.3 series. IEEE stands for Institute of Electrical and Electronics Engineers.

[0056] The processor 1202 reads and executes software (computer programs) from the memory 1203 to perform the processes of the measuring device 10 and the like described using the flowcharts. The processor 1202 may be, for example, a microprocessor, an MPU, or a CPU. The processor 1202 may include multiple processors.

[0057] The memory 1203 is configured by a combination of volatile memory and non-volatile memory. The memory 1203 may include storage located remotely from the processor 1202. In this case, the processor 1202 may access the memory 1203 via an I / O (Input / Output) interface (not shown).

[0058] 8, the memory 1203 is used to store a group of software modules. The processor 1202 can perform processing of the information processing device 10, etc. by reading and executing the group of software modules from the memory 1203.

[0059] As described with reference to FIG. 8, each of the processors included in the information processing device 10 or the like executes one or more programs including a group of instructions for causing a computer to execute the algorithm described with reference to the drawings.

[0060] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0061] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0062] Each drawing is merely an example for describing one or more embodiments. Each drawing may relate not only to one particular embodiment, but also to one or more other embodiments. As will be understood by those skilled in the art, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings to create, for example, an embodiment not explicitly shown or described. Not all features or steps shown in any one drawing are necessary to describe an exemplary embodiment, and some features or steps may be omitted. The order of steps described in any drawing may be changed as appropriate.

[0063] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) a recognition unit that recognizes at least one piece of component data included in a predetermined area from 3D data indicating the predetermined area; a setting unit that sets a bounding box that encompasses the part data within a 3D space that displays the 3D data; a determination unit that determines a viewpoint position when the component data is displayed on a display unit based on the size of the surfaces that make up the bounding box. (Appendix 2) The determination unit 2. The information processing device according to claim 1, wherein the position at which the widest surface of the plurality of surfaces constituting the bounding box is entirely displayed on the display unit is set as the position at which the part data is observed. (Appendix 3) The determination unit 3. The information processing device according to claim 2, wherein the position at which the entire widest surface and the entirety of at least one surface adjacent to the widest surface are displayed on the display unit is set as the position at which the part data is observed. (Appendix 4) 4. The information processing device according to claim 1, further comprising: a display control unit that causes the part data to be displayed on the display unit so as to overlap with a center position of the display unit. (Appendix 5) a generating unit that generates a list including the recognized at least one piece of part data; 5. The information processing device according to claim 4, further comprising: a selection unit that accepts a selection of a part to be displayed on the display unit from the list. (Appendix 6) The selection unit accepting selection of second component data when the first component data is displayed on the display unit; The determination unit 6. The information processing device according to claim 5, wherein a position of a viewpoint is changed from a position for observing the first part data to a position for observing the second part data. (Appendix 7) The display control unit 7. The information processing device according to claim 5, wherein the part data and the list are simultaneously displayed on the display unit. (Appendix 8) The recognition unit 7. The information processing device according to any one of claims 1 to 6, wherein the at least one part data is segmented from the 3D data by performing semantic segmentation on the 3D data. (Appendix 9) The information processing device according to any one of appendices 1 to 8, wherein the 3D data is point cloud data generated using a distance measuring device. (Appendix 10) a display device that displays display data; A display system comprising an information processing device having: a recognition unit that recognizes at least one piece of part data included in a predetermined area from 3D data showing the area; a generation unit that generates display data when the part data is observed from a viewpoint of any position in a 3D space in which the 3D data is displayed; and a determination unit that determines a position at which to observe the part data based on a display area of ​​the part data when the display data is displayed on the display device. (Appendix 11) Recognizing at least one piece of part data included in a predetermined area from 3D data indicating the predetermined area; generating display data for when the part data is observed from a viewpoint at an arbitrary position within a 3D space where the 3D data is displayed; A data generating method for determining a position for observing the component data based on a display area of ​​the component data when the display data is displayed on a display unit. (Appendix 12) Recognizing at least one piece of part data included in a predetermined area from 3D data indicating the predetermined area; generating display data for when the part data is observed from a viewpoint at an arbitrary position within a 3D space where the 3D data is displayed; A program that causes a computer to execute the following: determining a position for observing the component data based on a display area of ​​the component data when the display data is displayed on a display unit.

[0064] Some or all of the elements (e.g., configurations and functions) described in Supplementary Notes 2 to 9 that are dependent on Supplementary Notes 1 may also be dependent on Supplementary Notes 10 to 12 in the same dependency relationship as Supplementary Notes 2 to 9. Some or all of the elements described in any Supplementary Note may be applied to various hardware, software, recording means for recording software, systems, and methods. [Explanation of symbols]

[0065] 10. Information processing equipment 11 Recognition part 12 Setting section 13 Decision Section 20 Information processing equipment 21 3D data division section 22 List Generation Unit 23 Selection section 24 Setting section 25 Camera position determination unit 26 Display control unit 30 Display device 41 parts 42 Bounding Box 43 plane 44 Virtual Camera 50 images 51 Arms 52 Tip

Claims

1. a recognition unit that recognizes at least one piece of part data included in a predetermined area from 3D data indicating the predetermined area; a setting unit that sets a bounding box that encompasses the part data within a 3D space that displays the 3D data; a determination unit that determines a viewpoint position when the component data is displayed on a display unit based on the size of the surfaces that constitute the bounding box, The determination unit an information processing device, wherein the position at which the entire widest face of the multiple faces constituting the bounding box and the entirety of at least one face adjacent to the widest face are displayed on the display unit is set as the position at which the part data is observed.

2. The information processing apparatus according to claim 1 , further comprising a display control unit that displays the part data on the display unit so that the part data overlaps with a center position of the display unit.

3. a generating unit that generates a list including the recognized at least one piece of part data; The information processing apparatus according to claim 2 , further comprising: a selection unit that accepts a selection of a part to be displayed on the display unit from the list.

4. The selection unit accepting selection of second component data when the first component data is displayed on the display unit; The determination unit The information processing apparatus according to claim 3 , wherein a viewpoint position is changed from a position for observing the first part data to a position for observing the second part data.

5. The recognition unit The information processing apparatus according to claim 1 , wherein the at least one part data is segmented from the 3D data by performing semantic segmentation on the 3D data.

6. a display device that displays display data; a display system including an information processing device having: a recognition unit that recognizes at least one piece of part data included in a predetermined area from 3D data showing the area; a generation unit that generates display data when the part data is observed from a viewpoint of an arbitrary position within a 3D space in which the 3D data is displayed; and a determination unit that determines a position at which the part data is observed, based on a display area of ​​the part data when the display data is displayed on the display device.

7. Recognizing at least one piece of part data included in a predetermined area from 3D data indicating the predetermined area; generating display data for when the part data is observed from a viewpoint at an arbitrary position within a 3D space where the 3D data is displayed; A data generating method for determining a position for observing the component data based on a display area of ​​the component data when the display data is displayed on a display unit.

8. Recognizing at least one piece of part data included in a predetermined area from 3D data indicating the predetermined area; generating display data for when the part data is observed from a viewpoint at an arbitrary position within a 3D space where the 3D data is displayed; A program that causes a computer to execute the following: determining a position for observing the component data based on a display area of ​​the component data when the display data is displayed on a display unit.

Images