Display control device, display control method, and computer program
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- AISAN TECH
- Filing Date
- 2023-08-31
- Publication Date
- 2026-07-31
Smart Images

Figure 0007898115000001 
Figure 0007898115000002 
Figure 0007898115000003
Abstract
Description
Technical Field
[0001] The present disclosure relates to a display control device and a display control method.
Background Art
[0002] In recent years, the utilization of three-dimensional measurement data related to the three-dimensional shape of ground features such as terrain has been promoted. The three-dimensional measurement data is point cloud data that describes the three-dimensional coordinates of a huge number of points. In a system that projects a point cloud onto a display, for example, each region constituting the point cloud is represented using pixels or voxels.
[0003] However, in a system that performs arithmetic processing and graphic tracing processing for estimating the object surface and contour from the gradient change of the three-dimensional coordinate data of the point cloud, the amount of arithmetic processing is enormous, and the processing load on the system is large.
[0004] Therefore, in the prior art, in order to reduce the amount of arithmetic processing, decimation processing including noise removal and uniformization of point density is performed on the point cloud obtained by three-dimensional measurement to reduce the data amount of the point cloud data (see, for example, Patent Document 1).
Prior Art Documents
Patent Documents
[0005]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0006] Performing preprocessing such as decimation processing on point cloud data with a large data amount requires time. Depending on the decimation, there is also a possibility that the feature points of the three-dimensional shape are lost from the point cloud data.
[0007] Furthermore, when displaying computer graphics based on point cloud data, since point cloud data is point information without size, it is necessary to determine the size of each point in the point cloud data in order to display it on the display. Generally, the size of the points is determined using pixels on the display, so that all points are displayed at a constant size.
[0008] In this method, if the viewpoint is placed far from the point cloud, the spacing between the points projected on the display becomes narrower, making it easier to see the shape of the object. However, if the viewpoint is placed close to the point cloud, the spacing between the points projected on the display becomes wider, making it difficult to see the shape of the object.
[0009] To ensure that the visibility of object shapes does not change depending on the viewpoint or viewing direction, the following display method has been considered. Specifically, a group of objects, such as spheres, whose size can be changed according to the distance from the viewpoint, are to be placed at the positions corresponding to the point cloud, serving as objects for displaying each point.
[0010] However, this display method represented the surface of an object using numerous surface elements, requiring the storage and processing of a vast amount of information in order to display computer graphics.
[0011] Therefore, according to one aspect of this disclosure, it is desirable to provide a novel technology that can reduce the processing load when displaying computer graphics based on point cloud data. [Means for solving the problem]
[0012] According to one aspect of this disclosure, a display control device is provided for displaying three-dimensional computer graphics relating to a three-dimensional shape on a display.
[0013] The display control device comprises an acquisition unit, an operation reception unit, and a control unit. The acquisition unit is configured to acquire point cloud data that represents the three-dimensional shape of an object as a set of coordinate points in a three-dimensional coordinate system. The operation reception unit is configured to accept specified operations related to gaze input through a user interface.
[0014] The control unit is configured to position predetermined planar figures at locations corresponding to each coordinate point in the point cloud data within a three-dimensional virtual space, so as to face a line of sight specified by a specified operation. The control unit is configured to display three-dimensional computer graphics on a display, representing the three-dimensional shape of the object as seen from the specified line of sight, by projecting this three-dimensional virtual space.
[0015] According to this display control device, the three-dimensional shape of an object is represented using planar figures rather than three-dimensional figures such as spheres. Therefore, according to one aspect of this disclosure, it is possible to reduce the processing load related to the display of computer graphics based on point cloud data.
[0016] According to one aspect of this disclosure, the control unit may be configured to place planar figures of the same size as planar figures at positions in the three-dimensional virtual space corresponding to each coordinate point. In this case, the control unit can project the three-dimensional virtual space to display the planar figures as elements of three-dimensional computer graphics on a display, with a size corresponding to the perspective from the viewpoint.
[0017] According to one aspect of this disclosure, the operation reception unit may be configured to accept a specification operation regarding the size of a planar figure. The control unit may be configured to place a planar figure of the size specified by the size specification operation at the position in the three-dimensional virtual space corresponding to each coordinate point, and to display the three-dimensional computer graphics on the display.
[0018] According to one aspect of the present disclosure, the planar figure can be a planar figure having a gradation representation in which the density increases from top to bottom in a three-dimensional virtual space. According to one aspect of the present disclosure, the planar figure can be a circle.
[0019] According to one aspect of the present disclosure, a computer program for causing a computer to function as at least part of the acquisition unit, operation reception unit, and control unit in the display control device described above may be provided. The computer program can be provided by being recorded on a computer-readable recording medium.
[0020] According to one aspect of the present disclosure, a display control method for displaying three-dimensional computer graphics related to the three-dimensional shape of an object on a display may be provided.
[0021] The display control method may include acquiring point cloud data representing the three-dimensional shape of the object by a set of coordinate points in a three-dimensional coordinate system. The display control method may further include receiving a designation operation related to the line of sight input through a user interface.
[0022] The display control method may further include arranging a predetermined planar figure at a position corresponding to each coordinate point in the point cloud data on the three-dimensional virtual space so as to face directly the line of sight designated by the designation operation, and displaying, on the display, three-dimensional computer graphics related to the three-dimensional shape when the object is viewed with the designated line of sight by projecting this three-dimensional virtual space.
[0023] According to such a display control method, as in the above-described display control device, it is possible to reduce the processing load related to display in displaying computer graphics based on point cloud data.
Brief Description of the Drawings
[0024] [Figure 1] It is a block diagram showing the configuration of an information processing apparatus. [Figure 2]It is a flowchart representing the display control process executed by the processor. [Figure 3] FIG. 3A is a diagram showing a simple example of a point cloud, and FIG. 3B is a diagram conceptually showing a group of circular objects arranged in a three-dimensional virtual space for the CG display of the three-dimensional shape corresponding to the point cloud. [Figure 4] FIG. 4A is an explanatory diagram regarding the gradation expression in a circular object, and FIG. 4B is a diagram showing an example of the gradation expression in a circular object.
Embodiments for Carrying Out the Invention
[0025] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. The information processing apparatus 10 of the present embodiment shown in FIG. 1 is configured to display three-dimensional computer graphics (hereinafter simply referred to as "CG") regarding the three-dimensional shape of a ground feature based on three-dimensional measurement data regarding a ground feature such as terrain. The information processing apparatus 10 includes a processor 11, a memory 13, a storage 15, a display 17, an operation unit 18, and a data input / output unit 19.
[0026] The processor 11 performs display control for the display 17 to display the above-mentioned CG on the display 17 by executing processing according to a computer program stored in the storage 15. By this display control, the information processing apparatus 10 functions as a display control apparatus. The memory 13 includes a RAM and is used as a working memory when the processor 11 executes processing.
[0027] The storage 15 includes a hard disk drive and / or a solid state drive and stores a computer program and data used when executing processing according to the computer program. The computer program stored in the storageThe display 17 is controlled by the processor 11 and is configured to display various images, including computer graphics, to the user operating the information processing device 10. An example of the display 17 is a liquid crystal display.
[0029] The operation unit 18 functions as a user interface for inputting operation signals and is configured to input user operation signals to the information processing device 10 to the processor 11. Examples of the operation unit 18 include a keyboard and a pointing device.
[0030] The data input / output unit 19 is configured to enable data input and output with an external device. The data input / output unit 19 may be equipped with a communication interface that enables communication with an external device via wired or wireless connection. The data input / output unit 19 may be equipped with a USB interface that enables reading and writing data to a USB memory device, and / or a media reader / writer that enables reading and writing data to a card-type recording medium.
[0031] In this information processing device 10, when a CG display command is input from the user through the operation unit 18, the processor 11 executes the display control process shown in Figure 2 according to the computer program. By executing the display control process, the processor 11 controls the display of the CG based on the three-dimensional measurement data specified by the user.
[0032] When the display control process shown in Figure 2 is started, the processor 11 acquires the three-dimensional measurement data specified by the user when the CG display command was input (S110). The acquired three-dimensional measurement data is three-dimensional point cloud data that represents the three-dimensional shape of terrain and other features in the target area using a point cloud. The three-dimensional measurement data describes each measurement point using three-dimensional coordinates. Specifically, the three-dimensional coordinates of each measurement point that makes up the point cloud are expressed in XYZ coordinates.
[0033] In the XYZ coordinate system, the XY plane corresponds to the horizontal plane, and the Z axis corresponds to the height direction from the horizontal plane. Hereafter, each point constituting a point cloud will be referred to as a coordinate point. That is, three-dimensional measurement data represents the three-dimensional shape of the target area as a set of coordinate points in the XYZ coordinate system.
[0034] This three-dimensional measurement data can be generated using equipment such as terrestrial laser scanners (TLS), mobile mapping systems (MMS), and unmanned aerial vehicles (UAVs), as well as technologies such as SLAM (Simultaneous Localization and Mapping).
[0035] The generated three-dimensional measurement data can be input to the information processing device 10 in advance through the data input / output unit 19 and stored in the storage 15. In S110, the processor 11 can acquire the three-dimensional measurement data by reading the specified three-dimensional measurement data from the storage 15.
[0036] In the subsequent S120, the processor 11 places a group of planar figures, specifically two-dimensional circular objects, in the three-dimensional virtual space to be displayed on the display 17 (see Figures 3A and 3B). The group of circular objects is arranged in the three-dimensional virtual space in a distribution corresponding to the point cloud shown by the three-dimensional measurement data.
[0037] In the subsequent S125, the processor 11 displays a three-dimensional virtual space on the display 17 in which circular objects are placed, thereby displaying a computer graphics (CG) on the display 17 that represents the three-dimensional shape of the features indicated by the three-dimensional measurement data.
[0038] The three-dimensional virtual space is a three-dimensional space that is virtually displayed through the display 17. Objects in the three-dimensional virtual space are displayed on the display 17 as computer graphics (CG) through an image transformation that projects them onto a two-dimensional plane corresponding to the display surface of the display 17.
[0039] The three-dimensional virtual space is represented by a three-dimensional coordinate system corresponding to the XYZ coordinate system described above. Here, the three-dimensional coordinate system of the three-dimensional virtual space is referred to as the UVW coordinate system. In the UVW coordinate system, the UV plane corresponds to the XY plane, and the W axis corresponds to the Z axis.
[0040] In S120, a circular object is prepared for each coordinate point that makes up the point cloud indicated by the three-dimensional measurement data. The circular object corresponding to each coordinate point is placed in the UVW coordinate system at the position corresponding to that coordinate point, with a predetermined orientation. The orientation will be described later.
[0041] At each coordinate point, a circular object of the same size and shape as the circular object corresponding to the other coordinate point is placed, and in the same orientation as the circular object corresponding to the other coordinate point. Figure 3B shows that circular objects are placed at the UVW coordinate system positions corresponding to each coordinate point in the XYZ coordinate system shown in Figure 3A. Figure 3B conceptually shows circular objects placed in a three-dimensional virtual interval when the line of sight is parallel to the V axis.
[0042] As shown in Figures 4A and 4B, the circular object has a surface in which the shades of color are expressed in gradation, i.e., gradient. Specifically, the surface of the circular object has the darkest part at the bottom and the lightest part at the top, with a part in between where the density changes gradually in one direction. Here, "up and down" refers to up and down along the Z-axis and W-axis directions.
[0043] Figure 4A shows the areas of a circular object's surface where the color is represented by a gradient, indicated by hatching. In the example shown in Figure 4A, the circular object has a gradient on its surface where the density gradually decreases in an upward-sloping direction, inclined from the vertical (i.e., the W-axis direction), from the bottom edge to a position slightly above the center of the circle.
[0044] To represent shadows, the bottom edge is black. As you move diagonally upwards from the bottom edge, the density of the black component in the corresponding area decreases, approaching the color of the top edge. In other words, as you approach the top edge, the density of the black component decreases to represent light. If the top edge is white, the surface of the circular object is represented with a gradient so that the whiteness is emphasized as you approach the top edge. Figure 4B is an example of a circular object with a gradient.
[0045] In three-dimensional measurement data, the three-dimensional shape of a feature, represented by a point cloud, is represented in a three-dimensional virtual space by a collection of circular objects that have such color representations on their surfaces.
[0046] In S125, the processor 11 performs an image transformation to project the three-dimensional virtual space, which includes a group of circular objects, onto a two-dimensional plane corresponding to the display surface of the display 17. Through this image transformation, the computer graphics (CG) of the circular objects in the three-dimensional virtual space are displayed on the display 17. The projection of the three-dimensional virtual space onto a two-dimensional plane and the resulting display of the CG on the display 17 can be achieved using known program modules.
[0047] In addition, the image transformation is performed so that a three-dimensional virtual space containing a group of circular objects is displayed on the display 17 as computer graphics (CG) viewed from a predetermined viewpoint and line of sight. That is, the image transformation is performed by projection onto a two-dimensional plane that takes into account the viewpoint and line of sight.
[0048] Furthermore, taking the above line of sight into consideration, in S120, each circular object is positioned in the three-dimensional virtual space facing the line of sight. That is, each circular object is positioned in the three-dimensional virtual space such that its surface is positioned perpendicular to the line of sight at a position in the UVW coordinate system corresponding to the position in the XYZ coordinate system of the corresponding coordinate point.
[0049] In addition, the information processing device 10 can accept operations to change the viewpoint and line of sight through the operation unit 18, and in S145, which will be described later, a group of circular objects in a three-dimensional virtual space viewed from a viewpoint and line of sight specified by the user is displayed on the display 17 as computer graphics.
[0050] At the point of displaying the CG in S125, user input for modification is not accepted. Therefore, in S120, each circular object is positioned in the three-dimensional virtual space facing the standard line of sight. In S125, the three-dimensional virtual space containing the group of circular objects is displayed on display 17 as CG seen from the standard viewpoint with the standard line of sight. For example, as shown in Figure 3B, the group of circular objects is displayed on display 17 as CG seen from the standard viewpoint with a line of sight parallel to the V-axis.
[0051] In addition, although all circular objects placed in the three-dimensional virtual space are the same size, when they are displayed on the display 17 as CG elements, they are displayed on the display 17 at a size corresponding to the perspective from the viewpoint due to projection onto the display surface.
[0052] After the CG is displayed on the display 17 by the processing in S125, the processor 11 accepts user input (S130). When an operation signal from the user is input through the operation unit 18 (Yes in S130), the processor 11 executes processing according to the operation signal.
[0053] When a viewpoint and line of sight change operation is performed (Yes in S140), the processor 11 rearranges the circular object in the three-dimensional virtual space so that it is facing the changed line of sight. The viewpoint and line of sight change operation is an operation to specify the viewpoint and line of sight input by the user through the operation unit 18.
[0054] The processor 11 performs image transformation based on the rearranged three-dimensional virtual space, and controls the display of CG on the display 17 so that a group of circular objects in the three-dimensional virtual space as seen from the changed viewpoint and line of sight is displayed as CG (S145). This display control causes the CG of the three-dimensional shape of the target area as seen from the changed viewpoint and line of sight to be displayed on the display 17. After the execution of process S145, the processor 11 returns to process S130 and waits for the input of the next operation signal.
[0055] On the other hand, when the processor 11 receives an operation to change the size of a circular object (Yes in S150), it rearranges a group of circular objects corresponding to the point cloud in the three-dimensional virtual space so as to change the size of the circular object from the standard size to the size specified by the user (S155). In other words, the processor 11 replaces a group of circular objects in the three-dimensional virtual space with a group of circular objects of the specified size.
[0056] The processor 11 performs an image transformation on the three-dimensional virtual space containing the rearranged group of circular objects, and controls the display of the CG on the display 17 so that the resized group of circular objects in the three-dimensional virtual space is displayed as CG at the corresponding size (S155). After that, the processor 11 returns to processing in S130 and waits for the next operation signal to be input.
[0057] The user can specify the size in real-world dimensions. The processor 11 can then place a circular object in the three-dimensional virtual space at a size corresponding to the specified real-world dimensions. Specifying the size in real-world dimensions helps the user intuitively understand the size and depth of the three-dimensional shape of the features displayed in the CG. For example, it can prevent the user from losing their sense of scale when the CG is scaled up or down.
[0058] In addition, when a termination operation is performed via the operation unit 18 (Yes in S160), the processor 11 closes the CG display screen shown on the display 17 (S170) and terminates the display control process shown in Figure 2.
[0059] When the processor 11 receives an operation signal other than the above-mentioned change operation and termination operation (No in S160), it performs processing according to the operation that was performed (S165), and then waits until the next operation signal is received (S130).
[0060] The information processing device 10 can, for example, receive an operation to specify a start point and an end point for distance measurement in a three-dimensional virtual space through the operation unit 18. When the processor 11 receives an input for specifying a start point and an end point (No in S160), it can perform a process to control the display 17 so that the distance from the start point to the end point is displayed as text on a scale in real space (S165).
[0061] The processor 11 can thus display the three-dimensional shape of the features of the target area represented by the three-dimensional measurement data as computer graphics on the display 17, and can also display additional information such as distances as needed. The processor 11 can measure distances using a method for measuring circular objects.
[0062] In the information processing device 10 of this embodiment described above, when placing objects at positions in a three-dimensional virtual space corresponding to a point cloud and displaying the three-dimensional shape of a feature corresponding to the point cloud using computer graphics, two-dimensional shapes, particularly circles, are used as objects, rather than three-dimensional shapes such as spheres.
[0063] A circular object can be defined by its center point and diameter or radius. That is, the processor 11 only needs to store information about the center point and diameter or radius as information about the circular object. Furthermore, the processor 11 can store texture information of the circular object's surface as common information for a group of circular objects.
[0064] Therefore, the processor 11 can display the three-dimensional shape of the feature corresponding to the point cloud on the display 17 with less processing load compared to when three-dimensional shapes are used as objects.
[0065] In this embodiment, a two-dimensional circular object having a gradient representation where the density increases from top to bottom is placed in a three-dimensional virtual space so as to be directly facing the line of sight, and this is displayed on the display 17 as CG. This makes it possible to achieve a three-dimensional CG display using a two-dimensional object.
[0066] The processor 11 places a group of circular objects of the same size in a three-dimensional virtual space and displays the CG corresponding to the group of circular objects on the display 17 by projection so that they are displayed with perspective according to their position in the three-dimensional virtual space. Therefore, the user can easily visually grasp the real-world size, depth, and relative distance between features of the three-dimensional shapes of the features corresponding to the displayed CG.
[0067] A circular object can be placed in a three-dimensional virtual space with a size that, for example, would be 30 cm or 50 cm in diameter in real space. This allows the three-dimensional shape of the feature to be displayed on the display 17 in a way that virtually represents the view when a 30 cm or 50 cm circular object is placed in real space.
[0068] Furthermore, as described above, the information processing device 10 can accept operations to specify the size of circular objects. Therefore, according to the information processing device 10 of this embodiment, it is possible to display the three-dimensional shape of a feature using CG with circular objects of a size corresponding to the sense of distance that the user wants to grasp.
[0069] [Other embodiments] This disclosure is not limited to the embodiments described above, and various forms can be adopted. For example, in the above embodiment, the information processing device 10 is configured to accept resizing operations for circular objects. However, the information processing device 10 may be configured not to accept resizing operations and to place fixed-size circular objects in a three-dimensional virtual space.
[0070] The objects placed at the coordinate points do not have to be circular; they may be other planar shapes such as rectangles. The gradient representation on the object surface only needs to be capable of representing light and shadow, and is not limited to those exemplified in Figures 4A and 4B.
[0071] The function of one component in the above embodiment may be distributed among multiple components. The functions of multiple components may be integrated into one component. Some parts of the configuration of the above embodiment may be omitted. At least some parts of the configuration of the above embodiment may be added to or replaced by the configuration of other above embodiments. Any aspect of the technical concept specified by the wording of the claims constitutes an embodiment of the present disclosure.
[0072] [Technical concepts disclosed in this specification] This specification can be understood to disclose the following technical concepts: [Item 1] A display control device for displaying three-dimensional computer graphics relating to the three-dimensional shape of an object on a display, An acquisition unit configured to acquire point cloud data representing the three-dimensional shape of the object as a set of coordinate points in a three-dimensional coordinate system, An operation receiving unit configured to accept specified operations related to gaze input through a user interface, A control unit configured to position predetermined planar figures at locations corresponding to each coordinate point in the point cloud data in a three-dimensional virtual space so as to be directly facing the line of sight specified by the specified operation, and to display the three-dimensional computer graphics relating to the three-dimensional shape of the object as seen from the line of sight, by projection of the three-dimensional virtual space, on the display; A display control device equipped with the following features. [Item 2] The display control device according to item 1, wherein the control unit places planar figures of the same size as planar figures at positions in the three-dimensional virtual space corresponding to each of the coordinate points, and displays the planar figures as elements of three-dimensional computer graphics on the display by projection, in a size corresponding to the distance from the viewpoint. [Item 3] The aforementioned operation receiving unit is capable of receiving a specification operation regarding the size of the planar figure, The control unit places the planar figures of the size specified by the size specification operation at the positions in the three-dimensional virtual space corresponding to each of the coordinate points, and displays the three-dimensional computer graphics on the display. The display control device described in item 2. [Item 4] The display control device according to any one of items 1 to 3, wherein the planar figure is a planar figure having a gradient representation in which the density increases from top to bottom in the three-dimensional virtual space. [Item 5] The aforementioned planar figure is a circle, as described in any one of items 1 to 4. [Item 6] A computer program for causing a computer to function as the acquisition unit, the operation reception unit, and the control unit in the display control device described in any one of items 1 to 5. [Item 7] A display control method for displaying three-dimensional computer graphics relating to a three-dimensional shape on a display, The three-dimensional shape of the aforementioned object is represented by a set of coordinate points in a three-dimensional coordinate system, and point cloud data is obtained. Accepting eye-gaze-related input through the user interface, A predetermined planar figure is positioned in a three-dimensional virtual space at a location corresponding to each coordinate point in the point cloud data, so as to be directly facing the line of sight specified by the specified operation, and the three-dimensional computer graphics relating to the three-dimensional shape of the object as seen from the line of sight are displayed on the display by projection of the three-dimensional virtual space. A display control method including the following. [Explanation of symbols]
[0073] 10... Information processing unit, 11... Processor, 13... Memory, 15... Storage, 17... Display, 18... Operation unit, 19... Data input / output unit.
Claims
1. A display control device for displaying three-dimensional computer graphics relating to the three-dimensional shape of an object on a display, An acquisition unit configured to acquire point cloud data representing the three-dimensional shape of the object as a set of coordinate points in a three-dimensional coordinate system, An operation receiving unit configured to accept specified operations related to gaze input through a user interface, A control unit configured to position predetermined planar figures at locations corresponding to each coordinate point in the point cloud data in a three-dimensional virtual space so as to be directly facing the line of sight specified by the specified operation, and to display the three-dimensional computer graphics relating to the three-dimensional shape of the object as seen from the line of sight, by projection of the three-dimensional virtual space, on the display; A display control device equipped with the following features.
2. The display control device according to claim 1, wherein the control unit places planar figures of the same size as planar figures at positions in the three-dimensional virtual space corresponding to each of the coordinate points, and displays the planar figures as elements of three-dimensional computer graphics on the display by projection, in a size corresponding to the distance from the viewpoint.
3. The aforementioned operation receiving unit is capable of receiving a specification operation regarding the size of the planar figure, The control unit places the planar figures of the size specified by the size specification operation at the positions in the three-dimensional virtual space corresponding to each of the coordinate points, and displays the three-dimensional computer graphics on the display. The display control device according to claim 2.
4. The display control device according to claim 1, wherein the planar figure is a planar figure having a gradient representation in which the density increases from top to bottom in the three-dimensional virtual space.
5. The display control device according to claim 1, wherein the planar figure is a circle.
6. A computer program for causing a computer to function as the acquisition unit, the operation reception unit, and the control unit in the display control device according to any one of claims 1 to 5.
7. A display control method for displaying three-dimensional computer graphics relating to a three-dimensional shape on a display, The three-dimensional shape of the aforementioned object is represented by a set of coordinate points in a three-dimensional coordinate system, and point cloud data is obtained. Accepting eye-gaze-related input through the user interface, A predetermined planar figure is positioned in a three-dimensional virtual space at a location corresponding to each coordinate point in the point cloud data, so as to be directly facing the line of sight specified by the specified operation, and the three-dimensional computer graphics relating to the three-dimensional shape of the object as seen from the line of sight are displayed on the display by projection of the three-dimensional virtual space. A display control method including the following.