Image display method, image display device, and program for image display device

Abstract

In the present invention, the range of visibility of an image which is on a display unit and which is based on incorporated three-dimensional information is enlarged, thereby increasing the convenience of observation based on the image. In the present invention, a first display region 19 and a plurality of second display regions 20 that surround the first display region 19 are set in a display unit 5, the display unit 5, by incorporating three-dimensional image information 12, is made to display an image 21 based on the three-dimensional display information 12 in the first display region 19. If one of the second display regions 20 is selected while the image 21 is displayed in the first display region 19, an image 22 is created that is obtained by rotating the three-dimensional image information 12 by a prescribed angle θ, using an axis, among virtual axes that pass through a central point O of the three-dimensional image information 12 that serves as the basis for the image 21 in the first display region 19, that extends in a direction orthogonal to a direction of arrangement of the selected second display region 20 and the first display region 19 and along the display unit 5 as an axis of rotation x, y, and the image 22 is displayed in the selected second display region 20.

Description

Image display method, image display device, and program for image display device

[0001] The present invention relates to a method for displaying an image on a display unit, an image display device that uses the image display method, and an image display device program that is used for the image display device.

[0002] A method using stereo images is known as a technique for imparting a three-dimensional effect to an image. In this method, two images with parallax are prepared as stereo images, and the stereo images are arranged side by side, with one image being viewed with the right eye and the other image being viewed with the left eye. In order to easily generate this stereo image from an existing image, it has been proposed to use luminance information of each pixel of the existing image as the depth of each pixel in the depth direction, as shown in Patent Document 1.

[0003] Recently, three-dimensional (hereinafter referred to as 3D) scanners have been used to collect 3D point cloud data, which is then processed using 3D modeling software to create 3D modeling data (data for expressing 3D objects, including those with added textures such as colors and photographs (3D image information)), and the 3D modeling data is then displayed on a two-dimensional display unit of a personal computer (PC) using a 3D viewer. In this case, by adjusting parameters such as the viewpoint and viewing angle, the 3D modeling data (3D object) can be rotated, allowing the user to view the image from any angle they like.

[0004] 3D modeling data, which provides such three-dimensional image information, is used for a variety of observations. Observation targets, such as ruins, plants, and animals, change over time (due to damage, decay, growth, etc.), making it difficult to observe them in the same condition. For this reason, 3D modeling data of the target is created and saved as needed using a 3D scanner or other device. This saved 3D modeling data is then displayed as an image of the target on a PC display using a 3D viewer. The image is then observed, taking into account its creation time. When doing so, the 3D modeling data that forms the basis of the target image is rotated (to change the viewpoint) as needed, and the orientation of the target image is adjusted to suit the observation.

[0005] Patent No. 5177668

[0006] However, when observing an image of an object to be observed, only one image of the object to be observed is displayed on the display unit of a PC, and even if the image of the object to be observed is based on 3D modeling data, it is displayed on a two-dimensional plane, and if the image of the object to be observed displayed on the display unit is viewed from the front, the further away from the front the image of the object to be observed becomes, the more difficult it becomes to see its side. Moreover, there are cases where one wants to use a specific image portion displayed on the display unit as a reference portion and compare and observe other portions, but if the reference portion and other portions are not displayed on the display unit at the same time, such observation cannot be made.

[0007] The present invention has been made in consideration of the above circumstances, and its first object is to provide an image display method that can expand the viewable range of an image on a display unit based on captured three-dimensional image information, thereby improving the convenience of image-based observation. A second object is to provide an image display device that uses the image display method. A third object is to provide an image display device program for use with the image display device.

[0008] In order to achieve the first object, the present invention has the following configurations (1) to (5): (1) An image display method for providing a display area on a display unit and displaying an image based on three-dimensional image information in the display area, comprising: providing a first display area and a second display area arranged juxtaposed to the first display area as the display area; importing three-dimensional image information to display an image based on the three-dimensional image information in the first display area; and when the second display area is selected while the image is being displayed in the first display area, creating an image by rotating the three-dimensional image information by a predetermined angle around a virtual axis passing through a center point of the three-dimensional image information that forms the basis of the image in the first display area, the virtual axis being a rotation axis that is perpendicular to the juxtaposition direction of the selected second display area and the first display area and that extends in a direction along the display unit, and displaying the image in the selected second display area.

[0009] According to this configuration, in addition to the image in the first display area, an image including a depth portion that is normally not visible simultaneously on the display unit is displayed in the second display area, thereby expanding the visible range of the image on the display unit based on the captured 3D image information and improving the convenience of observation based on the image. Moreover, in this case, the display image in the second display area juxtaposed to the first display area is obtained by rotating the 3D image information used in the first display area by a predetermined angle about a rotation axis that is orthogonal to the juxtaposition direction of the selected second display area and the first display area and extends in a direction along the display unit, among virtual axes passing through the center point of the 3D image information. Therefore, it is possible to provide an easily recognizable connection between the image in the first display area and the image in the second display area, and it is possible to minimize the inconvenience of using the image in the first display area and the image in the second display area. Therefore, when a specific portion of an image of an object to be observed is identified as a reference portion during observation, and other portions of the image are observed in relation to the reference portion, even if the other portions are located deep in the image and cannot be simultaneously displayed in the first display area of ​​the display unit, the reference portion and other portions can be displayed using the first and second display areas. Moreover, in this case, a common image portion is displayed in the first and second display areas, which allows for an easily recognizable connection between the image in the first display area and the image in the second display area, thereby enabling effective observation.

[0010] (2) Under the configuration of (1), a plurality of the second display areas are provided around the first display area so as to surround the first display area, and the plurality of second display areas are configured as a plurality of pairs of second display areas facing each other with the first display area in between, and each pair of second display areas in each set is set as the selected second display area, and when the image is displayed in each second display area, the three-dimensional image information used in each second display area is three-dimensional image information that forms the basis of the display image in the first display area, rotated by a predetermined angle in opposite rotation directions around the rotation axis based on the state of the three-dimensional image information.

[0011] According to this configuration, in each pair of second display areas facing each other with the first display area in between, an image can be displayed that continues on both sides of the display image of the first display area as a reference, and that includes parts (depth parts) that cannot normally be viewed simultaneously on the display unit, thereby further expanding the viewing range of the image on the display unit based on the captured three-dimensional image information.

[0012] (3) Under the configuration of (2), the first display area is set to a rectangular shape, and the plurality of second display areas are arranged contiguously at least on both sides of the first display area in the vertical direction and on both sides of the horizontal direction, with the first display area at the center. The second display areas on both sides of the first display area in the vertical direction and the second display areas on both sides of the first display area in the horizontal direction are each used as a pair of second display areas in each set.

[0013] With this configuration, it is possible to display an image including parts (depth parts) that cannot normally be viewed simultaneously on the display unit above, below, left, and right, with the display image in the first display area at the center.This makes it possible to take advantage of human sensitivity to the up, down, left, and right, while expanding the visible range of the image on the display unit based on the captured three-dimensional image information, thereby improving the observation effect based on the image.

[0014] (4) Under the configuration of (1), when creating an image by rotating the three-dimensional image information by a predetermined angle, the direction of rotation is set such that, in the image to be created, the image portion of the first display area that is farther from the selected second display area is no longer displayed due to the rotation.

[0015] According to this configuration, the selected second display area can display an image that includes the depth portion (non-visible portion) of the display image in the first display area on the selected second display area side, and human sensitivity can be utilized when grasping the image in the expanded visible range based on the position of the selected second display area and the image newly displayed by the selected second display area (the selected second display area displays the portion of the display image in the first display area that was not visible on the selected second display area side).

[0016] (5) Under the configuration of (1), provided that the attitude of the display image in the first display area can be changed, when it is detected that the attitude of the display image in the first display area has been changed while the display image is being displayed in the second display area, an image is created by rotating the three-dimensional image information that forms the basis of the display image in the attitude-changed first display area by a predetermined angle around the rotation axis, and the display image in the second display area is changed to this image.

[0017] According to this configuration, when images are displayed in both the first and second display areas, if the display image in the first display area is changed in position, the display image in the second display area is also changed accordingly to correspond to the change in position of the display image in the first display area, and the relationship between the two images in the first and second display areas can be maintained even when the display image in the reference first display area is changed in position.

[0018] In order to achieve the second object, the present invention has the following configurations (6) to (9).

[0019] (6) An image display device that provides a display area on a display unit and displays an image based on three-dimensional image information in the display area, comprising: a storage unit that stores display area setting information for setting the display area; a display area setting unit that reads out the display area setting information from the storage unit and sets a first display area and a second display area arranged side by side with the first display area on the display unit; a first display processing unit that displays an image based on the three-dimensional image information in the first display area set by the display area setting unit on condition that three-dimensional image information is taken in; and a second display processing unit that, when an image based on the three-dimensional image information is displayed in the first display area based on the first display processing unit and it is detected that the second display area has been selected, creates an image by rotating the three-dimensional image information by a predetermined angle around a virtual axis that passes through a center point of the three-dimensional image information that is the basis of the image in the first display area and that extends in a direction along the display unit and perpendicular to the juxtaposition direction of the selected second display area and the first display area as a rotation axis, and displays the image in the selected second display area. The configuration is such that it includes the following.

[0020] According to this configuration, it is possible to specifically provide an image display device that uses the image display method (1) above.

[0021] (7) Under the configuration of (6), the first display area is set to be rectangular, and the second display areas are multiple, so that the multiple second display areas are set to be continuously arranged at least on both the vertical and horizontal sides of the first display area, with the first display area as the center; and when the second display processing unit displays the image in each second display area on both the vertical and horizontal sides of the first display area, the second display processing unit is configured to use, as the three-dimensional image information to be used in each second display area, three-dimensional image information that forms the basis of the display image in the first display area, rotated by a predetermined angle in opposite rotation directions around the rotation axis, based on the state of the three-dimensional image information.

[0022] According to this configuration, it is possible to specifically provide an image display device that uses the image display methods (2) and (3) above.

[0023] (8) Under the configuration of (6), the second display processing unit is configured such that when creating an image by rotating the three-dimensional image information by a predetermined angle, the rotation direction is such that, in the created image, the image portions of the first display area that are farther from the selected second display area are no longer displayed due to the rotation.

[0024] According to this configuration, it is possible to specifically provide an image display device that uses the image display method (4) above.

[0025] (9) Under the configuration of (6), a third display processing unit is provided that, upon input of rotation instruction information, rotates three-dimensional image information that forms the basis of the display image in the first display area, thereby changing the attitude of the display image in the first display area; the second display processing unit is configured to determine that a display image is being displayed in the second display area based on information from the second display processing unit, and when it determines that the attitude of the display image in the first display area has been changed based on information from the third display processing unit, to create an image obtained by rotating the three-dimensional image information that forms the basis of the display image in the changed attitude first display area by a predetermined angle around the rotation axis in accordance with the change in attitude of the display image in the first display area, and to change the display image in the second display area to that image.

[0026] According to this configuration, it is possible to specifically provide an image display device that uses the image display method (5) above.

[0027] In order to achieve the third object, the present invention has the following configuration (10).

[0028] (10) A program for an image display device includes a storage device that stores display area setting information for setting a display area on a display unit, and in the presence of the storage device, reads the display area setting information from the storage device to set a display area on the display unit and displays an image based on three-dimensional image information in the display area, the program comprising: a computer; a display area setting unit that sets a first display area and a second display area arranged in parallel to the first display area as the display area on the display unit; a first display processing unit that displays an image based on the three-dimensional image information in the first display area set by the display area setting unit on the condition that three-dimensional image information is taken in; When an image based on the three-dimensional image information is displayed in the first display area based on the first display processing unit and it is detected that the second display area has been selected, the second display processing unit is configured to function as a second display processing unit that creates an image by rotating the three-dimensional image information by a predetermined angle along a virtual axis that passes through the center point of the three-dimensional image information that forms the basis of the image in the first display area and that extends in a direction perpendicular to the juxtaposition direction of the selected second display area and the first display area and along the display unit as a rotation axis, and displays the image in the selected second display area.

[0029] According to this configuration, it is possible to provide a program for an image display device used in the image display device of (6) above.

[0030] According to the present invention, it is possible to provide an image display method that can expand the visible range of an image on a display unit based on imported three-dimensional image information, thereby improving the convenience of image-based observation, an image display device that uses this image display method, and a program for an image display device that is used with this image display device.

[0031] 1 is an explanatory diagram showing the configuration of an image display device according to an embodiment. FIG. 1 is an explanatory diagram illustrating display areas according to an embodiment that are set on a display unit of an image display device. FIG. 1 is an explanatory diagram illustrating 3D modeling data being imported and an image based on the 3D modeling data being displayed in a first display area. FIG. 1 is an explanatory diagram illustrating processing when a second display area is selected while an image is displayed in the first display area. FIG. 2 is an explanatory diagram illustrating the state of each image when images are displayed in the first and second display areas. FIG. 2 is an explanatory diagram illustrating obtaining an image in the second display area by rotating 3D modeling data (displayed as a 3D figure) by a predetermined angle. FIG. 3 is an explanatory diagram illustrating how, when second display areas on both the vertical and horizontal sides of the first display area are selected, images are displayed in each of the second display areas. FIG. 4 is an explanatory diagram illustrating changing the orientation of the displayed image in the first display area. FIG. 5 is an explanatory diagram illustrating the flow of an image display method according to an embodiment. FIG. 6 is a flowchart illustrating an example of processing from identifying 3D modeling data to be imported as an observation target to selecting a second display area. FIG. 7 is a flowchart illustrating an example of processing when the orientation of the image in the first display area is changed while images exist in the first display area and the second display area.

[0032] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An image display device 1 according to this embodiment is configured by incorporating (installing) image display processing software into a personal computer (hereinafter referred to as a PC), and an image display method according to this embodiment is used in the PC incorporating the image display processing software. Therefore, the image display device 1 according to this embodiment will be described, and then the operation of the image display device 1 and the image display method according to this embodiment will be described.

[0033] 1, the image display device 1 according to this embodiment is equipped with a communication unit 2 that receives information from the outside, an operation input unit (mouse, keyboard) 3 that inputs requested operations, an information input / output unit (input / output terminal, media drive, etc.) 4 that imports information from a removable storage medium, a display unit (also a touch panel) 5 that displays various information as images, etc., a storage unit 6 that serves as a storage device for storing various information, and an arithmetic and control unit 7. Of these, the communication unit 2 and the information input / output unit 4 can import, from the outside, the image display processing software, 3D modeling data as three-dimensional image information including three-dimensional coordinates used in the image display processing software, etc.

[0034] The storage unit 6 is composed of memory elements such as a read-only memory (ROM) and a random-access memory (RAM). The storage unit 6 includes a program storage unit 8, a 3D modeling data storage unit 9, a setting information storage unit 10, and an image display information storage unit 11. The program storage unit 8 stores various programs (image display processing software) for operating the image display device 1, in addition to basic PC programs. The 3D modeling data storage unit 9 stores 3D modeling data (3D image information including 3D coordinates in 3D space) for various observation targets. The setting information storage unit 10 stores display area information (e.g., area pattern, area shape, size, number, and area placement) used to set the display area on the display unit 5, as well as setting information such as the predetermined angle, rotation direction, and rotation speed for rotating the 3D modeling data (rotational transformation of 3D coordinates (world transformation or view transformation)) used in the image display processing described below. The image display information storage unit 11 stores image information with new information added each time processing information is incorporated into the display area information. Other necessary information is stored in the storage unit 6 as appropriate.

[0035] In this embodiment, the calculation control unit 7 is composed of a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit), and the calculation control unit 7 functions as a display area setting unit 15, first to third display processing units 16 to 18, etc. based on a program read from the memory unit 6, as shown in FIG. 1.

[0036] As shown in FIG. 2, the display area setting unit 15 has a function of setting a first display area 19 and a plurality of second display areas 20 on the display unit 5 .

[0037] For this reason, the display area setting unit 15 reads out the display area information from the memory unit 6 (setting information saving unit 10) and, on the display unit 5, positions the first display area 19 in the center and displays a plurality of second display areas 20 surrounding the first display area 19. Specifically, the first display area 19 and each of the second display areas 20 are formed in a rectangular shape, and the plurality of second display areas 20 are configured with a plurality of pairs of second display areas 20 facing each other with the first display area 19 sandwiched therebetween. In this embodiment, the plurality of pairs of second display areas 20 are configured with the first display area 19 at the center as second display areas 20u, 20d on both sides of the first display area 19 in the vertical direction, second display areas 20r, 20l on both sides of the first display area 19 in the horizontal direction, and second display areas 20rdu, 20rdd (20ldu, 20ldd) on both diagonal sides of the first display area 19 in the left and right directions. The first display area 19 and each of the multiple second display areas 20 constitute a window that independently performs a display function, and the first display area 19 and each of the multiple second display areas 20 can display an image of the object to be observed based on the performance of their display functions.

[0038] 3, the first display processing unit 16 performs the function of importing 3D modeling data (hereinafter, for convenience, referred to as a 3D figure and designated by the reference numeral 12) as three-dimensional image information including three-dimensional coordinates, and displaying an image 21 based on the 3D modeling data 12 in the first display area 19. In this case, the 3D modeling data 12 in FIG. 3 and the image 21 based thereon (the image shown in the first display area 19) are based on a simplified Earth model.

[0039] When the 3D modeling data 12 is used, it is read from the memory unit 6 (3D modeling data storage unit 9) of the image display device 1 or directly input from an external device. This 3D modeling data 12 is used to ultimately display a three-dimensional object (observation target) as a two-dimensional image on the display unit 5, which forms a two-dimensional plane, based on image display processing software. The 3D modeling data 12 is generally prepared as follows: First, a three-dimensional scanner is used to irradiate the observation target (object) with a laser or light, and the reflected light is detected by a sensor, thereby acquiring the surface shape of the observation target as three-dimensional coordinate data. Next, the acquired coordinate data is processed using dedicated software to convert it into point cloud data, which is then processed using CAD software or the like to convert it into formats such as polygon data or mesh data. Texture and color information are then added to the polygon data or mesh data, and the resulting processed data is the 3D modeling data 12 for the observation target.

[0040] When displaying an image 21 based on the 3D modeling data 12 in the first display area 19 (display unit 5), the first display processing unit 16 creates the 3D modeling data 12, including three-dimensional coordinates, as the image 21 to be displayed in the first display area 19. To this end, the first display processing unit 16 performs necessary known 3D to 2D (two-dimensional) conversion processing (projection coordinate conversion, screen coordinate conversion, etc.). In this case, the reason why the first display processing unit 16 displays the image 21 based on the 3D modeling data 12 in the first display area 19 is to enable the invisible depth areas around the image to be displayed in multiple second display areas 20 surrounding it.

[0041] 4 to 6, when an image 21 based on the 3D modeling data 12 is displayed in the first display area 19 and when it is detected that at least one of the plurality of second display areas 20 has been selected, the second display processing unit 17 exhibits a function of displaying an image 22 obtained by rotating the display image 21 of the first display area 19 by a predetermined angle in the selected second display area 20. As an example of the above aspect, FIG. 4 shows that when a second display area 20r on one lateral side of the first display area 19 (the right side in FIGS. 4 to 6) is selected, an image 22 obtained by rotating the display image 21 of the first display area 19 by a predetermined angle is displayed in the second display area 20r.

[0042] In this case, whether "image 21 is displayed in first display area 19" is determined based on information from first display processing unit 16, and whether "at least one of the plurality of second display areas 20 has been selected" is determined by detecting the presence or absence of selected information when pointer P is positioned on one of second display areas 20 and a mouse click operation as input means is performed (see the diagram on the right side of the upper row in FIG. 4), or by detecting the presence or absence of selected information when one of the second display areas 20 is touched with a finger on display unit 5. Selection of this second display area 20 can be performed on one or more second display areas 20 within the number of second display areas 20, and further, the selection can be performed simultaneously or after a certain interval.

[0043] In the above process of "displaying an image 22 obtained by rotating the display image 21 of the first display area 19 by a predetermined angle in the selected second display area 20," information regarding the 3D modeling data 12 that forms the basis of the display image 21 in the first display area 19 is retrieved from the storage unit 6 (image display information storage unit 11), and the 3D modeling data 12 is used. This is because, by using a rotational (rotational) transformation of three-dimensional coordinates based on the viewpoint (world coordinate transformation, view coordinate transformation), it is possible to create an image that shows a depth portion that cannot be displayed in the display image 21 in the first display area 19. For this reason, the 3D modeling data 12 is rotated by a predetermined angle with the rotation axis set to "a virtual axis that passes through the center point of the 3D modeling data 12, in a direction perpendicular to the juxtaposition direction of the selected second display area 20 and the first display area 19 and that extends in a direction along the display unit 5," and an image for the selected second display area 20 is created based on the image rotated by the predetermined angle. In this case, the above-mentioned axis is used as the rotation axis in order to take advantage of the parallel arrangement relationship between the selected second display area 20 and the first display area 19, and to create an easily recognizable connection between the image 21 in the first display area 19 and the image 22 in the second display area 20.

[0044] Specifically, a case where the second display area 20r located on one lateral side of the first display area 19 (the right side in FIGS. 4 and 5 ) is selected will be described with reference to FIG. 6 . When the second display area 20r is selected, the 3D modeling data 12 (as displayed) that forms the basis of the display image 21 in the first display area 19 is retrieved from the memory unit 6 (image information storage unit 11). At this time, the juxtaposition direction of the selected second display area 20r and the first display area 19 is the left-right direction, and the direction perpendicular to the juxtaposition direction and along the display unit 5 is the up-down direction. Therefore, the rotation axis is set to the rotation axis y extending in the up-down direction, and the 3D modeling data is rotated by a predetermined angle θ (rotational transformation of three-dimensional coordinates). This is then converted into an image 22 (2D converted) through 2D coordinate transformation, and the image 22 is displayed in the selected second display area 20r. As a result, an image 22 including the depth portion of the display image 21 in the first display area 19 is displayed in the second display area 20r, which is aligned horizontally with the first display area 19. In Fig. 6, the symbol O indicates the center point of the 3D modeling data (3D figure), and the symbols W0 and Wθ are imaginary angle grasping planes provided to make it easier to understand the angle (0°, θ°) associated with rotation. Also, x indicates an imaginary rotation axis perpendicular to the rotation axis y.

[0045] Furthermore, the "predetermined angle θ" when rotating the 3D modeling data 12 is preferably a predetermined value of 90 degrees or less, and more preferably a predetermined value of 50 degrees or less, specifically a predetermined value in the range of 20 degrees to 40 degrees, with the state of the image 21 displayed in the first display area 19 as the reference (state W0 in FIG. 6 ). This is because the display image 22 in the second display area 20 displays not only the depth portion that cannot be displayed (visibly recognized) in the display image 21 in the first display area 19, but also the common portion with the display image 21 in the first display area 19, thereby providing an easily recognizable connection between the display image 21 in the first display area 19 and the display image 22 in the second display area 20.

[0046] Furthermore, the "rotation direction" of the predetermined angle θ around the rotation axis may be either a direction in which the image 22 to be created is rotated so that the image portions of the display image 21 in the first display area 19 that are farther from the selected second display area 20 are not displayed, or a direction opposite to that direction. However, as shown in Figures 4 to 6, the former direction (the direction of the arrow in Figure 6) is preferable. This is because, of the depth portions of the display image 21 in the first display area 19, those closer to the selected second display area 20 are displayed in the selected second display area 20, which makes it easier for the human eye to recognize that the visible range has been expanded and to identify the expanded portion.

[0047] 7 shows a case where second display areas 20r and 20l are arranged on both lateral sides of first display area 19, and second display areas 20u and 20d are arranged on both vertical sides of first display area 19 are selected. In this case, as shown in FIG. 7, the depth portions of display image 21 in first display area 19 on both lateral sides (left and right sides in FIG. 7) are displayed as images 22 in second display areas 20r and 20l on both lateral sides (left and right sides in FIG. 6) of first display area 19, and the depth portions of display image 21 in first display area 19 on both vertical sides (top and bottom sides in FIG. 6) are displayed as images 22 in second display areas 20u and 20d on both vertical sides (top and bottom sides in FIG. 7) of first display area 19. This significantly expands the field of view. In this case, when an image 22 is displayed in the second display areas 20u, 20d on both the top and bottom sides of the first display area 19 (both the top and bottom sides in Figure 7), the rotation axis of the 3D modeling data 12 that forms the basis of the image 22 extends horizontally through the center of the 3D modeling data 12 (see rotation axis x in Figure 6).

[0048] Of course, when rotating the display image 21 in the first display area 19 by a predetermined angle to create the image 22 to be displayed in the selected second display area 20, various known coordinate transformation processes (projection coordinate transformation, screen coordinate transformation, etc.) are performed.

[0049] The third display processing unit 18 has the function of changing the posture of the image displayed in the first display area 19 when an image is displayed in the first display area 19, in response to an input operation.

[0050] For this reason, the third display processing unit 18 is able to determine whether or not an image is being displayed in the first display area 19 based on information from the first display processing unit 16. When it is determined based on information from the first display processing unit 16 that an image is being displayed in the first display area 19, by performing a mouse operation or the like in the first display area 19, the 3D modeling data that forms the basis of the image displayed in the first display area 19 can be rotated (a three-dimensional coordinate rotation transformation (world coordinate transformation or view coordinate transformation)) by that mouse operation.

[0051] 8 , the 3D modeling data can be rotated (rotational transformation of the three-dimensional coordinates) around one of the virtual axes passing through the center point O of the 3D modeling data (form formed by the three-dimensional coordinates), which extends in a direction along the display unit 5, for example, x or y, as the rotation axis. To rotate the 3D modeling data, a drag operation (mouse operation) is performed in the first display area 19 by pressing the mouse button and moving the cursor. This drag operation specifies the rotation axis, the amount of rotation, the rotation direction (indicated by the straight lines mx and my with arrows), etc. As a result, if necessary, an arbitrary rotation axis can be selected to rotate the 3D modeling data, thereby changing the orientation of the display image 21 in the first display area 19, and the orientation of the display image 21 in the first display area 19 can be determined as desired by the viewer.

[0052] In addition, when the second display processing unit 17 determines that the position of the display image 21 in the first display area 19 has been changed while the display images 21 and 22 are displayed in the first display area 19 and the second display area 20, it also performs the function of changing the display image 22 in the second display area 20 to one that is rotated by a predetermined angle θ relative to the display image in the first display area 19 whose position has been changed, in accordance with the change in position of the display image 21 in the first display area 19.

[0053] For this reason, the second display processing unit 17 determines whether or not a display image 21 is being displayed in the first display area 19 based on information from the first display processing unit 16, determines whether or not a display image 22 is being displayed in the second display area 20 based on information from the second display processing unit 17 itself, and further determines whether or not the position of the display image 21 in the first display area 19 has been changed based on information from the third display processing unit 18. When the second display processing unit 17 determines that all of these conditions are met, it performs the above-mentioned function of the second display processing unit 17 in the second display area 20 in which the display image 22 is being displayed. That is, depending on the second display area 20 in which the display image is being displayed, the second display processing unit 17 rotates the 3D modeling data that forms the basis of the display image 21 in the first display area 19 (including an image whose position is being changed) by a predetermined angle around a rotation axis corresponding to each second display area 20, and changes the display image 22 in each second display area 20 to the image at that time. Of course, in this case, the various known coordinate conversion processes (world coordinate conversion, view coordinate conversion, projection coordinate conversion, screen coordinate conversion) described above are also performed to create the image 22 to be displayed in the second display area 20. In this case, the image 22 will ultimately be at a predetermined angle θ with respect to the display image 21 in the first display area 19, but the image 22 may be recognizable as an image that changes over time by adjusting the rotation speed of the 3D modeling data until it reaches its final state.

[0054] Next, the operation of the image display device 1 and the image display method used in the image display device 1 will be described in accordance with the operation and processing sequence shown in FIG.

[0055] (1) First, when the image display processing software according to this embodiment is activated in the image display device 1 and display area information is read, a first display area 19 and a plurality of second display areas 20 are displayed on the display unit 5 of the image display device 1 as display areas, with the plurality of second display areas 20 surrounding the first display area 19 (see FIG. 2). This is to enable the display of a core image 21 that is representative of the object of observation and an image 22 that includes any of the depth portions around the core image 21, thereby expanding the viewable range of the image of the object of observation on the display unit 5 (see FIG. 7).

[0056] (2) Next, with the first display area 19 and the plurality of second display areas 20 displayed on the display unit 5, when 3D modeling data serving as the three-dimensional image information underlying the object of observation is identified from among those stored in the storage unit 6, the 3D modeling data is imported, and an image based on the 3D modeling data is displayed in the first display area 19 (see FIG. 3 ). This is because the viewable range of the object of observation in any direction can be expanded by displaying a core image 21 of the object of observation in the first display area 19 and using the plurality of second display areas 20 arranged around the core image 21.

[0057] At this time, the orientation of the image 21 displayed in the first display area 19 can be changed by rotating the underlying 3D modeling data (rotational transformation of three-dimensional coordinates) to make the object of observation easier to observe. Therefore, when it is desired to change the orientation of the displayed image 21 in the first display area 19, the orientation change is performed by operating the mouse (see FIG. 8).

[0058] (3) Next, with an image 21 based on the 3D modeling data displayed in the first display area 19, when at least one of the second display areas 20 surrounding the first display area 19 is selected, an image 22 is created in which the 3D modeling data used in the first display area 19 (the same 3D modeling data) is rotated by a predetermined angle according to the selected second display area 20 (using the various known coordinate conversion processes described above (world coordinate conversion, view coordinate conversion, projection coordinate conversion, and screen coordinate conversion processes)), and the image 22 is displayed in the selected second display area 20 (see FIGS. 4 and 5 ). This is because the image 22, including the invisible depth portion of the image 21 displayed in the first display area 19, is displayed in the selected second display area 20 to expand the visible range of the image 21 to be observed. At this time, at least one of the second display areas 20 can be selected by simply clicking or touching an area of ​​the second display area 20, and the image 22 is displayed in the second display area 20. Therefore, when the image 22 is displayed in the second display area 20, the burden and hassle on the observer when observing can be minimized.

[0059] (4) Furthermore, when the display image 21 in the first display area 19 is displayed and the display image 22 in the second display area 20 is displayed, if it is desired to change the orientation of the display image 21 in the first display area 19, the orientation of the display image 21 in the first display area 19 is changed by operating the mouse (see FIG. 8 ). Accordingly, an image 22 is created by rotating the 3D modeling data that forms the basis of the display image 21 in the first display area 19 whose orientation has been changed by a predetermined angle around the aforementioned rotation axis, and the display image 22 in the second display area 20 is changed to this new image 22. This is to maintain the relationship between the images 21 and 22 in the first and second display areas 19 and 20 even when the orientation of the display image 21 in the first display area 19 that serves as the reference is changed.

[0060] Therefore, in the image display device 1 (image display method), the first display area 19 and the second display area 20 are used to expand the viewable range of the image on the display unit 5 based on the imported 3D modeling data, thereby improving the convenience of image-based observation. Moreover, an easily recognizable connection can be provided between the image 21 in the first display area 19 and the image 22 in the second display area 20, and ease of use can be minimized even when using the image 21 in the first display area 19 and the image 22 in the second display area 20.

[0061] Next, a processing example in the image display device 1 will be specifically described with reference to the flowchart shown in Fig. 10. Fig. 10 shows a processing example from the identification of 3D modeling data (shown as 3D data in Fig. 10) to be imported as an observation target to the selection of the second display area 20. The symbol S in Fig. 10 indicates a step. It is assumed that the 3D modeling data 12 has been stored in advance in the storage unit 6 (3D modeling data storage unit 9).

[0062] First, when the image display processing software is started, in S1 it is determined whether or not this is the first time that the image 21 is being displayed in the first display area 19. If S1 is YES, in S2 the display area information is read, and the first display area 19 and the multiple second display areas 20 arranged around it are displayed on the display unit 5. Next, in S3 it is determined whether or not the 3D modeling data to be imported has been identified (specified, selected). If S3 is NO, the determination in S3 is repeated to wait for the 3D modeling data to be identified. On the other hand, if S3 is YES, the 3D modeling data (3D data in the figure) is read from the memory unit 6, and an image based on that data is displayed in the first display area 19 (S4, S5).

[0063] Next, in S6, it is determined whether an image 22 already exists in the second display area 20. This is to address the possibility that an image may already exist in the second display area 20 if the image 21 display process in the first display area 19 is performed for the first time or later. Therefore, if S6 is YES, the image 22 in the second display area 20 is erased in S7. If S6 is NO, the process proceeds directly to S8, where it is determined whether images 22 are displayed in all of the second display areas 20 to determine whether any of the second display areas 20 is available for use. Initially, since S8 is NO, the process proceeds to S9, where it is determined whether an unused second display area 20 has been selected. If S9 is NO, the process returns to S8. If S9 is YES, the process returns to S8. If S9 is YES, the image 22 for the selected second display area 20 is created in S10 using the 3D modeling data used in the first display area (or the same 3D modeling data). In the next step S11, the image 22 is displayed in the second display area 20 selected in step S9, and the process returns to step S8, where observation continues using the image displayed in the selected second display area 20 and the image displayed in the first display area 19.

[0064] If S8 is YES, the image 22 is being displayed in all of the second display areas 20, and in this case, in S12, it is determined whether or not to terminate the operation (series of processes) based on the observer's operational input. If S12 is YES, the operation is terminated, whereas if S12 is NO, the process returns to S1. This means that the display process of the image 21 in the first display area 19 is the first or subsequent time (S1 NO), and the process proceeds to S3, where observation continues with the image 22 being displayed in all of the second display areas 20 unless new 3D modeling data is specified.

[0065] 11 shows an example of processing when the orientation of the image 21 in the first display area 19 is changed when images 21 and 22 are present in the first display area 19 and the second display area 20. The symbol Q in FIG. 11 indicates a step.

[0066] First, in Q1, it is determined whether or not an image 21 is being displayed in the first display area 19. This is to determine whether or not there is a display image 21 in the first display area 19 whose orientation may be changed. Therefore, if Q1 is NO, the processing ends, while if Q1 is YES, it is determined in Q2 whether or not the 3D modeling data that forms the basis of the display image 21 in the first display area 19 has been rotated (whether or not the orientation has been changed). If Q2 is NO, the processing returns to Q1, while if Q2 is YES, an image with a changed orientation is created as the display image 21 in the first display area 19 in Q3, and the changed orientation image is displayed as the display image 21 in the first display area 19 in the next Q4.

[0067] In the next step Q5, it is determined whether or not image 22 is being displayed in second display area 20. This is to determine whether image 22 in second display area 20 needs to be changed in response to a change in the orientation of image 21 in first display area 19. Therefore, if Q5 is NO, the processing ends. However, if Q5 is YES, in Q6, an image 22 for display in second display area 20 is created that is changed in response to the change in orientation of the image 21 in first display area 19, and this image is displayed in second display area 20 as image 22 for second display area 20 in the next step Q7.

[0068] Although the embodiments have been described above, the present invention also encompasses the following aspects. (1) The number of second display areas 20 may be one. (2) The dividing lines of the first and second display areas 19, 20 as display areas may be set to be transparent or invisible. (3) When creating an image for the second display image 20 obtained by rotating three-dimensional image information by a predetermined angle θ, the rotation direction may be set to a direction such that, in the created image, the image portion of the image displayed in the first display area closer to the selected second display area is hidden by the rotation. In this case, if the second display area 20 is selected by moving the second display area 20 to be selected from the first display area 19 in a dragging operation or by touching the display unit 5 with a finger, the selection operation can be tailored to human sensibilities in relation to the image 22 displayed in the selected second display area 20. Furthermore, such an operation and the display of the image 22 in the second display area 20 based on that operation may be used alone or together with the selection (click operation, etc.) of the second display area 20 in the above embodiment, and the display state of the image 22 in the selected second display area 20 may be changed depending on the selection operation. (4) A plurality of second display areas 20 surrounding the first display area 19 may be formed over multiple layers, not just one, and the predetermined angle of each layer (second display area 20) may be increased the farther from the first display area 19. (5) When new 3D modeling data is loaded as the image 21 in the first display area 19, an image 21 based on the newly loaded 3D modeling data may be displayed in the first display area 19, replacing the display image 21 that was previously displayed in the first display area 19. (6) The image display device 1 is configured to include a terminal device and a server device that can communicate with the terminal device via a network, and the server device is provided with a memory unit 6 that performs processing similar to that of the above-described embodiment, a display area setting unit 15, and first and second display processing units 16 and 17 (and, if necessary, a third display processing unit 18).That is, the image display device of (6) shown in paragraph

[0019] may be configured to include a terminal device having the display unit 6 and a server device capable of communicating with the terminal device via a network, the server device being provided with the storage unit 6, the display area setting unit 15, the first display processing unit 16, and the second display processing unit 17, the display area setting unit 15, the first display processing unit 16, and the second display processing unit 17 being configured to perform processing in accordance with input information from the terminal device and send each piece of processing information back to the terminal device, and the terminal device being configured to display an image in accordance with each piece of processing information on the display unit 6 of the terminal device, on condition that it receives each piece of processing information. This makes it possible to provide an apparatus that embodies the image display device of (6) shown in paragraph

[0019] above, which uses the image display method of (1) shown in paragraph

[0008] over a network.

[0069] The present invention can be used to expand the viewable range of an image on the display unit 5 based on imported 3D modeling data (three-dimensional image information), thereby improving the convenience of observation based on the image.

[0070] REFERENCE SIGNS LIST 1 Image display device 5 Display unit 6 Memory unit (storage device) 7 Arithmetic and control unit 12 3D modeling data (three-dimensional image information) 15 Display area setting unit 16 First display processing unit 17 Second display processing unit 18 Third display processing unit 19 First display area 20 Second display area 21 Image displayed in first display area 22 Image displayed in second display area O Center point of 3D modeling data (3D figure) P Pointer x, y Rotation axis θ Predetermined angle

Claims

1. In an image display method in which a display area is provided in a display unit and an image based on three-dimensional image information is displayed in the display area, the display area is provided with a first display area and a second display area arranged side by side with the first display area; by capturing three-dimensional image information, an image based on the three-dimensional image information is displayed in the first display area; when the second display area is selected while the image is being displayed in the first display area, among the virtual axes passing through the center point of the three-dimensional image information that is the basis of the image in the first display area, an axis that extends in a direction along the display unit and is orthogonal to the juxtaposition direction of the selected second display area and the first display area is used as a rotation axis, and an image obtained by rotating the three-dimensional image information by a predetermined angle is created and the image is displayed in the selected second display area. An image display method characterized by the above.

2. According to claim 1, a plurality of the second display areas are provided so as to surround the first display area around the first display area; the plurality of second display areas are configured with a plurality of pairs of second display areas that face each other with the first display area interposed therebetween; when the image is displayed in each of the pair of second display areas in each set as the selected second display area, as the three-dimensional image information used in each of the second display areas, the three-dimensional image information that is the basis of the display image in the first display area is rotated by a predetermined angle in opposite rotation directions around the rotation axis based on the state of the three-dimensional image information. An image display method characterized by the above.

3. According to claim 2, the first display area is set to be rectangular, and the plurality of second display areas are arranged continuously at least on both the upper and lower sides and both the left and right sides in the first display area with the first display area as the center; each of the second display areas on both the upper and lower sides in the first display area and each of the second display areas on both the left and right sides in the first display area are used as a pair of second display areas in each set. An image display method characterized by the above.

4. In claim 1, when creating an image obtained by rotating the three-dimensional image information by a predetermined angle, the rotation direction is set such that, in the created image, an image portion farther from the selected second display area among the display images in the first display area becomes less visible due to the rotation. An image display method characterized by this.

5. In claim 1, on the premise that the posture of the display image in the first display area can be changed, when it is detected that the posture of the display image in the first display area has been changed while the display image is being displayed in the second display area, an image obtained by rotating the three-dimensional image information serving as the basis of the display image in the first display area where the posture has been changed by a predetermined angle around the rotation axis is created, and the display image in the second display area is changed to this image. An image display method characterized by this.

6. In an image display device provided with a display area in a display unit and displaying an image based on three-dimensional image information in the display area, a storage unit that stores display area setting information for setting the display area, a display area setting unit that reads the display area setting information from the storage unit and sets a first display area and a second display area arranged side by side with the first display area in the display unit, a first display processing unit that displays an image based on the three-dimensional image information in the first display area set by the display area setting unit on the condition that the three-dimensional image information is captured, and when it is detected that the second display area has been selected while the image based on the three-dimensional image information is being displayed in the first display area based on the first display processing unit, among the virtual axes passing through the center point of the three-dimensional image information serving as the basis of the image in the first display area, a direction that is orthogonal to the side-by-side direction of the selected second display area and the first display area and extends along the display unit is used as the rotation axis, an image obtained by rotating the three-dimensional image information by a predetermined angle is created, and a second display processing unit that displays this image in the selected second display area. An image display device characterized by this.

7. In claim 6, the first display area is set to be rectangular, and since there are a plurality of the second display areas, the plurality of second display areas are set to be arranged continuously on at least both the upper and lower sides and both the left and right sides in the first display area with the first display area as the center. When the second display processing unit displays the image on each of the second display areas on both the upper and lower sides in the first display area or each of the second display areas on both the left and right sides in the first display area, as the three-dimensional image information to be used for each of the second display areas, the three-dimensional image information that is the basis of the display image in the first display area is used after being rotated by a predetermined angle in opposite rotation directions around the rotation axis based on the state of the three-dimensional image information. An image display device characterized by this.

8. In claim 6, when the second display processing unit creates an image obtained by rotating the three-dimensional image information by a predetermined angle, the rotation direction is set such that, in the image to be created, the farther the image portion from the selected second display area in the display image of the first display area is from the rotation, the less it is displayed. An image display device characterized by this.

9. In claim 6, a third display processing unit is provided that rotates the three-dimensional image information that is the basis of the display image in the first display area and changes the posture of the display image in the first display area on the condition of input of rotation instruction information. The second display processing unit determines that a display image is displayed in the second display area based on information from the second display processing unit, and when it determines that the display image in the first display area has been posture-changed based on information from the third display processing unit, an image obtained by rotating the three-dimensional image information that is the basis of the display image in the posture-changed first display area by a predetermined angle around the rotation axis is created along with the posture change of the display image in the first display area, and the display image in the second display area is set to be changed to this image. An image display device characterized by this.

10. A program for an image display device that has a storage device for storing display area setting information for setting a display area in a display unit, and under the condition of its existence, reads the display area setting information from the storage device to set a display area in the display unit and displays an image based on three-dimensional image information in the display area, the program causing a computer to function as: a display area setting unit that sets, in the display unit, as the display area, a first display area and a second display area arranged side by side with the first display area; a first display processing unit that displays an image based on the three-dimensional image information in the first display area set by the display area setting unit on the condition that the three-dimensional image information is captured; and a second display processing unit that, when it is detected that an image based on the three-dimensional image information is displayed in the first display area based on the first display processing unit and the second display area is selected, creates an image obtained by rotating the three-dimensional image information by a predetermined angle using, as a rotation axis, an imaginary axis passing through the center point of the three-dimensional image information that is the basis of the image in the first display area and extending in a direction along the display unit and orthogonal to the side-by-side direction of the selected second display area and the first display area, and displays the image in the selected second display area. A program for an image display device, characterized by the above.

Images