Image processing method and image processing system
The 360-degree spherical image processing system addresses distortion and blind spots by converting camera images to a 3D model, removing central obstacles, and presenting a continuous video from the passage center, enhancing remote site monitoring accuracy and realism.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- FUJITA CO LTD
- Filing Date
- 2022-03-10
- Publication Date
- 2026-06-16
Smart Images
Figure 0007874423000001 
Figure 0007874423000002 
Figure 0007874423000003
Abstract
Description
Technical Field
[0001] The present invention relates to an image processing method for processing images captured by a plurality of cameras arranged at a construction site, and an image processing system using the method.
Background Art
[0002] Regarding construction sites such as tunnels and roads, due to the impact of the recent COVID-19 pandemic, there is an increasing need to remotely patrol construction sites for the purpose of ensuring construction quality and safety at the site, and to provide appropriate guidance to on-site supervisors as needed. Here, regarding remote monitoring inside a tunnel, cameras are provided at regular intervals at relatively low positions on the side so that the disaster situation, the situation of victims, and the traffic situation inside the tunnel main line can be grasped remotely. A method has been known that enables remote monitoring by a supervisor by displaying the images inside the tunnel captured by these cameras on a control panel in a disaster prevention center (see, for example, Patent Document 1).
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] Applying the above method to a construction site makes it possible to remotely monitor the site's condition. However, when a camera is installed on the side of a walkway, the area directly in front of the camera and the area behind it (the side opposite to where the camera is installed) are captured as the central subject, and the subject becomes distorted as you move away from this central area. Therefore, it is difficult to accurately grasp the situation at the site at a glance. Furthermore, if there is an obstacle in front of the camera, the area behind the obstacle will not be captured by the camera, creating a blind spot, making it impossible to adequately patrol the site remotely. Thus, there are various problems with presenting images taken from the side of a walkway as they are, and improvements are desired.
[0005] Therefore, the present invention aims to provide a technology for presenting images of passageways in a manner that is easy to view. [Means for solving the problem]
[0006] To solve the above problems, the present invention employs the following image processing method and image processing system. Note that the following statements in parentheses are merely examples, and the present invention is not limited thereto.
[0007] In other words, the image processing method and image processing system of the present invention acquire multiple images captured by multiple cameras installed on the side of a passageway, create information representing the three-dimensional shape of the subject including the passageway based on these multiple images (for example, a 3D model or point cloud), specify a viewpoint approximately at the center of the passageway in the three-dimensional shape, and create and present a 360-degree spherical image of the three-dimensional shape viewed from that viewpoint.
[0008] According to this image processing method and image processing system, a 360-degree spherical image is created based on an image of the passage taken from the side, showing the three-dimensional shape from a viewpoint approximately at the center of the passage. This allows images of the construction site along the passage to be presented in an easily viewable manner, enabling users (such as remote patrol personnel) to grasp the situation at the construction site much more easily and accurately than when the side-taken images are presented as they are.
[0009] Preferably, in the image processing method and image processing system described above, after creating information representing three-dimensional information, information indicating an object located approximately in the center of the passage is removed therefrom, a viewpoint is specified for the three-dimensional shape from which the object information has been removed, and a 360-degree spherical image is created from that viewpoint as seen through the three-dimensional shape from which the object information has been removed.
[0010] According to this embodiment of the image processing method and image processing system, a 360-degree spherical image is created by removing information indicating an object located approximately in the center of the passage from the information representing the three-dimensional shape. This prevents the problem that can occur when creating a 360-degree spherical image by simply stitching together images captured by multiple cameras without converting them to a three-dimensional shape, where the position of the object is not correctly reflected (resulting in an image where the object appears to be in a different location than it actually is). As a result, a 360-degree spherical image that accurately reflects the situation at the construction site can be presented.
[0011] More preferably, the above-described image processing method and image processing system are characterized by continuously specifying viewpoints that are shifted at approximately constant intervals (approximately constant distances) along the approximate center of a passage in a three-dimensional shape, continuously creating a 360-degree spherical image corresponding to the continuously specified viewpoints, and presenting the continuously created 360-degree spherical image.
[0012] According to this embodiment of the image processing method and image processing system, a continuous 360-degree spherical image, i.e., a 360-degree video, is created and presented while shifting the viewpoint in the three-dimensional shape along approximately the center of the passageway. As a result, even without the user performing any viewpoint-related operations on the 360-degree video viewer, a video can be provided in real time, as if the user were patrolling the site while walking along the center of the passageway. [Effects of the Invention]
[0013] According to the present invention, images of passageways can be presented in a manner that is easy to view. [Brief explanation of the drawing]
[0014] [Figure 1] This is a block diagram showing the configuration of image processing system 1. [Figure 2] This is a simplified plan view showing the camera placement at a tunnel construction site. [Figure 3] This diagram shows the shooting range (angle of view) of each camera. [Figure 4] This flowchart shows an example of the processing procedure performed in image processing system 1. [Figure 5] This diagram supplements the flow from obstacle removal processing to the presentation of a 360-degree image. [Figure 6] This figure shows an example of a way to continuously specify the viewpoint. [Figure 7] As a comparative example, this diagram illustrates the process of creating a 360-degree spherical image without creating a 3D model or removing obstacles. [Modes for carrying out the invention]
[0015] Embodiments of the present invention will be described below with reference to the drawings. The following embodiments are preferred examples of image processing systems, and embodiments of the present invention are not limited to these examples. Furthermore, for convenience of explanation, directions related to the illustrated information may be indicated as up, down, left, and right, corresponding to the direction on the page of each drawing.
[0016] [Configuration of the image processing system] Figure 1 is a block diagram showing the configuration of the image processing system 1. The image processing system 1 is a system that enables remote patrolling of construction sites of passage-like structures such as tunnels and roads (hereinafter collectively referred to as "passages") by creating and presenting a 360-degree spherical image viewed from approximately the center of the passage, based on images captured by multiple cameras placed on the side of the passage.
[0017] The image processing system 1 operates in an environment where cameras installed at the construction site, the server 10 on which programs necessary for the processing of the image processing system 1 are installed, and the user terminal 20 are connected to a communication line. For example, the server 10 includes a captured image acquisition unit 11, a 3D model creation unit 12, a viewpoint designation unit 13, an omnidirectional image creation unit 14, and an omnidirectional image presentation unit 15, and the user terminal includes a viewer 21.
[0018] At the construction site, a plurality of cameras are arranged at regular intervals on both sides of the passage, and the captured images by these cameras are transmitted to the server 10 via the communication line. The arrangement mode of the cameras at the construction site will be further described later using another drawing.
[0019] In the server 10, the captured image acquisition unit 11 acquires the captured images transmitted from the plurality of cameras. The 3D model creation unit 12 creates a 3D model that restores the state of the construction site based on the captured images by the plurality of cameras, and removes the objects existing in the center of the passage from the 3D model as necessary. The viewpoint designation unit 13 designates the position of the viewpoint in the created 3D model. The omnidirectional image creation unit 14 creates an omnidirectional image (360-degree panoramic image in all directions of up, down, left, and right) viewed from the viewpoint designated using the 3D model. The omnidirectional image presentation unit 15 presents the created omnidirectional image. By continuously executing the processing by these functional units 11, 12, 13, 14, 15 in real time for a plurality of points on the passage, a continuous omnidirectional image, that is, an omnidirectional video, at a plurality of points can be created and presented. The specific content of the processing executed in the server 10 will be described in detail later while referring to another drawing.
[0020] In the user terminal 20, the omnidirectional video presented by the server 10 (omnidirectional image presentation unit 15) is displayed on the viewer 21. Here, the "user" refers to a person in charge of remote patrol at the construction site, for example, an employee on the head office side of a construction company. The user can use the operation menu prepared on the viewer 21 to change the position of the viewpoint, the direction of the line of sight, the display magnification, etc. of the omnidirectional video. For example, when the user gradually shifts the position of the viewpoint along the approximate center of the passage using the operation menu, an omnidirectional video corresponding to the viewpoint changed by this operation is presented, and a state can be pseudo-realized as if looking around while walking along the approximate center of the passage.
[0021] Note that as the viewer 21, a program installed on the user terminal 20 may be used in combination with the display of the user terminal 20, or a VR device connected to the user terminal 20 may be used.
[0022] Both the server 10 and the user terminal 20 are general-purpose computers equipped with a CPU, RAM, HDD, various I / Fs, etc. When programs necessary for the computer are installed, the CPU of the computer operates as each functional unit 11, 12, 13, 14, 15, 21. Also, data, specified values, etc. handled by each functional unit 11, 12, 13, 14, 15, 21 included in the image processing system 1 are stored in a storage unit (not shown) as necessary.
[0023] 〔Camera Arrangement at the Construction Site〕 FIG. 2 is a plan view schematically showing the camera arrangement situation at the construction site of a certain tunnel as an example of the camera arrangement mode at the construction site of the passage.
[0024] At the tunnel site, multiple cameras are arranged in a staggered pattern at approximately equal intervals (for example, every 10m) on both sides of the tunnel (both sides of the passageway). For example, as shown in Figure 2, starting from the beginning of the tunnel and moving towards the back, camera CM1 is placed near the tunnel entrance on the beginning side, camera CM2 is placed diagonally in front of camera CM1 across the passageway, camera CM3 is placed diagonally in front of camera CM2 across the passageway, camera CM4 is placed diagonally in front of camera CM3 across the passageway, and so on. In this manner, odd-numbered cameras are placed at approximately equal intervals on one side SW1, while even-numbered cameras are placed at approximately equal intervals on the other side SW2.
[0025] By arranging multiple cameras in this manner at a tunnel construction site, it becomes possible to check the conditions inside the tunnel without any blind spots. The reason the cameras are placed on the sides of the tunnel is that they cannot be placed in the center of the passageway because it would interfere with construction, whereas they do not interfere with construction on the sides. However, as mentioned above, images taken from the sides have some issues that need improvement.
[0026] Figure 3 shows the shooting range (angle of view) of each camera positioned on both sides of the tunnel. Each camera has substantially the same specifications, and here, the shooting range of camera CM15 and camera CM16, positioned diagonally in front of it, are used as examples for explanation. In Figure 3, (B) and (C) are substitute photographs, and are examples of images taken by camera CM15 and camera CM16, respectively, but these images were taken at approximately the same time.
[0027] Figure 3(A): Shows the horizontal shooting range of cameras CM15 and CM16. Camera CM15 can capture an area from slightly to the left of camera CM12, which is located on the opposite side SW2, to slightly to the right of camera CM18. Camera CM16 can capture an area from slightly to the left of camera CM13, which is located on the opposite side SW1, to slightly to the right of camera CM19.
[0028] Figure 3(B): This is a photograph used as a substitute for a drawing, showing an example of image SP15 taken by camera CM15. Image SP15 does not show a portion of the foreground of the passageway and ceiling directly in front of camera CM15, but it generally shows the entire passageway and ceiling slightly off-center from camera CM15. In the example shown, construction materials and road cones placed on the passageway in front of camera CM15 are visible in the center of image SP15.
[0029] Figure 3(C): This is a photograph used as a substitute for a drawing, showing an example of image SP16 taken by camera CM16. Image SP16 does not show a portion of the foreground of the passageway and ceiling directly in front of camera CM16, but it generally shows the entire passageway and ceiling slightly off-center from camera CM16. In the illustrated example, construction materials and road cones placed on the passageway to the right of camera CM16 are visible at the right edge of image SP16. These are the same construction materials and road cones shown in the center of image SP15 in Figure 3(B), but from a different viewpoint.
[0030] Thus, each camera can capture at least an area in the horizontal direction (left-right) that includes the positions of the four cameras positioned on the opposite side, and in the vertical direction (up-down), it can capture the area of the passageway and ceiling directly in front of the camera, excluding a portion of the front, and approximately the entire passageway and ceiling at a position slightly off-center from the front. Furthermore, any areas of the passageway and ceiling that cannot be captured by each camera directly in front can be captured by surrounding cameras.
[0031] [Processing flow in an image processing system] Figure 4 is a flowchart showing an example of the processing steps performed in the image processing system 1. The following explanation will follow this example.
[0032] Step S1: The captured image acquisition process is executed. In this process, the captured image acquisition unit 11 acquires the captured images transmitted from the camera at the construction site and stores them in the storage unit. The captured images from the camera may be transmitted in video format or in still image format periodically (for example, every second).
[0033] Step S2: The 3D model creation process is executed. In this process, the 3D model creation unit 12 creates a 3D model that reconstructs the construction site from multiple captured images acquired in step S1. Specifically, the 3D model creation unit 12 uses SfM (structure from motion) technology to generate a point cloud from multiple captured images taken from different viewpoints by multiple cameras, and then creates a 3D model by meshing (converting to surface data) the point cloud. Note that a program developed in-house may be used to create the 3D model, or existing SfM software may be used.
[0034] Step S3: Obstacle removal processing is performed. In this process, the 3D model creation unit 12 removes information from the 3D model created in step S2 that corresponds to objects (hereinafter referred to as "obstacles") that are located in or near the center of the passageway and that may obstruct camera photography. Obstacles can also be described as objects that may obstruct the view of the construction site along the approximate center of the passageway.
[0035] Furthermore, even if an object exists in the center of a passageway, if it is not very tall, it will not pose an obstacle and will not particularly affect subsequent image processing. Therefore, such an object does not qualify as an obstacle and does not necessarily need to be removed. Consequently, if no obstacles are present, the 3D model creation unit 12 can omit step S3.
[0036] Step S4: The viewpoint specification process is executed. In this process, the viewpoint specification unit 13 specifies the position of the viewpoint in the 3D model created in step S3 (or step S2 if step S3 is omitted). The viewpoint specification unit 13 specifies one of the positions on a virtual centerline passing through the approximate center of the passage (hereinafter abbreviated as "passage centerline") as the viewpoint.
[0037] Step S5: The 360-degree image creation process is executed. In this process, the 360-degree image creation unit 14 uses the 3D model created in step S3 (or step S2 if step S3 is omitted) to create a 360-degree image that simulates a view of the construction site from the viewpoint specified in step S4, based on images captured by cameras positioned around that viewpoint.
[0038] Step S6: The 360-degree image presentation process is executed. In this process, the 360-degree image presentation unit 15 presents the 360-degree image created in step S5 and stores it in the storage unit. Once the above steps are completed, return to step S1 and repeat steps S1 to S6. This makes it possible to present a continuous 360-degree image, or 360-degree video, in real time, as if viewing the construction site from a point on the centerline of a walkway.
[0039] Furthermore, while the above example procedure focuses on a single point for the sake of explanation, in reality, the series of processes described above are executed simultaneously at multiple points along the centerline of the walkway. By repeatedly and continuously executing this series of processes at multiple points, it is possible to present a 360-degree video of the construction site as seen from multiple (arbitrary) points along the centerline of the walkway.
[0040] In the example procedure described above, point cloud generation and meshing are performed in step S2. Alternatively, only point cloud generation may be performed in step S2, and then meshing may be performed in step S3 after removing obstacles and noise.
[0041] Figure 5 is a diagram that supplements the flow from obstacle removal processing to 360-degree image presentation processing (steps S3 to S6 in Figure 4), and all of the diagrams are simplified representations of the passageway as seen from above at the construction site.
[0042] Figure 5(A): This shows a state where an obstacle OB is present in the center of the passageway in front of camera CM14. When an obstacle OB is in front of camera CM14, the area behind it, i.e., the area AR between cameras CM13 and CM15 on the side SW1 opposite camera CM14, is blocked by the obstacle and cannot be captured. However, even if camera CM14 cannot capture area AR, the adjacent cameras CM12 and CM16 can capture area AR from an oblique angle. Therefore, in the 3D model created from the images captured by these cameras, the feature points corresponding to the obstacle OB become surfaces, resulting in the reconstruction of the area as if the obstacle OB is protruding from the center of the passageway.
[0043] Figure 5(B): This shows the state after removing obstacles OB located in the center of the passage in front of camera CM14. By removing the information corresponding to obstacles OB located in the center of the passage in the obstacle removal process (step S3 in Figure 4), it is possible to create a state in the 3D model where there are no obstacles OB in the center of the passage.
[0044] Figure 5 (C): This shows the view in all directions from viewpoint VP14, which is located approximately in the center of the passage in front of camera CM14. By specifying viewpoint VP14 in the viewpoint specification process (step S4 in Figure 4), creating a series of 360-degree images using a 3D model in the 360-degree image creation process (step S5 in Figure 4) to simulate the view of the construction site from viewpoint VP14, and then continuously presenting the 360-degree images in the 360-degree image presentation process (step S6 in Figure 4), it is possible to present a 360-degree video in real time, as if viewing the actual construction site from viewpoint VP14.
[0045] Figure 6 shows an example of how to change the viewpoint position in viewer 21. By repeatedly performing the above-described series of processes for multiple locations, a 360-degree video is created within the range of the passage shown in Figure 6, for example, showing the construction site from multiple viewpoints VP1 to VP24 designated at approximately equal intervals along the centerline of the passage. The viewer 21 also provides various operation menus for displaying the 360-degree video. For example, the user can use the viewpoint change menu to change the position of the viewpoint in the 360-degree video displayed in the image area of the viewer 21, and can use the gaze change menu to change the direction of the gaze on the displayed 360-degree video.
[0046] For example, consider a scenario where a user views a 360-degree video from various viewpoints, starting from the beginning of a tunnel and moving towards the back. The user first selects viewpoint VP1 from the map displayed in the viewpoint change menu. This displays a 360-degree video in the viewer 21's image area, showing the construction site from viewpoint VP1. The user can change the direction of their gaze in all directions using the gaze change menu, and checks the surroundings of viewpoint VP1 while appropriately changing the direction of their gaze. After finishing their check of viewpoint VP1, the user accesses the viewpoint change menu again and selects viewpoint VP2, displaying a 360-degree video in the image area showing the construction site from viewpoint VP2, and checks the surroundings of viewpoint VP2 using the gaze change menu. The user can then repeat this process, selecting adjacent viewpoints in sequence and checking their surroundings, allowing them to remotely patrol the construction site as if they were walking along the centerline of a walkway and observing the surroundings.
[0047] In the illustrated example, viewpoints VP1 to VP24 are specified at approximately equal intervals along the centerline of the passageway, corresponding to the positions of cameras CM1 to CM24. However, this is merely an example, and viewpoints may be specified at shorter or longer intervals. In any case, a 360-degree video of the construction site from each viewpoint is displayed in real time by the server 10 (360-degree image display unit 15), and these can be displayed in real time on the viewer 21 while changing viewpoints.
[0048] [Comparative Example] Figure 7 illustrates the process of creating a 360-degree spherical image by stitching together images captured by multiple cameras, without creating a 3D model or removing obstacles, as a comparative example to the embodiment described above. Both figures show a simplified representation of a walkway viewed from above at a construction site.
[0049] In Figure 7 (A): When creating a 360-degree spherical image of a construction site viewed from a viewpoint approximately centered on the walkway, images taken from each direction from that viewpoint are stitched together and combined. For example, when creating a 360-degree spherical image of a construction site viewed from viewpoint VP14, the image captured by camera CM16 is assigned to region AR1 between cameras CM11 and CM13, the image captured by camera CM15 is assigned to region AR2 between cameras CM12 and CM14, the image captured by camera CM14 is assigned to region AR3 between cameras CM13 and CM15, the image captured by camera CM13 is assigned to region AR4 between cameras CM14 and CM16, and the image captured by camera CM12 is assigned to region AR5 between cameras CM15 and CM17. These images are then stitched together and combined. Note that regions AR1 to AR5 in the figure are shown for explanatory purposes only and do not represent the exact regions for which images are assigned.
[0050] In Figure 7 (B): In this case, if an obstacle OB exists in the center of the passageway, for example, directly in front of camera CM14, the obstacle OB will be visible in the center of the image captured by camera CM14, while a portion of region AR3 will be obscured by the obstacle OB and will not be captured. As described above, the image captured by camera CM14 is assigned to region AR3 between cameras CM13 and CM15, so the 360-degree image created by stitching together the captured images will show the obstacle OB as being located in region AR3. However, the obstacle OB is actually located in front of camera CM14, closer to camera CM14, and not in region AR3.
[0051] Thus, simply stitching together images captured by multiple cameras positioned around the viewpoint to create a 360-degree image does not pose any particular problems if there are no obstacles or out-of-bounds (OB) in the center of a passageway, etc. However, if there are obstacles or OB in the center of a passageway, etc., the resulting 360-degree image (video) will not accurately reflect the construction site and will display incorrect information, which hinders remote patrolling of the construction site.
[0052] In contrast, according to the embodiment described above, the construction site is reconstructed into a 3D model from images captured by multiple cameras, and information corresponding to obstacles (OB) present in the center of the passageway is removed. Then, a 360-degree spherical image (video) is created using the 3D model. Therefore, even if obstacles (OB) exist in the center of the actual passageway, a 360-degree spherical image (video) that accurately reflects the construction site can be presented without being affected by these obstacles, contributing to highly accurate remote patrols.
[0053] According to the embodiment described above, the following advantages can be obtained. (1) At the construction site of the passageway, a 360-degree spherical image (video) is created from images taken by multiple cameras placed at regular intervals on both sides of the passageway, as if viewed from a viewpoint along approximately the center of the passageway. This allows images of the construction site of the passageway to be presented in an easily viewable manner. As a result, users can grasp the situation of the construction site much more easily and accurately compared to when images taken from the side of the passageway are presented as is.
[0054] (2) A 360-degree spherical image (video) created from images taken by multiple cameras placed at the construction site of the passageway is displayed in the viewer, giving the user the feeling that they are standing in the center of the passageway and looking around, thus providing a sense of presence to remote patrols of the construction site.
[0055] (3) Images captured by multiple cameras placed at the construction site of the passageway are converted into a 3D model. If there is an obstacle in the center of the passageway, information corresponding to this obstacle is removed from the 3D model, and a 360-degree spherical image (video) is created that shows the 3D model from a viewpoint approximately aligned with the center of the passageway. As a result, a 360-degree spherical image (video) that accurately reflects the state of the construction site can be presented without being affected by obstacles actually present in the passageway.
[0056] (4) Images of the construction site along the passageway are provided in a data format compatible with VR devices, etc. (360-degree images, 360-degree videos), so by utilizing VR devices, etc., remote patrols of the construction site can be carried out more efficiently with intuitive operation.
[0057] The present invention can be implemented in various ways without being limited to the embodiments described above.
[0058] In the embodiment described above, when the user changes the viewpoint position using the operation menu (viewpoint change menu) of the viewer 21, a 360-degree video showing the construction site from this viewpoint is displayed in the image area of the viewer 21. In addition to this function, the system may also be configured to automatically display a 360-degree video that moves the viewpoint position along the centerline of the passage at a substantially constant speed, even without the user changing the viewpoint position. Furthermore, the system may be configured to allow the user to select or arbitrarily set the speed at which the viewpoint position moves.
[0059] In the embodiments described above, a 360-degree image is created using a 3D model created from point cloud data. Alternatively, a 360-degree image may be created using data in which only the necessary parts are extracted from the point cloud data, obstacles are removed, and a mesh is applied only to the parts necessary for creating a 360-degree image.
[0060] In the embodiment described above, only the functions necessary for image processing are provided, but functions related to remote patrolling of construction sites may be further provided. For example, a function may be provided that allows the user to conduct a remote patrol while viewing a 360-degree video and provide feedback on what they notice to the site supervisor of the construction site, or a speaker built into the camera may be used to output the voice of the user conducting the remote patrol to the construction site, describing the area being captured by the camera.
[0061] In the embodiment described above, images are transmitted to the server 10 from multiple cameras placed at the construction site. Alternatively, a relay device placed at the construction site may acquire images from multiple cameras and then transmit the images to the server 10 from the relay device.
[0062] In the embodiment described above, all components of the image processing system 1 except the viewer 21 are provided on the server 10, but these components may be distributed across multiple servers.
[0063] Furthermore, all other examples shown with illustrations in the embodiments are merely preferred examples, and it goes without saying that modifications can be made as appropriate when implementing the present invention. [Explanation of Symbols]
[0064] 1. Image Processing System 11 Image acquisition unit 12. 3D Model Creation Department (3D Information Creation Department) 13. Point of View Designation Section 14. Spherical Image Creation Section 15. Spherical Image Display Unit 21 Viewer CM Camera VP perspective
Claims
1. The process involves acquiring multiple images captured almost simultaneously by multiple cameras installed on the side of the passageway, and A three-dimensional information creation step, which creates information representing the three-dimensional shape of a subject including the passage based on the aforementioned multiple images, A viewpoint designation step, which involves designating a viewpoint on a virtual centerline passing through the approximate center of the passage in the three-dimensional shape, A spherical image creation step is performed to create a spherical image of the three-dimensional shape as seen from the viewpoint by performing an image creation process based on the viewpoint on the center line using the aforementioned information. A spherical image presentation step that presents the aforementioned spherical image, Image processing methods including [specific details omitted].
2. In the image processing method described in claim 1, The process further includes an object removal step, which removes information from the information representing the three-dimensional shape that indicates an object located approximately in the center of the passage that may obstruct visibility or be displayed in an incorrect position when viewed from a viewpoint on the center line, In the aforementioned viewpoint designation process, Specify the viewpoint in the three-dimensional shape after the object removal step, In the aforementioned process of creating a 360-degree image, An image processing method characterized by creating a 360-degree spherical image of the three-dimensional shape obtained through the object removal step, viewed from the aforementioned viewpoint.
3. In the image processing method according to claim 1 or 2, In the aforementioned viewpoint designation process, The aforementioned viewpoints are continuously specified by shifting them at approximately constant intervals along the approximate center of the passage in the three-dimensional shape, In the aforementioned process of creating a 360-degree image, The 360-degree images corresponding to the continuously specified viewpoints are continuously created, In the aforementioned 360-degree image presentation process, An image processing method characterized by presenting the continuously generated 360-degree images.
4. An image acquisition unit that acquires multiple images captured almost simultaneously by multiple cameras installed on the side of the passageway, A 3D information creation unit creates information representing the three-dimensional shape of a subject including the passage, based on the aforementioned multiple images. A viewpoint designation unit that designates a viewpoint on a virtual centerline passing through the approximate center of the passage in the three-dimensional shape, A spherical image creation unit creates a spherical image of the three-dimensional shape as seen from the viewpoint by performing an image creation process based on the viewpoint on the center line using the aforementioned information. A 360-degree image display unit that displays the aforementioned 360-degree image, An image processing system including [specific components / features].