Display control device, display control method, and program

Abstract

To provide a technique for appropriately displaying an image captured by an image capturing apparatus which is installed in a movable body.SOLUTION: A display control apparatus which controls a display of a plurality of captured images captured by one or a plurality of image capturing apparatuses installed inside of a movable body includes determination means configured to determine whether movement of the movable body is stopped. Display control means is configured to control the display of the plurality of captured images on display means so that a first captured image having an angle of view for capturing a side of the movable body from the inside of the movable body and a second captured image different from the first captured image are selectively displayed among the plurality of captured images on the basis of at least a result of the determination by the determination means.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a display control technique. [Background technology]

[0002] Imaging devices are used for the safety of buses and trains. Patent Document 1 describes a method for a driver to monitor the state of a platform while remaining in the driver's seat. Specifically, an imaging device installed on the platform captures an image of the area near the boundary between the platform and the train. Then, when the train is stopped at the platform, the image captured by the imaging device is transmitted to the train via wireless communication. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2002-104189 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the conventional techniques disclosed in the above-mentioned patent documents assume that the imaging device is installed outside the vehicle, and therefore it is not possible to properly display images captured by the imaging device installed inside the vehicle.

[0005] SUMMARY OF THE PRESENT DISCLOSURE An object of the present invention is to provide a technique capable of appropriately displaying an image captured by an imaging device installed inside a moving body. [Means for solving the problem]

[0006] In order to solve the above problems, a display control device of the present invention has the following configuration: That is, the display control device controls display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, and includes a determination means for determining whether movement of the moving body has stopped, and a display control means for controlling display of the plurality of captured images on a display means so as to selectively display, based on at least a determination result by the determination means, a first captured image having an angle of view captured from inside the moving body in a lateral direction of the moving body and a second captured image different from the first captured image, among the plurality of captured images.

[0007] Also, a display control device of the present invention has the following configuration: That is, the display control device controls display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, and includes a determination means for determining whether movement of the moving body has stopped, and a display control means for controlling display of the plurality of captured images on a display means based on at least a determination result by the determination means, such that one of a first captured image having an angle of view captured from inside the moving body toward a side of the moving body and a second captured image different from the first captured image is displayed larger than the other. Effect of the Invention

[0008] According to the present invention, it is possible to appropriately display an image captured by an imaging device installed inside a moving object. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a system configuration according to each embodiment. [Diagram 2] FIG. 2 is a functional block diagram according to each embodiment. [Diagram 3] FIG. 2 is a diagram illustrating a bus according to each embodiment. [Figure 4] 1A and 1B are diagrams illustrating a relationship between a fisheye image and a panoramic display according to each embodiment. [Diagram 5]2 is a flowchart according to the first embodiment. [Figure 6] 10 is a flowchart showing a subroutine of a vehicle stop position confirmation process according to each embodiment. [Figure 7] 10A to 10C are diagrams showing examples of division when a fisheye image is panoramic-developed according to each embodiment. [Figure 8] 10 is a flowchart showing a subroutine of an image priority confirmation process according to each embodiment. [Figure 9] 11A and 11B are diagrams showing examples of how a dewarped image is displayed on a client device according to each embodiment. [Figure 10] 11A to 11C are diagrams showing a display example of a double panorama according to each embodiment. [Figure 11] 10 is a flowchart according to a second embodiment. [Figure 12] 1A and 1B are diagrams illustrating examples of fisheye images according to each embodiment. [Figure 13] 11A to 11C are diagrams illustrating display examples of captured images according to the embodiments. [Figure 14] FIG. 2 is a block diagram showing an example of the hardware configuration of an imaging apparatus and a client apparatus in each embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations described in the following embodiments. In the following description, a bus will be used as an example of a moving body on which an imaging device is installed, but other moving bodies such as trains and ordinary passenger cars may also be used. In any case, it is sufficient that the imaging device is installed inside the moving body and is capable of capturing at least an image of the inside of the moving body.

[0011] In each embodiment, the image captured by the imaging device may be a moving image or a still image.

[0012] In addition, in each embodiment, a surveillance camera is described as an example, but the present invention can also be applied to cameras other than those for surveillance. For example, the present invention can also be applied to an imaging device and a display control device that captures images for broadcasting purposes, movies, and personal purposes. For example, the present invention can also be applied to an imaging device that captures images for broadcasting purposes, movies, and personal purposes, rather than for surveillance purposes.

[0013] In the following embodiments, an imaging device capable of capturing an image called an omnidirectional image using an omnidirectional mirror or a fisheye lens will be described as an example. Such an imaging device is capable of capturing a wide range of surroundings, and can capture an annular or circular image (also called a fisheye image) of approximately 180 degrees with one imaging means. However, the omnidirectional image is only an example, and the present invention can be applied to a normal surveillance camera that does not use a fisheye lens, as described below. In addition, the term omnidirectional is not necessarily limited to capturing an image of the entire space in which the imaging device is installed. The terms "omnidirectional image", "omnidirectional camera", and "fisheye image" are commonly used terms, and therefore these terms will be used appropriately in each embodiment. In each embodiment, the "omnidirectional camera" may be an imaging device that captures images using an omnidirectional mirror or a fisheye lens. The "omnidirectional image (fisheye image)" may be an image captured by the imaging device.

[0014] First Embodiment First, an overview of this embodiment will be described with reference to Fig. 12. In Fig. 12, 1200 indicates an example of a fisheye image of the inside of a bus captured by an imaging device using a fisheye lens. Also, 1204 indicates the traveling direction (forward direction) of the bus. When monitoring the inside of a bus using a panoramic image obtained by dewarping a fisheye image, the priority of the area to be monitored differs depending on the operating status of the bus, so it is difficult for the user to manually adjust the position of the panoramic image to be generated each time depending on the operating status of the bus.

[0015] For example, when the bus stops at a bus stop, in order to monitor passengers getting on and off, it is preferable to display a captured image (first image) with an angle of view of the side of the bus from inside the bus, including the bus entrance (door) as in the area indicated by 1201 in FIG. 12. Conversely, when the bus is moving, in order to monitor passenger seats, it is preferable to display a captured image (second image) with an angle of view of the rear or front of the bus from inside the bus. In particular, it is preferable to display a captured image with an angle of view of the rear of the bus from at least the center of the bus or a position closer to the front than the center. For example, in FIG. 12, it is an image with the rear of the bus as the area indicated by 1203. Also, in order to monitor passenger seats when the bus is moving, it may be preferable to display a captured image with the front of the bus from the center of the bus or a position closer to the rear than the center. For example, in FIG. 12, it is preferable to monitor an image with the front of the bus as the area indicated by 1202.

[0016] These captured images may be captured by an imaging device using a fisheye lens, or may be captured by a normal imaging device that does not use a fisheye lens. However, when a fisheye lens is not used, it is often better to use multiple imaging devices. In the following description of the first embodiment, two examples will be described: an example of an imaging device that uses a fisheye lens, and an example of an imaging device that does not use a fisheye lens.

[0017] The direction of travel of the bus (front), rear, and side directions are as shown in Figure 3. Figure 3(a) is a top view of the inside of the bus, and Figure 3(b) is a side view of the bus.

[0018] First, the system configuration according to this embodiment will be described with reference to FIG.

[0019] In FIG. 1(a), imaging device (network camera) 100 is an imaging device that uses a fisheye lens to capture images. This allows it to capture a substantially circular fisheye image of 180 degrees. In this embodiment, imaging device 100 is installed on the ceiling of a bus or train with its imaging direction facing downward or diagonally downward. Note that, although FIG. 1(a) illustrates an example in which one omnidirectional camera is used, multiple omnidirectional cameras may also be used. When one omnidirectional camera is used, it is recommended that it be installed near the center of the vehicle.

[0020] In an imaging device using a fisheye lens, the captured image is circular. However, in order to make it easier for the user to view, a part of the fisheye image may be cut out, and the partial image may be subjected to distortion correction and expanded to generate a panoramic image. In this case, the fisheye image is divided at any line segment in the circumferential direction of the fisheye image. Then, there is a function to perform distortion correction processing (dewarping) so that the circumferential direction of the fisheye image is aligned with the vertical direction of the corrected image, thereby generating an image in which the subject is upright (hereinafter referred to as a panoramic image as appropriate). There is also a double panorama function that displays this panoramic image in two areas.

[0021] Next, FIG. 1(b) shows an example of a system configuration using an imaging device (network camera) 100 that does not use a fisheye lens. In this example, two imaging devices 100 are used, but three or more imaging devices 100 may be used. When two imaging devices 100 are used, one of them captures an image with an angle of view of the side of the bus from inside the bus, including the bus entrance (door) such as the area shown by 1201 in FIG. 12. The other captures an image with an angle of view of the rear from at least the center of the bus or a position closer to the front than the center. In addition, a further imaging device 100 may be provided that captures an image with an angle of view of the front from the center of the bus or a position closer to the rear than the center.

[0022] In FIGS. 1(a) and 1(b), an imaging device 100 and a client device (display control device) 200 are connected via a network 300 so as to be able to communicate with each other.

[0023] The client device 200 transmits commands to the imaging device 100 to specify the image quality of an image captured by the imaging device 100, the position of an area to cut out the image, and the like. The imaging device 100 executes operations corresponding to those commands and transmits responses to those commands to the client device 200. These communications can be performed in compliance with the ONVIF (Open Network Video Interface Forum) standard, for example, but the communication method is not limited to this, and various communication methods can be used. A communication method using a wired cable or a wireless communication method may be used.

[0024] In each embodiment, an example in which the client device 200 executes the functions of a display control device will be described, but the imaging device 100 may execute some or all of the functions of the display control device. In other words, the imaging device 100 and the client device 200 may cooperate to execute the functions of a display control device.

[0025] 2 is a functional block diagram and a system configuration diagram of the imaging device 100, the client device 200, and the map information database 400 according to the first embodiment of the present invention. The configuration and function of each part of the imaging device 100 will be described with reference to FIG.

[0026] The imaging device 100 includes an imaging unit 101, an image processing unit 102, a system control unit 103, a lens driving unit 104, a lens control unit 105, an audio input unit 106, and a communication unit .

[0027] The imaging unit 101 receives light that is focused through a lens at an imaging element, converts the received light into electric charges, and generates an imaging signal. For example, a CMOS image sensor (Complementary Metal Oxide Semiconductor Image Sensor) can be used as the imaging element. Alternatively, a CCD image sensor (Charge Coupled Device Image Sensor) may be used as the imaging element.

[0028] The image processing unit 102 digitizes the imaging signal converted by the imaging unit 101 to generate image data. At this time, the image processing unit 102 also performs various image processing for correcting image quality. The image processing unit 102 may further compress and code the image data to generate compressed and coded image data.

[0029] The communication unit 108 transmits image data generated by the image processing unit 102 to the client device 200. Note that the image data here is, for example, image data of a moving image. The communication unit 108 also receives commands transmitted from the client device 200 and transmits them to the system control unit 103. The communication unit 108 also transmits responses to the commands to the client device 200 under the control of the system control unit 103. In this way, the system control unit 103 also functions as a communication control means.

[0030] Furthermore, the system control unit 103 of the imaging device 100 analyzes a command received by the communication unit 108 and performs processing according to the command.

[0031] For example, in response to the command, the system control unit 103 causes the image processing unit 102 to adjust the image quality, and instructs the image processing unit 102 to adjust the image quality and the lens control unit 105 to control the zoom and focus.

[0032] The lens control unit 105 controls the lens driving unit 104 based on the transmitted instruction.

[0033] The lens driving unit 104 is composed of a driving system for the focus lens and zoom lens and a motor as a driving source thereof, and its operation is controlled by the lens control unit 105 .

[0034] The audio input unit 106 collects audio data from an audio input device such as a microphone. These devices are, for example, a MEMS microphone, a condenser microphone, and an audio codec device. These devices include those built into the camera as well as external devices.

[0035] Next, the configuration and functions of each part of the client device 200 will be described with reference to FIG.

[0036] The client device 200 can be realized by using a computer such as a personal computer. Alternatively, the client device 200 may be realized by installing specific software in an in-vehicle device such as a navigation device. Alternatively, the client device 200 may be realized by installing specific software in a smartphone or tablet terminal.

[0037] The display unit 201 displays an image based on the image data received from the imaging device 100. The display unit 201 also displays a graphic user interface (hereinafter referred to as GUI) for controlling the camera. These displays are performed under the control of the system control unit 203. In other words, the system control unit 203 also has a function as a display control means. The display unit 201 can be realized by a display device using a liquid crystal panel, an organic EL panel, or the like. As an example, the display unit 201 is installed at a position where it can be seen by the bus driver. Note that when the imaging device 100 and the client device 200 are connected by a wireless communication method, the client device 200 may be installed in a monitoring center outside the vehicle.

[0038] The input unit 202 can be realized by devices such as a keyboard and a mouse, and the user of the client device 200 performs operations on the GUI via the input unit 202. The input unit 202 may be realized by using a touch panel.

[0039] The system control unit 203 of the client device 200 generates a command in response to a user operation and transmits the command from the communication unit 204 to the imaging device 100. The system control unit 203 also causes the display unit 201 to display image data received from the imaging device 100 via the communication unit 204. In this manner, the system control unit 203 also functions as a communication control unit and a display control unit.

[0040] The sensor input unit 207 acquires input from sensor devices such as an acceleration sensor, an encoder for a sensor that detects a position, and a GPS. A position information acquisition means such as a GPS may be provided inside the client device 200. In addition, the position information of the bus may be acquired from a navigation device or the like provided in the bus. The sensor input unit 207 may also acquire information on the operation of the brakes of the bus and control information such as the speed of the bus from various electronic devices that control the bus.

[0041] As described above, the client device 200 can obtain image data from the imaging device 100 via the network 300 and display the image data. In addition, the client device 200 can control the imaging device 100 by transmitting commands via the network 300.

[0042] The map information database 400 is accessed by the imaging device 100 and the client device 200 via the network 300, and provides map information to the imaging device 100 and the client device 200. The map information database 400 may be provided inside the client device 200.

[0043] Here, Fig. 4 is a diagram showing the relationship between a fisheye image and a panoramic display in the example of Fig. 1(a). In Fig. 2, 1200 indicates a fisheye image, and 404 indicates a dividing line on the fisheye image when it is expanded into a panoramic image. The area indicated by 402 and the area indicated by 403, which are divided by the fisheye dividing line 404, become the panoramic expansion area. 405 indicates a display example of a double panorama in which a fisheye image is expanded in two parts. 406 indicates a panoramic image generated by dewarping (distortion correction) the area indicated by 403. Also, 407 indicates a panoramic image generated by dewarping the area indicated by 402.

[0044] Next, a flow of an image acquisition method according to an embodiment of the present invention will be described with reference to Fig. 5. Note that in the following description, a part or all of the operations of the system control unit 203 of the client device can also be executed by the system control unit 103 of the imaging device 100.

[0045] 5(a) and 5(b) are the same except for steps (hereinafter abbreviated as S) 507, S510, and S511. Fig. 5(a) is an example of the case where an imaging device 100 using a fisheye lens is used. Fig. 5(b) is an example of the case where multiple imaging devices 100 without fisheye lenses are used.

[0046] First, the example of Fig. 5(a) will be described. In step 501, the system control unit 203 detects the traveling direction of the bus on which the imaging device 100 is installed, from information obtained from the sensor input unit 207. Note that since the bus basically moves forward, S501 may be omitted.

[0047] In S502, the system control unit 203 determines whether the bus on which the imaging device 100 is installed is stopped (stopped moving) based on information obtained from the sensor input unit 207. If the speed of the bus is equal to or lower than a predetermined speed (for example, 5 km per hour), it may be determined that the bus is stopped. If the bus is stopped, the process proceeds to S503. If the bus is not stopped, the process proceeds to S506. In this way, the system control unit 203 also functions as a determination unit that determines whether the bus is stopped.

[0048] In S503, the system control unit 203 performs a bus stop position confirmation process to confirm where the bus is stopped.

[0049] Here, the subroutine for checking the stopping position will be described with reference to the flowchart of FIG.

[0050] In S601, the system control unit 203 uses the location information indicating the current location of the bus obtained from the sensor input unit 207 and the map information obtained from the map information database 400 to identify the location of the bus on the map.

[0051] Next, in S602, the system control unit 203 determines whether the bus stop position obtained in S601 is near a bus stop (predetermined position). For example, if the current bus position is within a predetermined range (e.g., within 10 m) from a bus stop identified from map information, the system control unit 203 determines that the bus is stopped near a bus stop. If it is determined that the bus stop position is near a bus stop, the system control unit 203 proceeds to S603, where it determines that the current bus stop position is a bus stop. If it is determined that the bus stop position is not near a bus stop, the system control unit 203 proceeds to S604, where it determines that the current bus stop position is not a bus stop.

[0052] In S602, the system control unit 203 may specify the bus stop position from the voice obtained from the voice input unit 106. For example, a method may be used in which a trained model that has trained on sounds generated when a bus stops at a bus stop, such as the sound of doors opening and closing and announcement sounds, is compared with the current voice signal to determine whether the current bus position is at a bus stop.

[0053] Also, in S602, the system control unit 203 may identify the bus stop position by performing image recognition processing and using a method of matching with an image that is visible when the bus is stopped. For example, a method may be used in which a trained model that has learned an image that occurs when a bus stops at a bus stop, such as an image of a bus stop, is matched with a captured image of the outside of the current bus to determine whether the current bus position is at a bus stop.

[0054] The subroutine for checking the bus stop position may be omitted. In other words, the display control described below may be performed depending on whether the bus is stopped or not, regardless of the bus stop position.

[0055] Returning to the explanation of FIG. 5 (S504), according to the result of the judgment explained in FIG. 6, if the bus stop location is a bus stop, the process proceeds to S505, and if the bus stop location is not a bus stop, the process proceeds to S506.

[0056] In S505, the system control unit 203 transmits a command to the imaging device 100 via the communication unit 204 to cause the imaging device 100 to generate a panoramic image by dividing the fisheye image by a line segment 701 in the direction along the traveling direction of the bus. As shown in Fig. 7, the image divided by the line segment 701 is divided into an image with an angle of view capturing the side direction (right direction) of the bus from inside the bus, and an image with an angle of view capturing the opposite side direction (left direction). Hereinafter, these images will be referred to as a side direction image (right) and a side direction image (left) as appropriate.

[0057] On the other hand, in S506, the system control unit 203 transmits a command to the imaging device 100 via the communication unit 204 to cause the imaging device 100 to generate a panoramic image by dividing the fisheye image by a line segment perpendicular to the traveling direction of the bus. As shown in Fig. 7, the image divided by the line segment 702 is divided into an image with an angle of view in which the rear of the bus is captured from inside the bus, and an image with an angle of view in which the front is captured. Hereinafter, these images will be referred to as a rear image and a front image as appropriate.

[0058] Next, in S507, the system control unit 203 of the imaging device 100 dewarps (corrects distortion) the divided fisheye images to create a panoramic image.

[0059] In the above example, the panoramic image is generated exclusively by the method described in S505 and the method described in S506. However, both the panoramic image by the method described in S505 and the panoramic image by the method described in S506 may be generated. In this case, the client device 200 selects the image to be preferentially displayed.

[0060] In S509, the system control unit 203 acquires the panoramic image generated in S507 via the communication unit 204. If it is determined that the current location is a bus stop, only the image including the bus entrance / exit from among the multiple panoramic images may be acquired, because the image on the side with the entrance / exit is more important than the other side images.

[0061] Also, if it is determined that the current location is not a bus stop, only an image with an angle of view capturing the rear of the bus from inside the bus may be generated, because an image with an angle of view capturing the rear is more important than an image of the front.

[0062] Furthermore, in S506, if a panoramic image is generated by dividing the fisheye image by a line segment perpendicular to the traveling direction of the bus, the following may be performed. That is, either the rear image or the front image may be preferentially selected and acquired from the imaging device 100. The method of determining this priority will be described with reference to FIG. 8.

[0063] In S801, the system control unit 203 determines whether the bus is backing up. If the bus is backing up, the process proceeds to S802. If the bus is not backing up, the process proceeds to S803.

[0064] In S802, the system control unit 203 determines that, among the dewarped panoramic images, the rear image has a higher priority.

[0065] In S803, the system control unit 203 determines that, among the dewarped panoramic images, the front image has a higher priority.

[0066] In the image priority information confirmation process of FIG. 8, a method of performing image recognition on a panoramic image and determining the priority may be used. For example, a panoramic image including an image of a door may be determined to have a high priority. Also, a panoramic image may be determined to have a high priority if a specific object such as a person is detected in the panoramic image. When determining whether or not a door is present, this method can also be applied to the panoramic image (each side view image) generated by the process of S506. In other words, the priority of the image including the door may be increased.

[0067] 8, the image determined to have a higher priority is displayed preferentially. Also, the captured image determined to have a lower priority may not be acquired from the imaging device 100.

[0068] When the process of Fig. 8 is performed, a display as shown in Fig. 10 may be performed. Fig. 10 shows an example in which a front image and a rear image are displayed in double panorama.

[0069] Reference numeral 1000 denotes a display frame of a double panorama in which a fisheye image is divided and expanded. A fisheye image 1200 is divided by a division line 702 on the fisheye image in the panorama expansion, and 1001 and 1002 on the fisheye image are areas expanded as a panorama image. Reference numeral 1005 denotes a display area of ​​a panorama image generated by dewarping a rear image 1001 that captures the rear of the bus. Reference numeral 1004 denotes a display area of ​​a panorama image generated by dewarping a front image 1002 that captures the front of the bus. When the priority of the front image is determined to be high by the process of FIG. 8, a panorama display area 1004 with high priority is highlighted with a thick line as in the frame 1003 and displayed. As in this example, a panorama display with high priority is highlighted. The highlighting method is not limited to a method of highlighting the frame, and may be expressed by changing the layout of the GUI or the size of the panorama display frame with high priority.

[0070] Next, the flowchart of Fig. 5(b) will be described, which is an example of a case where multiple imaging devices 100 that do not use fisheye lenses are used. Fig. 5(a) and Fig. 5(b) differ only in S510 and S511, and the other steps are similar. Note that, as an example, it is assumed here that the following four imaging devices 100 are installed on a bus. That is, there are four imaging devices 100: an imaging device 100 that captures a side direction image (right), an imaging device 100 that captures a side direction image (left), an imaging device 100 that captures a rearward image, and an imaging device 100 that captures a forward image.

[0071] According to the result of the determination described in FIG. 6, if the bus stop location is a bus stop, the process proceeds to S510, and if the bus stop location is not a bus stop, the process proceeds to S511.

[0072] In S510, the system control unit 203 causes the imaging device 100 capturing an image of the side (right direction) of the bus from inside the bus to generate an image via the communication unit 204. The system control unit 203 also causes the imaging device 100 capturing an image of the side (left direction) of the bus from inside the bus to generate an image. Note that it is also possible to cause only the imaging device 100 capturing an image of the side including the door to perform imaging.

[0073] On the other hand, in S511, the system control unit 203 causes the imaging device 100 capturing an image of the rear of the bus from inside the bus to generate a captured image via the communication unit 204. The system control unit 203 also causes the imaging device 100 capturing an image of the front of the bus from inside the bus to generate a captured image. Note that it is also possible to cause only the imaging device 100 capturing an image of the rear to perform imaging.

[0074] Then, in S509, the system control unit 203 of the client device 200 acquires the captured images captured in S510 or S511. Note that all of the side image (right), side image (left), rear image, and front image may be acquired in S509. In that case, the system control unit 203 of the client device 200 selects an image to display according to the priority of each image, as described below.

[0075] Next, with reference to FIG. 13, an image display control method in this embodiment will be described.

[0076] Fig. 13(a) is an example of a display when it is determined in S504 that the bus is not stopped at a bus stop. As shown in Fig. 13(a), when it is determined that the bus is not stopped at a bus stop, the system control unit 203 acquires at least a rear image and displays it on the display unit 201. Note that a forward image may also be displayed. Furthermore, regardless of the bus stop position, when the bus is not stopped, a display such as that shown in Fig. 13(a) may be performed.

[0077] FIG. 13(b) is an example of a display when it is determined in S504 that the bus is stopped at a bus stop. As shown in FIG. 13(b), when it is determined that the bus is stopped at a bus stop, the system control unit 203 acquires at least a side view image including the door and displays it on the display unit 202. Note that both side view images may be displayed. Also, when the bus is stopped regardless of the stopping position, a display such as that shown in FIG. 13(b) may be performed. In this way, in FIG. 13(a) and FIG. 13(b), a side view image and a rear image / front image are selectively displayed.

[0078] Fig. 13(c) is another display example when it is determined in S504 that the bus is not stopped at a bus stop. As shown in Fig. 13(c), the system control unit 203 displays the rear image larger than the side image (the other image). Also, regardless of the bus stop position, if the bus is not stopped, the display as shown in Fig. 13(c) may be performed.

[0079] Fig. 13(d) is another display example when it is determined in S504 that the bus is stopped at a bus stop. As shown in Fig. 13(d), the system control unit 203 displays the side image larger than the rear image (the other image). Also, regardless of the bus stop position, if the bus is stopped, the display as shown in Fig. 13(d) may be performed.

[0080] As described above, the system control unit 203 determines the display priority according to the determination result of whether the bus is stopped or not, and controls the display of each image according to the priority. Furthermore, the system control unit 203 determines the display priority according to the determination result of whether the bus is stopped at a bus stop, and controls the display of each image according to the priority. Note that when the client device 200 receives all the captured images, the system control unit 203 determines whether to display each image and how to display it.

[0081] Next, a method for making it easy to understand what area the image displayed on the display unit 202 was captured will be described with reference to FIG.

[0082] As an example, a rear image 900 will be described. An icon 901 is an icon indicating that it is a rear image. The part corresponding to the rear is indicated by hatching. Similarly, an icon 902 is an icon imitating a camera, which faces the part corresponding to the rear.

[0083] The icon is changed depending on whether the image to be displayed is a side view image (right), a side view image (left), a rear view image, or a front view image. The icon may be superimposed on the captured image. In addition to the icon, information indicating the display position on the GUI of the display screen may be displayed to indicate the position in the bus to which the captured image to be displayed corresponds.

[0084] As described above, according to this embodiment, the display is controlled depending on whether the moving body is moving or not, so that an image captured by an imaging device installed inside the moving body can be appropriately displayed.

[0085] <Second embodiment> In the second embodiment, an example will be described in which display is controlled depending on whether an abnormality is detected in addition to whether a moving object is moving. Note that the description of the same parts as in the first embodiment will be omitted as appropriate.

[0086] The flowchart of the second embodiment will be described with reference to Fig. 11. Only S1101 is different from the first embodiment.

[0087] In S1101, the system control unit 203 determines whether or not sudden braking has been performed according to the output from the acceleration sensor or the like input from the sensor input unit 207.

[0088] In Fig. 11(a), if sudden braking has occurred, the process proceeds to S506. The rear image and the front image are displayed with priority over the side image. If sudden braking has not occurred, the process proceeds to S503.

[0089] In Fig. 11B, if sudden braking has occurred, the process proceeds to S511. The rear image and the front image are displayed with priority over the side image. If sudden braking has not occurred, the process proceeds to S503.

[0090] In this way, when sudden braking occurs, the rear image and the front image can be displayed with priority over the side image, and it is particularly preferable to display the rear image with priority.

[0091] (Other Examples) Next, a hardware configuration for realizing the client device 200 and the image capture device 100 of each embodiment will be described with reference to FIG.

[0092] The RAM (Random Access Memory) 222 temporarily stores a computer program executed by the CPU (Central Processing Unit) 221. The RAM 222 also temporarily stores data (commands and image data) acquired from the outside via the communication interface 224. The RAM 222 also provides a work area used when the CPU 221 executes various processes. The RAM 222 also functions as, for example, a frame memory or a buffer memory.

[0093] The CPU 221 executes computer programs stored in the RAM 222. In addition to the CPU, a processor such as a DSP (Digital Signal Processor) or an ASIC (Application Specific Integrated Circuit) may also be used.

[0094] A hard disk drive (HDD) 223 stores an operating system program and image data, as well as computer programs.

[0095] Computer programs and data stored in the HDD 223 are loaded into the RAM 222 as appropriate under the control of the CPU 221, and executed by the CPU 221. Other storage media such as flash memory may be used instead of the HDD. A bus 225 connects the various pieces of hardware. The various pieces of hardware exchange data via the bus 225. The above is the hardware configuration in each embodiment.

[0096] The present invention can also be realized by a process in which one or more processors read and execute a program that realizes one or more functions of the above-mentioned embodiments. The program may be supplied to a system or device having a processor via a network or a storage medium. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions of the above-mentioned embodiments.

[0097] Moreover, each functional block shown in FIG. 2 may be realized by hardware shown in FIG. 14, or can be realized by software.

[0098] Furthermore, the present invention is not limited to the above-described embodiments, and various modifications are possible without departing from the scope of the present invention. For example, combinations of the embodiments are also included in the disclosure of this specification. [Explanation of symbols]

[0099] 100 Imaging device 200 Client Device 103,203 System control section

Claims

1. A display control device that controls display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, A determination means for determining whether the movement of the moving object has stopped; a display control means for controlling display of the plurality of captured images on a display means so as to selectively display, based on at least a result of the determination by the determination means, a first captured image having an angle of view obtained by capturing a side view of the moving body from inside the moving body and a second captured image different from the first captured image, among the plurality of captured images; A display control device comprising:

2. The display control means controls display of the plurality of captured images so as to change a state in which the second captured image is displayed to a state in which the first captured image is displayed when the determination means determines that the movement of the moving object has stopped.

2. The display control device according to claim 1.

3. A display control device that controls display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, A determination means for determining whether the movement of the moving object has stopped; a display control means for controlling display of the plurality of captured images on a display means based on at least a result of the determination by the determination means so that one of a first captured image having an angle of view obtained by capturing an image of a side direction of the moving body from inside the moving body and a second captured image different from the first captured image is displayed larger than the other of the first captured images. A display control device comprising:

4. The display control means controls display of the plurality of captured images so as to change a state in which the second captured image is displayed larger than the first captured image to a state in which the first captured image is displayed larger than the second captured image when the determination means determines that the movement of the moving object has stopped.

4. The display control device according to claim 3.

5. The determining means determines whether the movement of the moving object has stopped at a predetermined position.

5. The display control device according to claim 1, wherein the display control device is a display control device for controlling a display of a display.

6. the vehicle is a bus, The predetermined location is a bus stop.

6. The display control device according to claim 5.

7. the vehicle is a bus, The first captured image is an image with an angle of view captured in the direction of a side of the bus where a door is located among a plurality of sides of the bus.

7. The display control device according to claim 1, wherein the display control device is a display control device for controlling a display of a display.

8. The second captured image is an image with a field of view captured from inside the moving body, the field of view being either rearward or forward of the moving body.

8. The display control device according to claim 1, wherein the display control device is a display control device for controlling a display of a display.

9. the one or more imaging devices are imaging devices using a fisheye lens, The first captured image is an image generated by dividing a fisheye image, which is an image captured using the fisheye lens, by a line segment corresponding to the direction along which the bus travels, and performing distortion correction processing on the divided images.

9. The display control device according to claim 1, wherein the display control device is a display control device for controlling a display of a display.

10. the one or more imaging devices are imaging devices using a fisheye lens, The second captured image is an image generated by dividing a fisheye image, which is an image captured using the fisheye lens, along a line segment corresponding to a direction perpendicular to the direction along which the bus travels, and performing distortion correction processing on the divided images.

10. The display control device according to claim 1,

11. A display control method for controlling display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, comprising: a determination step of determining whether the movement of the moving object has stopped; a display control step of controlling display of the plurality of captured images on a display means so as to selectively display a first captured image having an angle of view obtained by capturing a side view of the moving body from inside the moving body and a second captured image different from the first captured image, based on at least a result of the determination step; A display control method comprising:

12. A display control method for controlling display of a plurality of captured images captured by one or a plurality of imaging devices installed inside a moving body, comprising: a determination step of determining whether the movement of the moving object has stopped; a display control step of controlling display of the plurality of captured images on a display means based on at least a result of the determination step, so that one of a first captured image having an angle of view obtained by capturing an image of a side direction of the moving body from inside the moving body and a second captured image different from the first captured image is displayed larger than the other of the first captured images. A display control method comprising:

13. A program causing a computer to function as each of the means of the display control device according to any one of claims 1 to 10.

Images