Display Processing Device

Abstract

To display plural images on a display part in an appropriate layout.SOLUTION: A display processing device includes: an image acquisition part for acquiring images photographed by plural photographing parts; a setting acquisition part for acquiring the number and the roll angles of plural images, and acquiring setting information regarding arrangement of split screens in the display part when displaying plural images side by side on the display part in a state in which the roll angles are retained; an image processing part for compositing plural images on the basis of setting information and generating display images; and a display control part for displaying the generated display images on the display part.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a display processing device. [Background technology]

[0002] 2. Description of the Related Art There is known a configuration in which a plurality of images, such as images captured by a plurality of imaging units, are divided and displayed on one display unit (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-145515 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the above configuration, if the settings for the number of images, the roll angle of the images (portrait, landscape, etc.), the arrangement angle of the display unit (portrait, landscape, etc.), etc. were different, it could be difficult to display multiple images in an appropriate layout on a single display unit.

[0005] The present invention has been made in view of the above, and has an object to provide a display processing device capable of displaying a plurality of videos on a display unit in an appropriate layout. [Means for solving the problem]

[0006] The display processing device of the present invention includes an image acquisition unit that acquires images captured by multiple shooting units, a setting acquisition unit that acquires the number and roll angle of the multiple images and acquires setting information regarding the arrangement of split screens on the display unit when the multiple images are displayed side by side on the display unit while maintaining the roll angle, an image processing unit that synthesizes the multiple images based on the setting information to generate an image for display, and a display control unit that displays the generated image for display on the display unit. Effect of the Invention

[0007] According to the present invention, a plurality of videos can be displayed on a display unit in an appropriate layout. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a display processing system according to the present embodiment. [Diagram 2] FIG. 2 is a functional block diagram showing an example of a display processing device according to the present embodiment. [Diagram 3] FIG. 3 is a diagram showing an example of a camera image. [Figure 4] FIG. 4 is a diagram showing an example of a layout of a split screen on the display unit. [Diagram 5] FIG. 5 is a diagram showing an example of a layout of a split screen on the display unit. [Figure 6] FIG. 6 is a diagram showing an example of a layout of a split screen on the display unit. [Figure 7] FIG. 7 is a diagram showing an example of a process in which a plurality of camera videos are combined by the video processing unit. [Figure 8] FIG. 8 is a diagram showing an example of a display image generated by the image processing unit. [Figure 9] FIG. 9 is a flowchart showing an example of the flow of processing in the display processing device 20 according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of a display processing device according to the present invention will be described with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, the components in the following embodiments include those that are replaceable and easy for a person skilled in the art, or those that are substantially the same.

[0010] FIG. 1 is a diagram showing a schematic example of a display processing system 100 according to the present embodiment. As shown in FIG. 1, the display processing system 100 includes an image capturing device 10, a display processing device 20, a display device 30, and an input device 40. In the present embodiment, the display processing system 100 is disposed, for example, at a station or a terminal platform of a railway or tramway, and captures and displays the door of a train or other vehicle T and the surrounding area. By displaying the captured image, the display processing system 100 allows, for example, a conductor or station staff member to check the safety of passengers getting on and off while looking at the display unit, and can safely open and close the doors.

[0011] The photographing device 10 has multiple cameras 11, 12, and 13. In this embodiment, the photographing device 10 has, for example, three cameras 11, 12, and 13, but is not limited to this and may have two or four or more cameras. The multiple cameras 11, 12, and 13 are arranged so that the sides of all of the cars T can be viewed when the train stops.

[0012] The image capturing device 10 inputs camera images captured by the cameras 11, 12, and 13 to the display processing device 20. The input signal of the camera images includes a blanking period. Information indicating the roll angle of the captured camera images is inserted into the blanking period. Regarding the roll angle of the camera images, in this embodiment, the horizontal state of the rectangular camera images can be set to a roll angle of 0° (reference angle), and the vertical state can be set to a roll angle of 90°. Note that the information indicating the roll angle is not limited to being inserted into the blanking period, and may be inserted into the extended information portion or other portion of the input signal of the camera images.

[0013] The display device 30 displays a plurality of camera images captured by the imaging device 10 together. The display device 30 has, for example, a rectangular display unit 31. The display device 30 is configured, for example, with the display unit 31 arranged horizontally (arrangement angle 0°). The display device 30 may be configured with the display unit 31 arranged vertically (arrangement angle 90°). The display device 30 may also be configured to be switchable between a state in which the display unit 31 is arranged horizontally and a state in which it is arranged vertically. The display device 30 may also be configured to be able to appropriately adjust the arrangement angle of the display unit 31 between 0° (landscape state) and 90° (portrait state).

[0014] The input device 40 may be a keyboard, a mouse, a touch panel, or the like. The input device 40 can input various information to the display processing device 20. The input device 40 can input information such as the purpose of the display device 30 to the display processing device 20, for example.

[0015] An information processing device such as a computer is used as the display processing device 20. Fig. 2 is a functional block diagram showing an example of the display processing device 20 according to the present embodiment. As shown in Fig. 2, the display processing device 20 has an input / output unit 21, a storage unit 22, and a control unit 23.

[0016] The input / output unit 21 receives input of a plurality of camera images captured by the imaging device 10. The input / output unit 21 outputs a display image synthesized by the control unit 23 to the display device 30. The input / output unit 21 receives input of information input by the input device 40.

[0017] The storage unit 22 stores various information. The storage unit 22 has storage such as a hard disk drive and a solid state drive. An external storage medium such as a removable disk may be used as the storage unit 22. The storage unit 22 stores table information used when synthesizing a plurality of camera images in the image processing unit 26 described later. The table information is information that associates the processing contents when synthesizing the display image with the use of the display device 30. The processing contents when synthesizing the display image will be described later. When the display device 30 is used on a station platform as in this embodiment, for example, it can be used for monitoring passengers getting on and off through the doors of the vehicle T. In addition, the display device 30 may be used for other purposes other than monitoring.

[0018] The control unit 23 has a processing device such as a CPU (Central Processing Unit) and a storage device such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The control unit 23 has a video acquisition unit 24, a setting acquisition unit 25, a video processing unit 26, and a display control unit 27.

[0019] The image acquisition unit 24 acquires the camera images taken by the cameras 11 , 12 , and 13 and input to the input / output unit 21 .

[0020] The setting acquisition unit 25 acquires the number of the multiple camera images and the roll angle. The setting acquisition unit 25 acquires setting information regarding the arrangement of the split screens in the display device 30 when the multiple camera images are displayed side by side on the display device 30 while maintaining the acquired roll angle. The setting information regarding the arrangement of the split screens includes, for example, information on the shape and arrangement angle (portrait, landscape, etc.) of the entire display unit 31 of the display device 30, information on the number of screens of the split screens corresponding to the number of the multiple camera images, information on the shape and arrangement angle of the split screens corresponding to the roll angle of the multiple camera images, and information on the position where the split screens are arranged. Regarding the arrangement angle of the entire display unit 31 of the display device 30 and the arrangement angle of the split screens, for example, when each is rectangular, the landscape state can be set to 0° and the portrait state can be set to 90°. The setting information regarding the arrangement of the split screens can be stored in the storage unit 22 as table information that associates the number of camera images and the roll angle with each of the above items on the display device 30 side. When the setting acquisition unit 25 acquires the number of camera images and the roll angle, it can acquire setting information regarding the layout of the split screens based on the acquired information and the table information stored in the storage unit 22. Note that the setting acquisition unit 25 may calculate setting information regarding the layout of the split screens on the display unit 31 based on the number of camera images and the roll angle, and acquire the calculation result.

[0021] The image processor 26 synthesizes the multiple camera images to generate a display image. The display image is a rectangular image corresponding to the rectangular display unit 31 of the display device 30. The display image is generated so that the multiple camera images are arranged in various states, which will be described later.

[0022] The video processor 26 can synthesize multiple camera images by, for example, either aspect priority processing or stretch priority processing. The aspect priority processing is a processing content in which synthesis is performed with priority given to maintaining the aspect ratio of each image. The stretch priority processing is a processing content in which vertical and horizontal stretching is adjusted so that the entire image is displayed on the entire split screen. It can be set, for example, in advance as to which processing, aspect priority processing or stretch priority processing, the video processor 26 will use for synthesis.

[0023] When compositing with aspect priority processing, the video processor 26 can perform either full-image display processing or full-screen display processing. Full-image display processing is processing content for compositing so as to display the entire camera video on the split screen. Full-screen display processing is processing content for compositing so as to display the camera video over the entire split screen.

[0024] In the full-image display process, when the aspect ratio of the camera image and the split screen differs, the image processor 26 can arrange strips at the vertical or horizontal ends of the split screen. In addition, in the full-screen display process, when the aspect ratio of the camera image and the split screen differs, the image processor 26 can remove the part of the vertical or horizontal end of the camera image that protrudes from the split screen. It can be set in advance, for example, as to which of the full-image display process and the full-screen display process is to be performed.

[0025] When the use of the display device 30 is specified, the image processor 26 synthesizes the multiple camera images with processing contents based on the stored table information. In this case, for example, when the use of the display device 30 is for monitoring passenger safety, it is important that the image in which the aspect ratio is maintained for the camera images can be confirmed on the display device 30. Therefore, in the table information, the use of the display device 30 for monitoring can be associated with aspect priority processing.

[0026] The display control unit 27 outputs the generated display image from the input / output unit 21 to the display device 30. Under the control of the display control unit 27, the display image is displayed on the display unit 31.

[0027] Next, an example of the operation of the display processing system 100 configured as above will be described. In the photographing device 10, camera images photographed by the multiple cameras 11, 12, and 13 are input to the input / output unit 21 of the display processing device 20. FIG. 3 is a diagram showing an example of the camera images. As shown in FIG. 3, the camera images M (M1, M2, and M3) photographed by the multiple cameras 11, 12, and 13 are rectangular and vertically long (roll angle 90°). The blanking period of the input signal of the camera images includes setting information indicating that the camera images photographed by each of the cameras 11, 12, and 13 are vertically long (roll angle 90°). The image acquisition unit 24 acquires the input camera images.

[0028] The setting acquisition unit 25 acquires the number of acquired camera images and the roll angle of each camera image. Furthermore, the setting acquisition unit 25 acquires setting information regarding the arrangement of split screens when multiple camera images are displayed side by side on the display unit 31, based on the number of acquired camera images and the roll angle of each camera image.

[0029] 4 to 6 are diagrams showing examples of layouts of split screens 32 in display unit 31. FIG. 4 shows an example in which the number of camera images is three and the images are vertically long (roll angle 90°). As shown on the left side of FIG. 4, in a display device 30 in which display unit 31 is horizontally long (arrangement angle 0°), three vertically long (arrangement angle 90°) split screens 32 are displayed side by side in the horizontal direction. In this embodiment, the vertical dimension of split screen 32 can be the same as the vertical dimension of display unit 31. Also, the horizontal dimension of split screen 32 can be the dimension obtained by dividing the horizontal dimension of display unit 31 into three equal parts.

[0030] 4, in a display device 30 in which the display unit 31 is arranged vertically (at an angle of 90°), four vertically long (at an angle of 90°) split screens 32 are arranged in two rows and one is not used. In this embodiment, the vertical and horizontal dimensions of the split screen 32 can be set to the dimensions obtained by dividing the vertical and horizontal dimensions of the display unit 31 in half.

[0031] FIG. 5 shows an example in which the number of camera images is three and the images are landscape (roll angle 0°). As shown on the left side of FIG. 5, in a display device 30 in which the display unit 31 is vertically oriented (arrangement angle 90°), three landscape (arrangement angle 0°) split screens 32 are displayed side by side in the vertical direction. In this embodiment, the vertical dimension of the split screens 32 can be equal to the vertical dimension of the display unit 31 divided into three. Also, the horizontal dimension of the split screens 32 can be equal to the horizontal dimension of the display unit 31.

[0032] 5, in a display device 30 in which the display unit 31 is arranged horizontally (arrangement angle 0°), four horizontally long (arrangement angle 0°) split screens 32 are arranged in two rows and one is not used. In this arrangement, the vertical and horizontal dimensions of the split screen 32 can be set to the dimensions obtained by dividing the vertical and horizontal dimensions of the display unit 31 in half.

[0033] FIG. 6 shows an example in which the number of camera images is four and the images are vertically long (roll angle 90°). As shown on the left side of FIG. 6, in a display device 30 in which the display unit 31 is arranged horizontally (arrangement angle 0°), four vertically long (arrangement angle 90°) split screens 32 are displayed side by side in the horizontal direction. In this embodiment, the vertical dimension of the split screens 32 can be the same as the vertical dimension of the display unit 31. Also, the horizontal dimension of the split screens 32 can be the dimension obtained by dividing the horizontal dimension of the display unit 31 into four equal parts.

[0034] 6, in a display device 30 in which the display unit 31 is arranged vertically (at an angle of 90°), four vertically long (at an angle of 90°) split screens 32 are arranged in two rows and two columns. In this configuration, the vertical and horizontal dimensions of the split screens 32 can be set to the dimensions obtained by dividing the vertical and horizontal dimensions of the display unit 31 in half.

[0035] Layout patterns of split screen 32 as shown in Figures 4 to 6 are stored as table information in storage unit 22. Layout patterns are set similarly when the number of camera images is two or five or more, or when display unit 31 is disposed at an angle between 0° and 90°.

[0036] The image processor 26 synthesizes the multiple camera images to generate an image for display based on the acquired setting information. As shown in the examples of Fig. 4 to Fig. 6, the aspect ratio of the split screen 32 varies depending on the arrangement state (vertical or horizontal, etc.) of the display unit 31 and the number of split screens 32. Therefore, when synthesizing multiple camera images, the image processor 26 can process how to display them on the split screen 32 in various ways.

[0037] 7 is a diagram showing an example of a process in which a plurality of camera images are combined by the image processor 26. The image processor 26 combines a plurality of camera images by a process that is preset between aspect priority processing and stretch priority processing.

[0038] First, a case where the image processor 26 combines a plurality of camera images using aspect priority processing will be described. The image processor 26 first obtains the difference between the aspect ratio of the camera images and the aspect ratio of the divided screen of the display unit 31.

[0039] For example, when performing the whole image display process, the image processor 26 enlarges or reduces the camera image so that the camera image does not go beyond the split screen of the display unit 31 and the aspect ratio maintains the aspect ratio of the camera image. For example, when the vertical and horizontal dimensions of the camera image and the split screen are different, the image processor 26 enlarges or reduces the camera image so that one of the vertical and horizontal widths of the camera image matches the corresponding vertical and horizontal widths of the split screen and the other of the vertical and horizontal widths of the camera image is smaller than the corresponding vertical and horizontal widths of the split screen. As shown in FIG. 7, the image processor 26 can generate a composite image MA in which, for example, black stripes are arranged at both ends of the vertical and horizontal ends of the split screen that are blank areas where no image exists. In this case, the composite image MA displays the entire camera image while maintaining the aspect ratio of the camera image.

[0040] Furthermore, when performing full screen display processing, for example, the image processing unit 26 enlarges or reduces the image so as to maintain the aspect ratio, and removes the portion that protrudes from the split screen. For example, when the vertical and horizontal dimensions of the camera image and the split screen are different, the image processing unit 26 enlarges or reduces the camera image so that one of the vertical and horizontal widths of the camera image matches the corresponding vertical and horizontal widths of the split screen, and the other of the vertical and horizontal widths of the camera image is larger than the corresponding vertical and horizontal widths of the split screen. Then, the image processing unit 26 removes the portion that protrudes from the split screen for the other of the vertical and horizontal widths of the camera image to generate a composite image MB. In this case, the composite image MB is displayed in its entirety for one of the vertical and horizontal dimensions of the camera image, but only a portion of the other is displayed. In this case, the enlargement or reduction ratio of the camera image is the same for the vertical and horizontal dimensions, and the composite image MB is displayed in a state in which it is enlarged to fill the entire split screen compared to the full image display processing.

[0041] Furthermore, when performing stretching priority processing, the image processor 26 adjusts the stretching and shrinking of the image vertically and horizontally so that the entire image is displayed on the split screen. For example, the image processor 26 determines the difference between the aspect ratio of the camera image and the aspect ratio of the split screen of the display unit 31. Next, the image processor 26 expands or contracts the image vertically and horizontally to fill the difference in aspect ratio, generating a composite image MC. In this case, the composite image MC adjusts the aspect ratio of the camera image to the aspect ratio of the split screen, so that the entire camera image is displayed on the entire split screen.

[0042] FIG. 8 is a diagram showing an example of a display image generated by the image processing unit 26. As shown in FIG. 8, the image processing unit 26 generates a display image MD in which a plurality of camera images are combined. The display image MD is an image in which three vertically long camera images are arranged horizontally with the same vertical and horizontal dimensions. The camera images included in the display image MD can be combined so that their layouts differ depending on the arrangement of the split screens in the display unit 31. The image processing unit 26 may combine the camera images so that the vertical and horizontal dimensions differ for each camera image. The image processing unit 26 may also combine the camera images so that the processing contents differ for each camera image, for example, by performing aspect priority processing on one camera image and stretch priority processing on the other camera images. For example, when there is a blank portion where no image exists, as in the composite image MA generated by the whole image display processing, the split screen may be shrunk to match the width of the camera image without placing a black belt, and the width of the other split screen may be extended accordingly.

[0043] The display control unit 27 causes the display image generated by the image processing unit 26 to be displayed on the display unit 31. By viewing the display image displayed on the display unit 31, the user can grasp the camera image in a manner appropriate for the purpose.

[0044] Next, an example of the operation of the display processing system 100 according to this embodiment will be described with reference to FIG. 9. FIG. 9 is a flowchart showing an example of the flow of processing in the display processing device 20 according to this embodiment. As shown in FIG. 9, the image acquisition unit 24 acquires the camera images M from the cameras 11, 12, and 13 input to the input / output unit 21 (step S10). The setting acquisition unit 25 acquires the number and roll angles of the multiple camera images M, and acquires setting information regarding the arrangement of the split screen 32 based on the acquired information (step S20). The image processing unit 26 synthesizes the multiple camera images M based on the setting information to generate a display image MD (step S30). The display control unit 27 causes the display unit 31 to display the generated display image MD (step S40).

[0045] As described above, the display processing device 20 of this embodiment includes an image acquisition unit 24 that acquires camera images M captured by multiple cameras 11, 12, and 13, a setting acquisition unit 25 that acquires the number and roll angles of the multiple camera images M and acquires setting information regarding the arrangement of the split screen 32 on the display unit 31 when the multiple camera images M are displayed side by side on the display unit 31 while maintaining the roll angle, an image processing unit 26 that synthesizes the multiple camera images M based on the setting information to generate a display image MD, and a display control unit 27 that displays the generated display image MD on the display unit 31.

[0046] According to this configuration, the arrangement (layout) of split screen 32 on display unit 31 side is automatically set so as to correspond to the number and roll angle of the multiple camera images M, and the multiple camera images are displayed on display unit 31 based on this setting. Therefore, regardless of the settings of the number and roll angle of camera images M, and the arrangement angle of display unit 31, the multiple camera images M can be displayed on display unit 31 in an appropriate layout according to the settings.

[0047] In the display processing device 20 according to this embodiment, the image processor 26 can composite multiple camera images M by aspect priority processing that prioritizes maintaining the aspect ratio of each camera image M, or by stretching priority processing that adjusts the vertical and horizontal stretching so that each camera image M is displayed over the entire split screen 32. With this configuration, the display image MD can be generated so as to have a flexible display mode according to the settings.

[0048] In the display processing device 20 according to this embodiment, the video processing unit 26, in the whole video display process, includes disposing band-shaped portions at the vertical or horizontal ends of the split screen 32 when the aspect ratios of the camera video M and the split screen 32 differ, and in the whole screen priority process, includes removing part of the vertical or horizontal ends of the camera video M when the aspect ratios of the camera video M and the split screen 32 differ. With this configuration, the display video MD can be generated to have a flexible display mode according to the settings.

[0049] In the display processing device 20 according to the present embodiment, information indicating the roll angle is inserted into a blanking period of an input analog signal of the camera image M or into an extended information portion of a digital signal. With this configuration, the roll angle of the camera image M can be efficiently acquired.

[0050] The technical scope of the present invention is not limited to the above-mentioned embodiment, and can be modified as appropriate without departing from the spirit of the present invention. For example, in the above-mentioned embodiment, the display processing system 100 is installed at a station or a terminal platform of a railway, a tram, or the like, but is not limited thereto. The display processing system 100 can also be applied to a case where, for example, images captured by a plurality of capture units in a broadcasting station or the like are divided and displayed on one or a plurality of display units.

[0051] In the above embodiment, the camera image and the split screen are rectangular in shape, but the present invention is not limited to this. The camera image and the split screen may be polygonal, such as a triangle, circular, elliptical, or other shapes other than rectangular. Similarly, the roll angle of the camera image and the arrangement angle of the display unit 31 and the split screen are not limited to 0° and 90°, and may be set between 0° and 90°.

[0052] In the above embodiment, the case where a plurality of camera images, all of which have the same roll angle, are displayed on the display unit 30 has been described as an example, but the present invention is not limited to this. The display processing system 100 of the above embodiment can also be applied to a case where a plurality of camera images, all of which have different roll angles, are displayed on the display unit 30. [Explanation of symbols]

[0053] M (M1, M2, M3)...camera image, MA, MB, MC...synthetic image, MD...image to be displayed, T...vehicle, 10...imaging device, 11, 12, 13...camera, 20...display processing device, 21...input / output unit, 22...storage unit, 23...control unit, 24...image acquisition unit, 25...setting acquisition unit, 26...image processing unit, 27...display control unit, 30...display device, 31...display unit, 32...split screen, 40...input device, 100...display processing system

Claims

1. an image acquisition unit that acquires images captured by a plurality of image capture units; a setting acquisition unit that acquires the number and roll angle of the plurality of videos, and acquires setting information including information regarding an arrangement of split screens on the display unit when the plurality of videos are displayed side by side on the display unit while maintaining the roll angle, and information regarding an arrangement angle of the display unit; an image processing unit that synthesizes the plurality of images based on the setting information to generate an image for display; a display control unit that displays the generated display image on the display unit; Equipped with When the arrangement angle of the display unit is changed from a state in which the screen of the display unit, which has a horizontal arrangement angle, is divided into three in a horizontal direction and three images are displayed on the divided screens, the image processing unit divides the screen of the display unit into a number obtained by adding one to the number of images, and generates an image for display by synthesizing the images without using one of the divided screens obtained by adding one to the number of images. Display processing device.

2. An image acquisition unit that acquires images captured by a plurality of image capture units; a setting acquisition unit that acquires the number and roll angle of the plurality of videos, and acquires setting information including information regarding an arrangement of split screens on the display unit when the plurality of videos are displayed side by side on the display unit while maintaining the roll angle, and information regarding an arrangement angle of the display unit; an image processing unit that synthesizes the plurality of images based on the setting information to generate an image for display; a display control unit that displays the generated display image on the display unit; Equipped with When the arrangement angle of the display unit is changed to landscape from a state in which the screen of the display unit, which has a vertical arrangement angle, is divided into three split screens in a vertical direction and three images are displayed on the split screens, the image processing unit divides the screen of the display unit into a number obtained by adding one to the number of images, and generates an image for display by synthesizing the images without using one of the split screens divided into a number obtained by adding one to the number of images. Display processing device.

3. The image processor is capable of synthesizing the plurality of images by aspect-prioritizing processing that prioritizes maintaining the aspect ratio of each of the images, or by stretch-prioritizing processing that adjusts the vertical and horizontal stretching so that each of the images is displayed on the entire split screen. The display processing device according to claim 1 or 2.

4. An input unit for inputting usage information which is information on the usage of the display unit, When the inputted use information is a surveillance use, the image processing unit synthesizes the plurality of images by the aspect priority processing. The display processing device according to claim 3 .

Images