Image processing system and image processing method

Abstract

This image processing system (1) comprises a first acquisition unit (11), a second acquisition unit (12), a processing unit (14), and an output unit (15). The first acquisition unit (11) acquires an image signal including image data of an original image. The second acquisition unit (12) acquires a selection signal for selecting a predetermined region from among a plurality of regions included in the original image. The processing unit (14) generates a plurality of cut-out images obtained by cutting out each of the plurality of regions from the original image. The output unit (15) outputs the plurality of cut-out images. On the basis of the selection signal, the processing unit (14) executes distortion correction processing for correcting distortion in the cut-out image obtained by cutting out the predetermined region.

Description

Image processing system and image processing method

[0001] The present disclosure relates to an image processing system and an image processing method.

[0002] Patent Literature 1 discloses a local user device operable to conduct a video conference with a remote user device, wherein the local user device defines a region of interest (ROI) within a field of view of a camera of the remote user device, and receives encoded video within the defined ROI from the remote user device.

[0003] Patent Document 2 discloses an image processing device that generates an image in which distortion contained in a captured image generated by an imaging unit has been corrected.

[0004] JP 2021-52415 A International Publication No. 2017 / 179722

[0005] The present disclosure provides an image processing system and the like that can easily reduce the processing load when outputting an image.

[0006] An image processing system according to one aspect of the present disclosure includes a first acquisition unit, a second acquisition unit, a processing unit, and an output unit. The first acquisition unit acquires an image signal including image data of an original image. The second acquisition unit acquires a selection signal for selecting a predetermined region from a plurality of regions included in the original image. The processing unit generates a plurality of cropped images by cropping each of the plurality of regions from the original image. The output unit outputs the plurality of cropped images. The processing unit performs distortion correction processing based on the selection signal to correct distortion in the cropped images obtained by cropping the predetermined region.

[0007] The present disclosure has an advantage in that it is easy to reduce the processing load when outputting an image.

[0008] Fig. 1 is a block diagram showing an overall configuration including an image processing system according to an embodiment. Fig. 2 is an explanatory diagram of distortion correction processing by the image processing system according to an embodiment. Fig. 3 is a flowchart showing an example of operation of the image processing system according to an embodiment. Fig. 4 is an explanatory diagram of an example of use of the image processing system according to an embodiment.

[0009] [1. Findings that Form the Basis of the Present Disclosure] First, the inventor's viewpoint will be explained below.

[0010] For example, a technique for extracting a region of interest (ROI) from a bird's-eye view image, such as that disclosed in Patent Document 1, is effective for cost-saving imaging because it can extract images from a plurality of different angles from an image captured by a single camera. However, the technique disclosed in Patent Document 1 has a problem in that when an image is extracted from an image of a subject captured at a wide angle, wide-angle distortion occurs in the extracted image, making the extracted image appear unnatural to the viewer.

[0011] Patent Literature 2 discloses a technique for converting a cut-out image into an image that appears natural to a viewer by performing distortion correction processing on the cut-out image based on information such as the focal length of the camera. However, the distortion correction processing disclosed in Patent Literature 2 is a relatively high-load process that requires calculation for each pixel. Therefore, when using the technique disclosed in Patent Literature 2, there is a problem in that, for example, when cutting out images from multiple different angles from a bird's-eye view image, the number of images that can be cut out is limited.

[0012] In view of the above, the inventors have come up with the present disclosure.

[0013] Hereinafter, embodiments will be described with reference to the drawings. Note that the embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection forms, steps, step order, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. Furthermore, among the components in the following embodiments, components not described in independent claims will be described as optional components.

[0014] It should be noted that the drawings are schematic diagrams and are not necessarily strict illustrations. In addition, in the drawings, substantially the same components are denoted by the same reference numerals, and overlapping descriptions may be omitted or simplified.

[0015] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not refer to the number or order of components unless otherwise specified, but are used for the purpose of avoiding confusion and distinguishing between components of the same type.

[0016] (Embodiment) [2. Configuration] The following describes the overall configuration including an image processing system 1 according to an embodiment. Fig. 1 is a block diagram showing the overall configuration including an image processing system 1 according to an embodiment. The image processing system 1 is a system that generates an image (hereinafter also referred to as a "cut-out image P1" (see Fig. 2)) by cutting out a part of an acquired image (hereinafter also referred to as an "original image P0" (see Fig. 2)), and outputs the generated cut-out image. Both the original image P0 and the cut-out image P1 may be moving images or still images.

[0017] In the embodiment, the image processing system 1 generates a cut-out image P1 by cutting out an area specified on the operation terminal 5 from an original image P0 captured by the camera 2, and outputs the generated cut-out image P1 to the switcher 3 and the display device 4. Also, in the embodiment, there are multiple first operators, and each first operator performs operations using the operation terminal 5 while looking at the display device 4. Therefore, in the embodiment, there are multiple display devices 4 and multiple operation terminals 5, and the number of display devices 4 and the number of operation terminals 5 both correspond to the number of first operators. Note that the role of the first operators corresponds to the role of a cameraman in a conventional video production site or a live broadcast site.

[0018] Camera 2 is an imaging device capable of capturing moving images or still images. In the embodiment, camera 2 is capable of capturing moving images or still images with a relatively high resolution, such as 6K or 8K. In the embodiment, camera 2 is installed in a manner fixed to a base, wall, ceiling, or the like at a video production site or a live broadcast site, for example. Camera 2 includes an optical system 21, an imaging unit 22, an image processing unit 23, and a transmission unit 24.

[0019] The optical system 21 includes, for example, a focus lens and a zoom lens. The focus lens is made up of a combination of one or more lenses. The zoom lens is made up of a combination of one or more lenses.

[0020] The imaging unit 22 generates image data from the optical information input via the optical system 21. The imaging unit 22 has an image sensor that converts the optical information into an electrical signal (analog signal) and an A / D converter that converts the analog signal into a digital signal (image signal). The image sensor is, for example, a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor.

[0021] The image processing unit 23 generates image data by performing appropriate processing on the image signal converted by the imaging unit 22. For example, the image processing unit 23 generates image data by encoding the image signal in a predetermined format. Metadata, which is information indicating the imaging conditions at the time of imaging, is added to the image data. The imaging conditions may include, for example, the focal length, exposure, shutter speed, or angle of view of the camera 2.

[0022] The transmitter 24 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a local area network (LAN), SDI (Serial Digital Interface) transmission, or ST2110 IP (Internet Protocol) transmission. The communication between the transmitter 24 and the image processing system 1 may be wired communication or wireless communication. The transmitter 24 transmits the image data generated by the image processing unit 23 to the image processing system 1 via a network, SDI transmission, ST2110 IP transmission, or the like.

[0023] The switcher 3 is a device for switching between broadcast or distributed video. In the embodiment, the switcher 3 is realized by installing dedicated software for the switcher 3 in, for example, a server device or a general-purpose information terminal such as a desktop or laptop personal computer. The switcher 3 may also be realized by dedicated hardware. The switcher 3 includes an operation processing unit 31, a display unit 32, an image processing unit 33, a transmission unit 34, and a reception unit 35.

[0024] The operation processing unit 31 executes processing in response to an operation input by a second operator who operates the switcher 3. For example, when the operation processing unit 31 receives an input by the second operator to select program image P2 (see FIG. 4), it generates a selection signal indicating that the program image P2 has been selected. Also, for example, when the operation processing unit 31 receives an input by the second operator to select candidate image P3 (see FIG. 4) of program image P2, it generates a selection signal indicating that the candidate image P3 has been selected.

[0025] Here, program image P2 refers to the image to be broadcast or distributed. Also, candidate image P3 refers to the image to be broadcast or distributed next after program image P2. Therefore, as the second operator sequentially selects program image P2 and candidate image P3, program image P2 is broadcast or distributed sequentially.

[0026] In the embodiment, the program image P2 and the candidate image P3 are both one of a plurality of regions included in the original image P0, in other words, one of a plurality of cut-out images P1 cut out from the original image P0. In other words, the selection signal indicating that the program image P2 (or the candidate image P3) has been selected corresponds to a signal that selects a predetermined region from a plurality of regions included in the original image P0.

[0027] In the embodiment, the selection signal is a so-called tally signal. Conventionally, a tally signal is a signal transmitted to a camera to illuminate a tally light or other indicator light provided on the camera in a predetermined color. For example, when a camera receives a red tally signal, the cameraman operating the camera can understand that the image captured by the camera he or she operates will be used as program image P2. Furthermore, when a camera receives a green tally signal, the cameraman operating the camera can understand that the image captured by the camera he or she operates is candidate image P3, i.e., will be used next as program image P2.

[0028] Therefore, in this embodiment, when the operation processing unit 31 receives an input from the second operator to select program image P2, it generates a red tally signal as a selection signal.Further, when the operation processing unit 31 receives an input from the second operator to select candidate image P3, it generates a green tally signal.

[0029] The display unit 32 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays the original image P0 received by the receiving unit 35. Furthermore, if the receiving unit 35 receives a plurality of cut-out images P1, the display unit 32 displays the plurality of cut-out images P1. Note that the display unit 32 may display the original image P0 and the plurality of cut-out images P1 all at once, or may switch between displaying the original image P0 and the plurality of cut-out images P1 in response to an operation input by the second operator.

[0030] The image processing unit 33 decodes the image data received by the receiving unit 35. The image (original image P0 or a plurality of cut-out images P1) obtained by decoding the image data is displayed on the display unit 32.

[0031] The transmitter 34 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a LAN, SDI transmission, or IP transmission of ST2110. The communication between the transmitter 34 and the image processing system 1 may be wired communication or wireless communication. The transmitter 34 transmits the selection signal generated by the operation processing unit 31 to the image processing system 1 via the network.

[0032] The receiving unit 35 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a LAN, SDI transmission, IP transmission of ST2110, etc. The communication between the receiving unit 35 and the image processing system 1 may be wired communication or wireless communication. The receiving unit 35 receives image data of the original image P0 and image data of the multiple cropped images P1 output from the image processing system 1 via a network, SDI transmission, IP transmission of ST2110, etc.

[0033] The display device 4 is a device for the first operator to check the image (original image P0) captured by the camera 2. The display device 4 includes a display unit 41, an image processing unit 42, and a receiving unit 43.

[0034] The display unit 41 is, for example, a liquid crystal display or an organic EL display, and displays the original image P0 received by the receiving unit 35.

[0035] The image processing unit 42 decodes the image data received by the receiving unit 43. The image (original image P0) obtained by decoding the image data is displayed on the display unit 41.

[0036] The receiving unit 43 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a LAN, SDI transmission, IP transmission of ST2110, etc. The communication between the receiving unit 43 and the image processing system 1 may be wired communication or wireless communication. The receiving unit 43 receives image data of the original image P0 output from the image processing system 1 via a network, SDI transmission, IP transmission of ST2110, etc.

[0037] The operation terminal 5 is a terminal on which the first operator performs operations required for generating the cut-out image P1, such as specifying an area to be cut out from the image (original image P0) captured by the camera 2. The operation terminal 5 may include a pointing device such as a mouse, or may include a joystick. The operation terminal 5 includes an operation processing unit 51 and a transmission unit 52.

[0038] The operation processing unit 51 executes processing in response to an operation input by a first operator. For example, when the operation processing unit 51 receives an input specifying an area to be cut out from the original image P0, it generates a control signal including information indicating the specified area of ​​the original image P0. Furthermore, when the operation processing unit 51 further receives an input instructing at least one of pan, tilt, and zoom control on the specified area, it generates a control signal that further includes information indicating the instruction.

[0039] The transmitter 52 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a LAN. The communication between the transmitter 52 and the image processing system 1 may be wired or wireless. The transmitter 52 transmits the control signal generated by the operation processor 51 to the image processing system 1 via the network.

[0040] The image processing system 1 is realized by an image processing unit including a processing circuit such as an FPGA (Field-Programmable Gate Array). The image processing system 1 may also be realized by an information processing device other than the image processing unit, such as a personal computer. The image processing system 1 includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, a processing unit 14, and an output unit 15.

[0041] The first acquisition unit 11 is a communication interface for communicating with the camera 2 via a network such as the Internet or a LAN, SDI transmission, or ST2110 IP transmission. The communication between the first acquisition unit 11 and the camera 2 may be wired communication or wireless communication. The first acquisition unit 11 receives image data transmitted from the camera 2 via a network, SDI transmission, ST2110 IP transmission, or the like. In other words, the first acquisition unit 11 acquires an image signal including image data of the original image P0.

[0042] The second acquisition unit 12 is a communication interface for communicating with the switcher 3 via a network such as the Internet or a LAN, SDI transmission, or IP transmission of ST2110. The communication between the second acquisition unit 12 and the switcher 3 may be wired communication or wireless communication. The second acquisition unit 12 receives (acquires) the selection signal transmitted from the switcher 3 via a network, SDI transmission, or IP transmission of ST2110.

[0043] The third acquisition unit 13 is a communication interface for communicating with each of the multiple operation terminals 5 via a network such as the Internet or a LAN. The communication between the third acquisition unit 13 and each of the multiple operation terminals 5 may be wired communication or wireless communication. The third acquisition unit 13 receives (acquires) a control signal transmitted from each of the multiple operation terminals 5 via the network.

[0044] The processing unit 14 decodes the image data of the original image P0. The processing unit 14 also generates a plurality of cut-out images P1 by respectively cutting out a plurality of regions from the original image P0. Specifically, the processing unit 14 generates the plurality of cut-out images P1 from the original image P0 based on a plurality of control signals acquired from a plurality of operation terminals 5. For example, assume that there are three operation terminals 5 (i.e., three first operators). Here, the three operation terminals 5 are referred to as the "first operation terminal," the "second operation terminal," and the "third operation terminal." Furthermore, the control signals transmitted by the "first operation terminal," the "second operation terminal," and the "third operation terminal" are referred to as the "first control signal," the "second control signal," and the "third control signal," respectively.

[0045] In the above example, the processing unit 14 generates a "first cut-out image" by cutting out from the original image P0 an area specified by a "first control signal" transmitted from the "first operation terminal." The processing unit 14 also generates a "second cut-out image" by cutting out from the original image P0 an area specified by a "second control signal" transmitted from the "second operation terminal." The processing unit 14 also generates a "third cut-out image" by cutting out from the original image P0 an area specified by a "third control signal" transmitted from the "third operation terminal."

[0046] In the embodiment, the processing unit 14 generates a plurality of cut-out images P1 at a resolution lower than the resolution of the original image P0. For example, if the resolution of the original image P0 is 8K, the processing unit 14 generates a plurality of cut-out images P1 at a resolution of 2K.

[0047] Here, for each of the plurality of cut-out images P1, the processing unit 14 generates the cut-out image P1 using either the first cut-out processing unit 141 or the second cut-out processing unit 142 determined based on the selection signal. In the embodiment, both the first cut-out processing unit 141 and the second cut-out processing unit 142 are functions that can be executed by the processing unit 14.

[0048] The first cutout processing unit 141 cuts out a specified area from the original image P0 based on the control signal to generate a cutout image P1. The first cutout processing unit 141 does not perform distortion correction processing on the cutout image P1.

[0049] The second cutout processing unit 142 cuts out a designated area from the original image P0 based on the control signal, similar to the first cutout processing unit 141. The second cutout processing unit 142 performs distortion correction processing on the cutout image P1.

[0050] The distortion correction process will now be described with reference to Fig. 2. Fig. 2 is an explanatory diagram of the distortion correction process performed by the image processing system 1 according to the embodiment. Fig. 2 shows an image obtained by capturing an image of a soccer match with a camera 2. Note that the illustration of stand seats, goals, and the like is omitted from Fig. 2. Fig. 2(a) shows an original image P0 captured by the camera 2, and Fig. 2(b) shows a cut-out image P1 obtained by cutting out a specified area from the original image P0. Fig. 2(c) shows an image obtained by performing distortion correction process on the cut-out image P1 shown in Fig. 2(b).

[0051] For example, as shown in FIG. 2B, when a subject is captured with a wide-angle lens, distortion generally appears on the subject as the distance from the center of the captured image increases. Hereinafter, this type of distortion will also be referred to as "wide-angle distortion." Wide-angle distortion is a type of distortion whose magnitude increases with distance from the center of the captured image. Specifically, it is a type of distortion in which the width of the image is expanded in both the horizontal and vertical directions. Therefore, in this embodiment, the following distortion correction process is performed to correct a cropped image P1 having wide-angle distortion as shown in FIG. 2B to a cropped image P1 in which the wide-angle distortion has been eliminated as shown in FIG. 2C.

[0052] In the distortion correction process, the processing unit 14 sets a spherical surface centered on the optical center (here, the optical center of the focus lens of the optical system 21) and maps each point (pixel) on the mapping surface corresponding to the pre-correction image onto the spherical surface. Next, the processing unit 14 sets a correction mapping surface parallel to the mapping surface and maps each pixel mapped onto the spherical surface onto the correction mapping surface. The processing unit 14 then maps each pixel mapped onto the correction mapping surface back to the original mapping surface so as to restore the original size of the mapping surface. Here, the correspondence between the position of each pixel on the mapping surface of the pre-correction image and the position of each pixel in the corrected image is calculated in advance by defining the above series of mappings, and stored in, for example, a memory referenced by the processing unit 14. By using the correspondence, the processing unit 14 can determine each pixel in the pre-correction image that corresponds to each pixel in the corrected image, thereby correcting wide-angle distortion in the image.

[0053] Based on the selection signal, the processing unit 14 designates an area (predetermined area) designated as a program image P2 or a candidate image P3 among multiple areas included in the original image P0 as a "selection target area," and designates other areas as "non-selection target areas." When cutting out a selection target area, the processing unit 14 generates a cut-out image P1 using the second cut-out processing unit 142, and when cutting out a non-selection target area, the processing unit 14 generates a cut-out image P1 using the first cut-out processing unit 141. In other words, the processing unit 14 performs distortion correction processing on the selection target area, but does not perform distortion correction processing on the non-selection target area. In this way, based on the selection signal, the processing unit 14 performs distortion correction processing on the cut-out image P1 obtained by cutting out the predetermined area (selection target area).

[0054] Furthermore, when the control signal includes information instructing at least one of pan, tilt, and zoom control, the processing unit 14 generates the cutout image P1 based on that information. For example, when the control signal includes information instructing pan control, the processing unit 14 generates the cutout image P1 while continuously changing the cutout position, thereby making it possible to obtain an image equivalent to an image obtained by panning a normal camera that is placed in the same position as the camera 2 and outputs an image of the same size as the cutout image P1.

[0055] The output unit 15 outputs the original image P0 captured by the camera 2 and the cut-out image P1 generated by the processing unit 14. Specifically, the output unit 15 outputs an image obtained by encoding the original image P0 captured by the camera 2 to the switcher 3 and each display device 4. The output unit 15 also outputs an image obtained by encoding the cut-out image P1 generated by the processing unit 14 to the switcher 3 and the corresponding display device 4. Here, the "corresponding display device 4" refers to the display device 4 used by the first operator who instructed the generation of the cut-out image P1. The output unit 15 also distributes or broadcasts the image selected by the switcher 3 (program image P2).

[0056] 3. Operation The operation of the image processing system 1 according to the embodiment, that is, the image processing method according to the embodiment, will be described below. Fig. 3 is a flowchart showing an example of the operation of the image processing system 1 according to the embodiment.

[0057] First, the image processing system 1 acquires an image signal including image data of the original image P0 transmitted from the camera 2 (S1). Although not shown here, after acquiring the image signal, the image processing system 1 transmits the image data of the original image P0 to the switcher 3 and each of the plurality of display devices 4. As a result, the original image P0 is displayed on the display unit 32 of the switcher 3 and the display unit 41 of each of the plurality of display devices 4.

[0058] Next, the image processing system 1 acquires a plurality of control signals transmitted from each of the plurality of operation terminals 5 (S2). Although not shown here, the image processing system 1 generates a plurality of cut-out images P1 by cutting out areas specified by each of the plurality of control signals from the original image P0, and transmits image data of the generated cut-out images P1 to the switcher 3. As a result, the display unit 32 of the switcher 3 displays the plurality of cut-out images P1.

[0059] Next, the image processing system 1 acquires the selection signals transmitted from the switcher 3 (S3). In the embodiment, as already described, the image processing system 1 acquires a red tally signal as a selection signal and a green tally signal as a selection signal.

[0060] Next, the image processing system 1 determines whether to perform distortion correction processing on the multiple regions indicated by each of the multiple control signals based on the selection signal. If the region is a region to be selected (a predetermined region) (S4: Yes), the image processing system 1 generates a cropped image P1 in which distortion correction processing has been performed on the region (S5). On the other hand, if the region is not a region to be selected (S4: No), the image processing system 1 generates a cropped image P1 in which distortion correction processing has been performed on the region (S6).

[0061] Then, the image processing system 1 outputs the cut-out image P1 generated in step S5 or step S6 (S7). Here, the image processing system 1 transmits the generated cut-out image P1 to the display device 4 of the first operator who cut out the area. Furthermore, if the generated cut-out image P1 is a program image P2, the image processing system 1 distributes or broadcasts the cut-out image P1.

[0062] If there are other unprocessed areas (S8: Yes), the image processing system 1 repeats steps S4 to S7. If there are no other unprocessed areas (S8: No), the image processing system 1 ends the process.

[0063] [4. Usage Example] A usage example of the image processing system 1 according to the embodiment will be described below. Fig. 4 is an explanatory diagram of a usage example of the image processing system 1 according to the embodiment. Note that in the example shown in Fig. 4, wide-angle distortion is not expressed in all images, but in reality, wide-angle distortion occurs in one or more images out of all images.

[0064] The example shown in Fig. 4 illustrates a situation in which three first operators are operating the operation terminal 5 while viewing the display device 4. The example shown in Fig. 4 also illustrates a situation in which one second operator is operating the switcher 3 to select a program image P2 and a candidate image P3. The top row in Fig. 4 illustrates the scene being photographed by the camera 2, and the second row from the top in Fig. 4 illustrates the original image P0. The third row from the top in Fig. 4 illustrates an image displayed on the display unit 32 of the switcher 3, and the bottom row in Fig. 4 illustrates an image (here, program image P2) displayed on the display unit 41 of one of the display devices 4. In the example shown in Fig. 4, it is assumed that time passes in the order of Fig. 4(a), Fig. 4(b), Fig. 4(c), and Fig. 4(d).

[0065] A specific example of use of the image processing system 1 will be described below with reference to FIG. 4A. Note that the image processing system 1 performs the same operations in FIGS. 4B, 4C, and 4D as in FIG. 4A, except for different times. Therefore, a description thereof will be omitted. As shown in FIG. 4A, three first operators each designate a first region A1, a second region A2, and a third region A3 in the original image P0. Therefore, the image processing system 1 acquires a control signal including information designating the first region A1, a control signal including information designating the second region A2, and a control signal including information designating the third region A3. The image processing system 1 then generates a cutout image P1 cut out of the first region A1, a cutout image P1 cut out of the second region A2, and a cutout image P1 cut out of the third region A3, and transmits the image data of these generated cutout images P1 to the switcher 3.

[0066] A plurality of cut-out images P1 (here, three cut-out images P1) are displayed on the display unit 32 of the switcher 3. The second operator selects a program image P2 and a candidate image P3 while viewing the plurality of cut-out images P1 displayed on the display unit 32. In the example shown in Fig. 4, the cut-out image P1 cut out of the first area A1 surrounded by a solid-line frame represents the program image P2, and the cut-out image P1 cut out of the second area A2 surrounded by a dashed-line frame represents the candidate image P3. Note that the cut-out image P1 cut out of the third area A3 surrounded by a dotted-line frame is neither the program image P2 nor the candidate image P3.

[0067] The switcher 3 transmits a red tally signal to the image processing system 1, the destination of which is the display device 4 of the first operator that cut out the first area A1. The switcher 3 also transmits a green tally signal to the image processing system 1, the destination of which is the display device 4 of the first operator that cut out the second area A2.

[0068] When the image processing system 1 receives a red tally signal as a selection signal, it generates a cutout image P1 by performing distortion correction processing on the area to be selected (here, the first area A1), and transmits image data of the generated cutout image P1 to the display device 4 of the first operator who cut out the first area A1. When the image processing system 1 receives a green tally signal as a selection signal, it generates a cutout image P1 by performing distortion correction processing on the area to be selected (here, the second area A2), and transmits image data of the generated cutout image P1 to the display device 4 of the first operator who cut out the second area A2. Note that the image processing system 1 generates a cutout image P1 without performing distortion correction processing on an area not to be selected (here, the third area A3), and transmits image data of the generated cutout image P1 to the display device 4 of the first operator who cut out the third area A3.

[0069] [5. Advantages, etc.] Advantages of the image processing system 1 (image processing method) according to the embodiment will be described below. As described above, the image processing system 1 according to the embodiment performs distortion correction processing on a cut-out image P1 obtained by cutting out a predetermined area from among multiple areas included in an original image P0, based on a selection signal. Therefore, the image processing system 1 according to the embodiment performs distortion correction processing only on a cut-out image P1 that is likely to be viewed by a viewer, such as an image to be distributed or broadcast, and therefore has the advantage of easily reducing the processing load when outputting an image, compared to when distortion correction processing is performed on all cut-out images P1.

[0070] In other words, in the image processing system 1 according to the embodiment, distortion correction processing is performed only on the necessary images at the necessary timing, and therefore the problem of limiting the number of images that can be extracted when extracting images from multiple different angles from an overhead image as described in [1. Knowledge forming the basis of the present disclosure] is unlikely to occur.

[0071] 6. Other Embodiments Although the embodiments have been described above, the present disclosure is not limited to the above-described embodiments.

[0072] For example, in the above embodiment, the processing unit 14 performs distortion correction processing using mapping, but this is not limiting. That is, the processing unit 14 only needs to perform distortion correction processing on the cut-out image P1 obtained by cutting out a predetermined area based on the selection signal, and the distortion correction processing method may be a method other than a method using mapping.

[0073] For example, in the above embodiment, the image processing system 1 is realized by a device such as an image processing unit separate from the switcher 3, but this is not limiting. For example, the image processing system 1 may be mounted on the switcher 3.

[0074] For example, in the above embodiment, the image processing system 1 is realized by a single device, but this is not limiting, and the image processing system 1 may be realized by a plurality of devices.

[0075] In the above-described embodiment, the processing performed by a specific processing unit may be performed by another processing unit. The order of multiple processing operations may be changed, or multiple processing operations may be performed in parallel.

[0076] In the above-described embodiments, each component may be realized by executing a software program suitable for that component, or by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0077] Furthermore, each component may be realized by hardware. Each component may be a circuit (or integrated circuit). These circuits may form a single circuit as a whole, or each may be a separate circuit. Furthermore, each of these circuits may be a general-purpose circuit or a dedicated circuit.

[0078] Furthermore, the general or specific aspects of the present disclosure may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0079] The present disclosure may also be realized as an image processing method executed by a computer such as the image processing system of the above-described embodiment. The present disclosure may also be realized as a program (computer program product) for causing a computer to execute such an image processing method, or as a computer-readable non-transitory recording medium on which such a program is recorded.

[0080] In addition, this disclosure also includes forms obtained by applying various modifications to each embodiment that a person skilled in the art would think of, or forms realized by arbitrarily combining the components and functions of each embodiment within the scope that does not deviate from the intent of this disclosure.

[0081] (Summary) As described above, the image processing system 1 according to the first aspect includes a first acquisition unit 11, a second acquisition unit 12, a processing unit 14, and an output unit 15. The first acquisition unit 11 acquires an image signal including image data of an original image P0. The second acquisition unit 12 acquires a selection signal for selecting a predetermined region from a plurality of regions included in the original image P0. The processing unit 14 generates a plurality of cropped images P1 by cropping each of the plurality of regions from the original image P0. The output unit 15 outputs the plurality of cropped images P1. The processing unit 14 performs distortion correction processing to correct distortion in the cropped images P1 by cropping the predetermined regions based on the selection signal.

[0082] In such an image processing system 1, distortion correction processing is performed only on cut-out images P1 that are likely to be seen by viewers, such as images to be distributed or broadcast, and this has the advantage of making it easier to reduce the processing load when outputting images compared to when distortion correction processing is performed on all cut-out images P1.

[0083] Also, for example, in the image processing system 1 relating to the second aspect, in the first aspect, the selection signal is a program image, which is an image to be distributed or broadcast, or a tally signal that designates a predetermined area as a candidate image for the program image.

[0084] In such an image processing system 1, the tally signal used in an existing switcher can be used as a selection signal, which has the advantage that there is no need to generate a selection signal separate from the tally signal, making the image processing system 1 easier to use.

[0085] Furthermore, for example, the image processing system 1 according to the third aspect further includes a third acquisition unit 13 that acquires a control signal instructing control of at least one of pan, tilt, and zoom for one of the multiple areas in the first or second aspect. The processing unit 14 generates a cut-out image P1 corresponding to one area based on the control signal.

[0086] Such an image processing system 1 has the advantage that the range of expression of the cut-out image P1 can be easily expanded compared to when the cut-out image P1 is simply generated from the original image P0.

[0087] Also, for example, in the image processing system 1 according to the fourth aspect, in any one of the first to third aspects, the processing unit 14 generates multiple cut-out images P1 at a resolution lower than the resolution of the original image P0.

[0088] Such an image processing system 1 has the advantage that the processing load when outputting an image can be further reduced compared to when a plurality of cut-out images P1 are generated at the same resolution.

[0089] Furthermore, for example, the image processing system 1 according to a fifth aspect is any one of the first to fourth aspects and further includes a camera 2. The camera 2 captures an original image P0 and transmits an image signal including image data of the captured original image P0.

[0090] In such an image processing system 1, distortion correction processing is performed only on cut-out images P1 that are likely to be seen by viewers, such as images to be distributed or broadcast, and this has the advantage of making it easier to reduce the processing load when outputting images compared to when distortion correction processing is performed on all cut-out images P1.

[0091] For example, in an image processing method according to a sixth aspect, an image signal including image data of an original image P0 is acquired (S1). A selection signal for selecting a predetermined region from among a plurality of regions included in the original image P0 is acquired (S3). A plurality of cut-out images P1 are generated by respectively cutting out a plurality of regions from the original image P0 (S5, S6). The plurality of cut-out images P1 are output (S7). A distortion correction process is performed on the cut-out images P1, each of which has a predetermined region, based on the selection signal (S5).

[0092] In such an image processing method, distortion correction processing is performed only on cut-out images P1 that are likely to be seen by viewers, such as images to be distributed or broadcast, and this has the advantage of making it easier to reduce the processing load when outputting an image compared to when distortion correction processing is performed on all cut-out images P1.

[0093] The image processing system and the like of the present disclosure can be used as a switcher or the like for switching between video images to be output.

[0094] 1 Image processing system 11 First acquisition unit 12 Second acquisition unit 13 Third acquisition unit 14 Processing unit 141 First cut-out processing unit 142 Second cut-out processing unit 15 Output unit 2 Camera 21 Optical system 22 Imaging unit 23 Image processing unit 24 Transmission unit 3 Switcher 31 Operation processing unit 32 Display unit 33 Image processing unit 34 Transmission unit 35 Reception unit 4 Display device 41 Display unit 42 Image processing unit 43 Reception unit 5 Operation terminal 51 Operation processing unit 52 Transmission unit A1 First area A2 Second area A3 Third area P0 Original image P1 Cut-out image P2 Program image P3 Candidate image

Claims

1. An image processing system comprising: a first acquisition unit that acquires an image signal including image data of an original image; a second acquisition unit that acquires a selection signal that selects a predetermined area from a plurality of areas included in the original image; a processing unit that generates a plurality of cut-out images by cutting out each of the plurality of areas from the original image; and an output unit that outputs the plurality of cut-out images, wherein the processing unit performs distortion correction processing to correct distortion in the cut-out images by cutting out the predetermined area based on the selection signal.

2. The image processing system according to claim 1, wherein the selection signal is a program image, which is an image to be distributed or broadcast, or a tally signal that designates the predetermined area as a candidate image for the program image.

3. An image processing system as described in claim 1 or 2, further comprising a third acquisition unit that acquires a control signal that instructs control of at least one of pan, tilt, and zoom for any one of the plurality of regions, and the processing unit generates a cut-out image corresponding to the one region based on the control signal.

4. The image processing system according to claim 1 or 2, wherein the processing unit generates the plurality of cut-out images at a resolution lower than that of the original image.

5. The image processing system according to claim 1 or 2, further comprising a camera that captures the original image and transmits the image signal including the image data of the captured original image.

6. An image processing method comprising: acquiring an image signal including image data of an original image; acquiring a selection signal for selecting a predetermined area from a plurality of areas included in the original image; generating a plurality of cut-out images by cutting out each of the plurality of areas from the original image; outputting the plurality of cut-out images; and performing distortion correction processing for correcting distortion in the cut-out images by cutting out the predetermined areas based on the selection signal.

Images