Image processing system and image processing method

Abstract

An image processing system (1) includes a first acquisition unit (11), a second acquisition unit (12), a processing unit (14), and an output unit (15). The first acquisition unit (11) acquires an image signal including image data of an original image. The second acquisition unit (12) acquires a selection signal indicating that a prescribed region among a plurality of regions included in the original image is selected. The processing unit (14) generates a plurality of cropped images by cropping the plurality of regions from the original image, respectively. The output unit (15) outputs the plurality of cropped images. The processing unit (14) performs distortion correction processing based on the selection signal, the distortion correction processing being processing of performing distortion correction on a cropped image obtained by cropping the prescribed region.

Description

Technical Field

[0001] This disclosure relates to image processing systems and image processing methods. Background Technology

[0002] Patent Document 1 discloses a local user device capable of performing video conferencing with a remote user device. This local user device defines a Region of Interest (ROI) within the field of view of the camera on the remote user device and receives encoded video within the defined ROI from the remote user device.

[0003] Patent Document 2 discloses an image processing apparatus. This image processing apparatus generates an image after correcting for distortions contained in the image captured by the camera unit.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent Document 1: Japanese Patent Application Publication No. 2021-52415

[0007] Patent Document 2: International Publication No. 2017 / 179722 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] This disclosure provides an image processing system, etc., that easily reduces the processing load when outputting images.

[0010] Methods for solving problems

[0011] One aspect of the image processing system disclosed herein includes a first acquisition unit, a second acquisition unit, a processing unit, and an output unit. The first acquisition unit acquires an image signal containing image data of an original image. The second acquisition unit acquires a selection signal, which indicates that a predetermined region among a plurality of regions contained in the original image is selected. The processing unit extracts the plurality of regions from the original image to generate a plurality of cropped images. The output unit outputs the plurality of cropped images. The processing unit performs distortion correction processing based on the selection signal; the distortion correction processing is a process of correcting the distortion of the cropped images obtained by extracting the predetermined regions.

[0012] Invention Effects

[0013] This disclosure has the advantage of easily reducing the processing load when outputting images. Attached Figure Description

[0014] Figure 1This is a block diagram illustrating the overall configuration of the image processing system involved in the embodiments.

[0015] Figure 2 This is an explanatory diagram of the distortion correction processing performed by the image processing system involved in the embodiment.

[0016] Figure 3 This is a flowchart illustrating an example of the operation of an image processing system according to an embodiment.

[0017] Figure 4 This is an explanatory diagram illustrating an example of the use of the image processing system involved in the implementation method. Detailed Implementation

[0018] [1. The knowledge that forms the basis of this disclosure]

[0019] First, the inventors' focus will be explained as follows.

[0020] For example, the technique for extracting a region of interest (ROI) from a top-down image disclosed in Patent Document 1 can extract images from multiple different angles from an image captured by a single camera, thus being effective for cost-saving shooting. However, the technique disclosed in Patent Document 1 has the following problem: when extracting an image from an image obtained by shooting a subject at a wide angle, the extracted image will suffer from wide-angle distortion, making the extracted image appear unnatural to the viewer.

[0021] Patent Document 2 discloses a technique that performs distortion correction processing on a captured image based on information such as the camera's focal length, thereby converting the captured image into an image that appears natural to the viewer. However, the distortion correction processing disclosed in Patent Document 2 is a computationally intensive process that requires calculations for each pixel. Therefore, when using the technique disclosed in Patent Document 2, the following problem exists: for example, when capturing images from multiple different angles from a top-down view, the number of images that can be captured is limited.

[0022] In view of the above, the inventors developed this disclosure.

[0023] The following description refers to the accompanying drawings. Furthermore, the embodiments described below are general or specific examples. The numerical values, shapes, materials, constituent elements, arrangement positions of constituent elements, connection methods, steps, and order of steps shown in the following embodiments are all examples and are not intended to limit this disclosure. In addition, constituent elements not shown as independent technical solutions in the following embodiments are described as arbitrary constituent elements.

[0024] Furthermore, the diagrams are schematic and may not be strictly illustrated. Additionally, in each diagram, substantially identical components are given the same symbols, and repetitive explanations are omitted or simplified.

[0025] Furthermore, in this specification, ordinal numbers such as "first" and "second" do not imply the quantity or order of constituent elements unless otherwise specified, but are used to distinguish them from each other to avoid confusion of the same constituent elements.

[0026] (Implementation Method)

[0027] [2. Composition]

[0028] The overall configuration of the image processing system 1, which includes the embodiments, will be described below. Figure 1 This is a block diagram illustrating the overall configuration of the image processing system 1 according to the embodiments. The image processing system 1 processes the acquired image (hereinafter also referred to as "raw image P0") (see reference). Figure 2 The image is generated by cropping a portion of the image (hereinafter also referred to as "cropped image P1"). Figure 2 The system generates and outputs the cropped image. Both the original image P0 and the cropped image P1 can be either animated or still images.

[0029] In this embodiment, the image processing system 1 extracts a cropped image P1 from the original image P0 captured by the camera 2, specifying a region as defined by the operation terminal 5, and outputs the cropped image P1 to the switcher 3 and the display device 4. Furthermore, in this embodiment, there are multiple first operators, each operating the operation terminal 5 while viewing the display device 4. Therefore, in this embodiment, there are multiple display devices 4 and multiple operation terminals 5, with the number of display devices 4 and operation terminals 5 corresponding to the number of first operators. Additionally, the role of the first operators is equivalent to that of a cameraman in a traditional video production or broadcasting environment.

[0030] Camera 2 is a camera device capable of capturing moving or still images. In this embodiment, camera 2 is capable of capturing moving or still images with high resolution, such as 6K or 8K. Furthermore, in this embodiment, camera 2 is installed, for example, in a video production or broadcasting location, by fixing it to a pedestal, wall, or ceiling. Camera 2 includes an optical system 21, a camera unit 22, an image processing unit 23, and a transmission unit 24.

[0031] The optical system 21 includes, for example, a focusing lens and a zoom lens. The focusing lens is composed of a combination of one or more lenses. Similarly, the zoom lens is composed of a combination of one or more lenses.

[0032] The camera unit 22 generates image data based on optical information input via the optical system 21. The camera unit 22 includes an image sensor that converts optical information into electrical signals (analog signals) and an A / D converter that converts analog signals into digital signals (image signals). The image sensor is, for example, a CCD (Charge-Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor.

[0033] The image processing unit 23 generates image data by performing appropriate processing on the image signal obtained by the imaging unit 22. For example, the image processing unit 23 generates image data by encoding the image signal in a prescribed format. The image data may include metadata, such as information indicating the shooting conditions at the time of shooting. Shooting conditions may include, for example, the focal length, exposure, shutter speed, or angle of view of the camera 2.

[0034] The transmitting unit 24 is a communication interface used to communicate with the image processing system 1 via a network such as the Internet or a LAN (Local Area Network), SDI (Serial Digital Interface) transmission, or IP (Internet Protocol) transmission of the ST2110. Communication between the transmitting unit 24 and the image processing system 1 can be wired or wireless. The transmitting unit 24 transmits image data generated by the image processing unit 23 to the image processing system 1 via a network, SDI transmission, or IP transmission of the ST2110.

[0035] The switcher 3 is a device for switching broadcast or distributed images. In one embodiment, the switcher 3 is implemented by installing dedicated software on a general-purpose information terminal, such as a server device or a desktop or laptop personal computer. Alternatively, the switcher 3 can also be implemented using dedicated hardware. The switcher 3 includes an operation processing unit 31, a display unit 32, an image processing unit 33, a transmission unit 34, and a receiving unit 35.

[0036] The operation processing unit 31 performs processing of the operation input from the second operator to the operation switcher 3. For example, when the operation processing unit 31 receives the second operator's selection of program image P2 (see reference 3), it performs processing. Figure 4 Upon receiving input from the second operator, a selection signal indicating that program image P2 has been selected is generated. Furthermore, for example, the operation processing unit 31, upon receiving candidate image P3 (see reference) of program image P2 selected by the second operator, generates a selection signal indicating that program image P2 has been selected. Figure 4 After inputting ), a selection signal is generated indicating that the candidate image P3 has been selected.

[0037] Here, program image P2 refers to the image to be broadcast or distributed. Furthermore, candidate image P3 refers to the image to be broadcast or distributed after program image P2. Therefore, by having the second operator sequentially select program image P2 and candidate image P3, program image P2 is broadcast or distributed sequentially.

[0038] In this implementation, both program image P2 and candidate image P3 are any one of the multiple regions contained in the original image P0, or in other words, any one of the multiple cropped images P1 extracted from the original image P0. That is, the selection signal indicating that program image P2 (or candidate image P3) has been selected is equivalent to a signal indicating that a specified region within the multiple regions contained in the original image P0 has been selected.

[0039] In this implementation, the selection signal is called a tally signal. Conventionally, a tally signal is a signal sent to the camera to illuminate a tally light or other indicator light on the camera in a specified color. For example, when the camera receives a red tally signal, the cameraman operating the camera can know that the image captured by his camera will be used as program image P2. Furthermore, for example, when the camera receives a green tally signal, the cameraman operating the camera can know that the image captured by his camera is candidate image P3, which will be used next as program image P2.

[0040] Therefore, in this embodiment, after receiving input from the second operator selecting program image P2, the operation processing unit 31 generates a red tally signal as a selection signal. Furthermore, after receiving input from the second operator selecting candidate image P3, the operation processing unit 31 generates a green tally signal.

[0041] The display unit 32 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, which displays the original image P0 received by the receiving unit 35. Furthermore, when the receiving unit 35 receives multiple cropped images P1, the display unit 32 displays multiple cropped images P1. Alternatively, the display unit 32 can display the original image P0 and the multiple cropped images P1 simultaneously, or it can switch between displaying any one of the original image P0 and the multiple cropped images P1 based on input from a second operator.

[0042] The image processing unit 33 decodes the image data received by the receiving unit 35. The image obtained by decoding the image data (original image P0 or multiple cropped images P1) is displayed on the display unit 32.

[0043] The transmitting unit 34 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or LAN, SDI transmission, or IP transmission of the ST2110. Communication between the transmitting unit 34 and the image processing system 1 can be wired or wireless. The transmitting unit 34 transmits the selection signal generated by the operation processing unit 31 to the image processing system 1 via the network.

[0044] The receiving unit 35 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or LAN, SDI transmission, or IP transmission of the ST2110. Communication between the receiving unit 35 and the image processing system 1 can be wired or wireless. The receiving unit 35 receives image data of the original image P0 output from the image processing system 1, as well as image data of multiple cropped images P1, via a network, SDI transmission, or IP transmission of the ST2110.

[0045] The display device 4 is a device for a first operator to confirm the image (original image P0) captured by the camera 2. The display device 4 includes a display unit 41, an image processing unit 42, and a receiving unit 43.

[0046] The display unit 41 is, for example, a liquid crystal display or an organic EL display, which displays the original image P0 received by the receiving unit 35.

[0047] The image processing unit 42 decodes the image data received by the receiving unit 43. The image obtained by decoding the image data (original image P0) is displayed on the display unit 41.

[0048] The receiving unit 43 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or LAN, SDI transmission, or IP transmission of the ST2110. Communication between the receiving unit 43 and the image processing system 1 can be wired or wireless. The receiving unit 43 receives image data of the raw image P0 output from the image processing system 1 via a network, SDI transmission, or IP transmission of the ST2110.

[0049] The operation terminal 5 is a terminal used to perform operations required to generate the cropped image P1, such as the first operator specifying a region to be cropped from the image (original image P0) captured by the camera 2. The operation terminal 5 may include a pointing device such as a mouse, or a joystick, etc. The operation terminal 5 includes an operation processing unit 51 and a transmission unit 52.

[0050] The operation processing unit 51 performs processing on the operation input of the first operator. For example, after receiving input specifying a region to be cropped from the original image P0, the operation processing unit 51 generates a control signal containing information representing the specified region in the original image P0. Furthermore, for example, after receiving input instructing at least one of pan, tilt, and zoom control for the specified region, the operation processing unit 51 generates a control signal further containing information representing the instruction.

[0051] The transmitting unit 52 is a communication interface for communicating with the image processing system 1 via a network such as the Internet or a LAN. Communication between the transmitting unit 52 and the image processing system 1 can be wired or wireless. The transmitting unit 52 transmits control signals generated by the operation processing unit 51 to the image processing system 1 via the network.

[0052] The image processing system 1 is implemented using an image processing unit, for example, equipped with processing circuitry such as an FPGA (Field-Programmable Gate Array). Alternatively, the image processing system 1 can also be implemented using an information processing device different from the image processing unit, such as a personal computer. The image processing system 1 includes a first acquisition unit 11, a second acquisition unit 12, a third acquisition unit 13, a processing unit 14, and an output unit 15.

[0053] The first acquisition unit 11 is a communication interface for communicating with the camera 2 via a network such as the Internet or LAN, SDI transmission, or IP transmission of the ST2110. Communication between the first acquisition unit 11 and the camera 2 can be wired or wireless. The first acquisition unit 11 receives image data sent from the camera 2 via a network, SDI transmission, or IP transmission of the ST2110. In other words, the first acquisition unit 11 acquires an image signal containing image data of the original image P0.

[0054] The second acquisition unit 12 is a communication interface for communicating with the switch 3 via, for example, a network such as the Internet or LAN, SDI transmission, or IP transmission of the ST2110. The communication between the second acquisition unit 12 and the switch 3 can be wired or wireless. The second acquisition unit 12 receives (acquires) selection signals sent from the switch 3 via a network, SDI transmission, or IP transmission of the ST2110.

[0055] The third acquisition unit 13 is a communication interface for communicating with each of the multiple operating terminals 5 via a network such as the Internet or a LAN. The communication between the third acquisition unit 13 and each of the multiple operating terminals 5 can be wired or wireless. The third acquisition unit 13 receives (acquires) control signals sent from each of the multiple operating terminals 5 via the network.

[0056] The processing unit 14 decodes the image data of the original image P0. Furthermore, the processing unit 14 extracts multiple regions from the original image P0 to generate multiple cropped images P1. Specifically, the processing unit 14 generates multiple cropped images P1 from the original image P0 based on each of the multiple control signals obtained from each of the multiple operation terminals 5. For example, assume there are 3 operation terminals 5 (i.e., there are 3 first operators). Here, the 3 operation terminals 5 are referred to as the "first operation terminal," the "second operation terminal," and the "third operation terminal," respectively. Furthermore, the control signals sent by the "first operation terminal," the "second operation terminal," and the "third operation terminal" are referred to as the "first control signal," the "second control signal," and the "third control signal," respectively.

[0057] In the above examples, processing unit 14 generates a "first cropped image" by cropping the region specified by the "first control signal" sent from the "first operation terminal" from the original image P0. Furthermore, processing unit 14 generates a "second cropped image" by cropping the region specified by the "second control signal" sent from the "second operation terminal" from the original image P0. Additionally, processing unit 14 generates a "third cropped image" by cropping the region specified by the "third control signal" sent from the "third operation terminal" from the original image P0.

[0058] In this embodiment, the processing unit 14 generates multiple cropped images P1 at a resolution lower than that of the original image P0. For example, if the resolution of the original image P0 is 8K, the processing unit 14 generates multiple cropped images P1 at a resolution of 2K.

[0059] Here, for each of the plurality of cropped images P1, the processing unit 14 uses either a first cropping processing unit 141 or a second cropping processing unit 142, which is determined based on a selection signal, to generate a cropped image P1. In the embodiment, both the first cropping processing unit 141 and the second cropping processing unit 142 are functions that the processing unit 14 can perform.

[0060] The first cropping processing unit 141 generates a cropped image P1 by cropping a region specified based on a control signal from the original image P0. The first cropping processing unit 141 does not perform distortion correction processing on the cropped image P1.

[0061] The second cropping processing unit 142, like the first cropping processing unit 141, crops a region specified based on a control signal from the original image P0. The second cropping processing unit 142 performs distortion correction processing on the cropped image P1.

[0062] Here, utilizing Figure 2 The distortion correction process is explained. Figure 2 This is an explanatory diagram of the distortion correction processing performed by the image processing system 1 according to the embodiment. Figure 2 This refers to images obtained by filming a football match using camera 2. Additionally, in Figure 2 The illustrations of the stands and goals are omitted. Figure 2 (a) represents the original image P0 captured by camera 2. Figure 2 (b) represents the cropped image P1 obtained by extracting the specified region from the original image P0. Furthermore, Figure 2 (c) indicates that Figure 2 The image shown in (b) is the cropped image P1 after distortion correction.

[0063] For example Figure 2 As shown in (b), when shooting a subject with a wide-angle lens, distortion typically increases with distance from the center of the image. This distortion will be referred to as "wide-angle distortion" below. Wide-angle distortion refers to distortion where the degree of distortion increases with distance from the center of the image; specifically, it is a distortion where the width of the image is expanded in both the horizontal and vertical directions. Therefore, in this embodiment, distortion correction processing is performed as follows to correct the distortion. Figure 2 The cropped image P1 with wide-angle distortion shown in (b) is corrected as follows: Figure 2 The cropped image P1 shown in (c) has wide-angle distortion eliminated.

[0064] In the distortion correction process, the processing unit 14 sets a sphere centered on the optical center (here, the optical center of the focusing lens of the optical system 21), and maps each point (pixel) on the mapping surface corresponding to the image before correction onto this sphere. Next, the processing unit 14 sets a correction mapping surface parallel to the mapping surface, and maps each pixel mapped onto the sphere onto this correction mapping surface. Then, the processing unit 14 maps each pixel mapped onto the correction mapping surface back onto the original mapping surface to restore it to the same size as the original mapping surface. Here, by determining the above series of mappings, the correspondence between the positions of each pixel in the image before correction and the positions of each pixel in the image after correction is calculated in advance, and this correspondence is stored, for example, in a memory referenced by the processing unit 14. Therefore, by using the above correspondence, the processing unit 14 can determine each pixel in the image before correction that corresponds to each pixel in the image after correction, and thus can correct wide-angle distortion in the image.

[0065] Based on the selection signal, the processing unit 14 designates the region (prescribed region) within the multiple regions contained in the original image P0 that is designated as either the program image P2 or the candidate image P3 as the "selection target region," and designates the remaining regions as "regions outside the selection target." Then, when the selection target region is captured, the processing unit 14 uses the second capture processing unit 142 to generate a captured image P1; when the region outside the selection target is captured, the processing unit 14 uses the first capture processing unit 141 to generate a captured image P1. In other words, the processing unit 14 performs distortion correction processing on the selection target region and does not perform distortion correction processing on the region outside the selection target. Thus, the processing unit 14 performs distortion correction processing on the captured image P1 obtained by capturing the prescribed region (the selection target region) based on the selection signal.

[0066] Furthermore, if the control signal includes information indicating control of at least one of panning, tilting, and zooming, the processing unit 14 generates a cropped image P1 based on that information. For example, if the control signal includes information indicating panning control, the processing unit 14 can generate the cropped image P1 while continuously changing the cropping position, thereby obtaining an image equivalent to the image obtained when panning control is performed on a conventional camera positioned at the same position as the camera 2 and outputting an image of the same size as the cropped image P1.

[0067] Output unit 15 outputs the original image P0 captured by camera 2 and the cropped image P1 generated by processing unit 14. Specifically, output unit 15 outputs the image obtained by encoding the original image P0 captured by camera 2 to switcher 3 and each display device 4. Furthermore, output unit 15 outputs the image obtained by encoding the cropped image P1 generated by processing unit 14 to switcher 3 and the corresponding display device 4. Here, "corresponding display device 4" refers to the display device 4 used by the first operator who generated the cropped image P1. In addition, output unit 15 distributes or broadcasts the image (program image P2) selected by switcher 3.

[0068] [3. Actions]

[0069] The operation of the image processing system 1 according to the embodiments, i.e. the image processing method according to the embodiments, will be described below. Figure 3 This is a flowchart illustrating an example of the operation of the image processing system 1 according to the embodiment.

[0070] First, the image processing system 1 acquires an image signal (S1) containing image data of the original image P0 sent from the camera 2. Although not shown here, after acquiring the image signal, the image processing system 1 sends the image data of the original image P0 to the switcher 3 and each of the plurality of display devices 4. As a result, the original image P0 is displayed on the display unit 32 of the switcher 3 and on the display units 41 of each of the plurality of display devices 4.

[0071] Next, the image processing system 1 acquires multiple control signals (S2) sent from each of the multiple operation terminals 5. Although not shown here, the image processing system 1 generates multiple cropped images P1 by cropping a region specified by each of the multiple control signals from the original image P0, and sends the image data of the generated multiple cropped images P1 to the switcher 3. As a result, the multiple cropped images P1 are displayed on the display unit 32 of the switcher 3.

[0072] Next, the image processing system 1 acquires the selection signal sent from the switcher 3 (S3). In the embodiment, as described above, the image processing system 1 acquires the red tally signal and the green tally signal, which are both selection signals.

[0073] Next, the image processing system 1 determines whether to perform distortion correction processing for each of the multiple regions represented by a plurality of control signals, based on a selection signal. If the region is the selected region (the specified region) ("Yes" in S4), the image processing system 1 generates a cropped image P1 for which distortion correction processing has been performed (S5). Conversely, if the region is outside the selected region ("No" in S4), the image processing system 1 generates a cropped image P1 for which distortion correction processing has not been performed (S6).

[0074] Then, the image processing system 1 outputs the cropped image P1 generated in step S5 or S6 (S7). Here, the image processing system 1 sends the generated cropped image P1 to the display device 4 of the first operator who has cropped the aforementioned area. Furthermore, if the generated cropped image P1 is a program image P2, the image processing system 1 distributes or broadcasts the cropped image P1.

[0075] Next, if there are other unprocessed areas ("Yes" in S8), image processing system 1 repeats steps S4 to S7. Then, if there are no other unprocessed areas ("No" in S8), image processing system 1 terminates the process.

[0076] [4. Usage Examples]

[0077] Hereinafter, an example of using the image processing system 1 according to the embodiment will be described. Figure 4 These are explanatory diagrams illustrating an example of using the image processing system 1 according to the embodiment. Additionally, in Figure 4 In the example shown, although wide-angle distortion is not shown in all images, it actually occurs in more than one of all images.

[0078] exist Figure 4 The example shown illustrates a situation where three operators are simultaneously viewing display device 4 and operating operation terminal 5. Furthermore, in Figure 4 The example shown illustrates a second operator using switcher 3 to select program image P2 and candidate image P3. Figure 4 The top section represents the scene captured by camera 2. Figure 4 The second paragraph from the top represents the original image P0. Furthermore, Figure 4 The third paragraph from the top shows the image displayed on the display unit 32 of the switcher 3. Figure 4 The bottom section represents the image displayed on the display section 41 of any display device 4 (in this case, program image P2). Furthermore, in Figure 4 In the example shown, time is set according to Figure 4 of (a) Figure 4 (b) Figure 4 (c) and Figure 4 The sequence of (d) is followed.

[0079] The following is based on Figure 4 (a) provides a specific example of using image processing system 1. Additionally, in Figure 4 of (b) Figure 4 (c) and Figure 4 In (d), only the moment and Figure 4 The difference is that (a) is different, while image processing system 1 performs the same action, therefore the explanation is omitted here. Figure 4 As shown in (a), the three first operators respectively designated a first region A1, a second region A2, and a third region A3 in the original image P0. Therefore, the image processing system 1 acquires a control signal containing information specifying the first region A1, a control signal containing information specifying the second region A2, and a control signal containing information specifying the third region A3. Then, the image processing system 1 generates a cropped image P1 obtained by cropping the first region A1, a cropped image P1 obtained by cropping the second region A2, and a cropped image P1 obtained by cropping the third region A3, and sends the image data of these cropped images P1 to the switcher 3.

[0080] Multiple captured images P1 (here, three captured images P1) are displayed on the display unit 32 of the switcher 3. The second operator selects a program image P2 and a candidate image P3 while viewing the multiple captured images P1 displayed on the display unit 32. Figure 4 In the example shown, the cropped image P1 obtained by cropping the first region A1 enclosed by a solid line frame represents the program image P2, and the cropped image P1 obtained by cropping the second region A2 enclosed by a dashed line frame represents the candidate image P3. Furthermore, the cropped image P1 obtained by cropping the third region A3 enclosed by a dotted line frame is neither the program image P2 nor the candidate image P3.

[0081] Switcher 3 sends a red tally signal to image processing system 1 with the first operator's display device 4, which captures the first area A1, as the destination. Additionally, switcher 3 sends a green tally signal to image processing system 1 with the first operator's display device 4, which captures the second area A2, as the destination.

[0082] After acquiring the red tally signal, which serves as a selection signal, the image processing system 1 generates a cropped image P1 that performs distortion correction processing on the selected area (here, the first area A1), and sends the image data of the generated cropped image P1 to the display device 4 of the first operator who has cropped the first area A1. Furthermore, after acquiring the green tally signal, which also serves as a selection signal, the image processing system 1 generates a cropped image P1 that performs distortion correction processing on the selected area (here, the second area A2), and sends the image data of the generated cropped image P1 to the display device 4 of the first operator who has cropped the second area A2. Additionally, the image processing system 1 generates a cropped image P1 without performing distortion correction processing on the area other than the selected area (here, the third area A3), and sends the image data of the generated cropped image P1 to the display device 4 of the first operator who has cropped the third area A3.

[0083] [5. Advantages, etc.]

[0084] The advantages of the image processing system 1 (image processing method) according to the embodiments will be explained below. As described above, in the image processing system 1 according to the embodiments, distortion correction processing is performed on the cropped image P1 obtained by cropping a predetermined region from multiple regions contained in the original image P0 based on a selection signal. Therefore, in the image processing system 1 according to the embodiments, distortion correction processing is performed only on the cropped image P1 that a viewer may view, such as an image that is being distributed or broadcast. Therefore, compared with the case where distortion correction processing is performed on all cropped images P1, it has the advantage of easily reducing the processing load when outputting the image.

[0085] That is, in the image processing system 1 according to the embodiment, distortion correction processing is performed only on the necessary images when necessary, so it is difficult to produce problems such as the limitation on the number of images that can be captured when cropping multiple images from different shooting angles from a top-view image as described in [1. Knowledge that forms the basis of this disclosure].

[0086] [6. Other implementation methods]

[0087] The implementation methods have been described above, but this disclosure is not limited to the above-described implementation methods.

[0088] For example, in the above embodiment, the processing unit 14 performs distortion correction processing using mapping, but it is not limited to this. That is, the processing unit 14 can perform distortion correction processing on the cropped image P1 obtained by cropping a specified region based on the selection signal, and the distortion correction processing method can be other than mapping.

[0089] For example, in the above embodiment, the image processing system 1 is implemented by a device different from the switcher 3, such as an image processing unit, but it is not limited to this. For example, the image processing system 1 may also be mounted in the switcher 3.

[0090] For example, in the above embodiment, the image processing system 1 is implemented by a single device, but it is not limited to this. For example, the image processing system 1 may also be implemented by multiple devices.

[0091] Furthermore, the processes executed by a specific processing unit in the above embodiments can also be executed by other processing units. Moreover, the order of multiple processes can be changed, and multiple processes can be executed in parallel.

[0092] Furthermore, in the above embodiments, each component can also be implemented by executing a software program suitable for that component. Each component can also be implemented by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory.

[0093] Furthermore, each component can also be implemented in hardware. Each component can be a circuit (or integrated circuit). These circuits can be constructed as a single circuit or as separate circuits. Moreover, these circuits can be either general-purpose circuits or special-purpose circuits.

[0094] Furthermore, the general or specific embodiments of this disclosure can be implemented by a system, apparatus, method, integrated circuit, computer program, or a recording medium such as a computer-readable CD-ROM. Moreover, they can also be implemented by any combination of systems, apparatus, methods, integrated circuits, computer programs, and recording media.

[0095] Furthermore, this disclosure can also be implemented as an image processing method executed by a computer, such as the image processing system described above. This disclosure can also be implemented as a program (computer program product) for causing a computer to execute such an image processing method, or as a computer-readable, non-transitory recording medium on which such a program is recorded.

[0096] Other forms, such as those derived by performing various modifications to the embodiments as would be conceived by those skilled in the art, or forms achieved by arbitrarily combining the constituent elements and functions of the embodiments without departing from the scope of this disclosure, are all included in this disclosure.

[0097] (Summarize)

[0098] As explained above, the image processing system 1 according to the first method includes a first acquisition unit 11, a second acquisition unit 12, a processing unit 14, and an output unit 15. The first acquisition unit 11 acquires an image signal containing image data of an original image P0. The second acquisition unit 12 acquires a selection signal, which indicates that a predetermined region among the multiple regions contained in the original image P0 is selected. The processing unit 14 extracts multiple regions from the original image P0 to generate multiple cropped images P1. The output unit 15 outputs the multiple cropped images P1. The processing unit 14 performs distortion correction processing based on the selection signal; the distortion correction processing is the process of correcting the distortion of the cropped images P1 obtained by extracting the predetermined regions.

[0099] In such an image processing system 1, distortion correction processing is performed only on the cropped image P1 that the viewer may see, such as the image being distributed or broadcast. Therefore, compared to performing distortion correction processing on all cropped images P1, it has the advantage of easily reducing the processing load when outputting the image.

[0100] Furthermore, for example, in the image processing system 1 involved in the second method, according to the first method, the selection signal is a tally signal that designates a specified area as an image, i.e., a program image, or a candidate image of a program image, for distribution or broadcast.

[0101] In such an image processing system 1, the tally signal used in an existing switcher can be used as a selection signal, so there is no need to generate a selection signal different from the tally signal, which has the advantage of making the image processing system 1 easy to use.

[0102] Furthermore, for example, the image processing system 1 according to the third embodiment, based on the first or second embodiment, further includes a third acquisition unit 13 for acquiring control signals. These control signals are signals instructing at least one of panning, tilting, and zooming on any one of multiple regions. The processing unit 14 generates a cropped image P1 corresponding to a given region based on the control signals.

[0103] In such an image processing system 1, compared to simply generating a cropped image P1 from the original image P0, it has the advantage of easily expanding the representation range of the cropped image P1.

[0104] Furthermore, for example, in the image processing system 1 involved in the fourth method, according to any one of the first to third methods, the processing unit 14 generates a plurality of cropped images P1 at a resolution lower than that of the original image P0.

[0105] In such an image processing system 1, compared with the case of generating multiple cropped images P1 at the same resolution, it has the advantage of easily reducing the processing load when outputting the image.

[0106] Furthermore, for example, in the image processing system 1 according to the fifth embodiment, the image processing system 1 also includes a camera 2 according to any one of the first to fourth embodiments. The camera 2 captures the original image P0 and sends an image signal containing the image data of the captured original image P0.

[0107] In such an image processing system 1, distortion correction processing is performed only on the cropped image P1 that the viewer may see, such as the image being distributed or broadcast. Therefore, compared to performing distortion correction processing on all cropped images P1, it has the advantage of easily reducing the processing load when outputting the image.

[0108] Furthermore, for example, in the image processing method according to the sixth method, an image signal containing image data of the original image P0 is acquired (S1). Furthermore, in this image processing method, a selection signal indicating that a predetermined region is selected from among the multiple regions contained in the original image P0 is acquired (S3). Furthermore, in this image processing method, multiple regions are respectively cropped from the original image P0 to generate multiple cropped images P1 (S5, S6). Furthermore, in this image processing method, multiple cropped images P1 are output (S7). Furthermore, in this image processing method, distortion correction processing is performed based on the selection signal; the distortion correction processing is a process of correcting the distortion of the cropped images P1 obtained by cropping the predetermined regions (S5).

[0109] In such an image processing method, distortion correction is performed only on the cropped image P1 that the viewer may see, such as the image being distributed or broadcast. Therefore, compared to performing distortion correction on all cropped images P1, it has the advantage of easily reducing the processing load when outputting the image.

[0110] Industrial availability

[0111] The image processing system disclosed herein can be used with a switcher, which is a device for switching output images.

[0112] Explanation of reference numerals in the attached figures

[0113] 1 Image Processing System

[0114] 11 First Acquisition Department

[0115] 12 Second Acquisition Department

[0116] 13 Third Acquisition Department

[0117] 14 Processing Department

[0118] 141 First Interception Processing Unit

[0119] 142 Second Interception Processing Unit

[0120] 15 Output Section

[0121] 2 cameras

[0122] 21 Optical System

[0123] 22 Camera Department

[0124] 23 Image Processing Department

[0125] 24. Sending Department

[0126] 3 Switch

[0127] 31 Operations Processing Department

[0128] 32 Display Section

[0129] 33 Image Processing Department

[0130] 34. Sending Department

[0131] 35 Receiving Section

[0132] 4 Display devices

[0133] 41 Display Section

[0134] 42 Image Processing Department

[0135] 43 Receiving Department

[0136] 5. Operating Terminal

[0137] 51 Operations Processing Department

[0138] 52. Sending Department

[0139] A1 First Area

[0140] A2 Second Area

[0141] A3 Third Area

[0142] P0 Original Image

[0143] P1 Capture Image

[0144] P2 program image

[0145] P3 candidate image.

Claims

1. An image processing system, comprising: The first acquisition unit acquires an image signal containing image data of the original image; The second acquisition unit acquires a selection signal, which is a signal indicating that a specified region among the multiple regions contained in the original image is selected; The processing unit extracts the multiple regions from the original image to generate multiple cropped images; as well as The output unit outputs the multiple cropped images. The processing unit performs distortion correction processing based on the selection signal. The distortion correction processing is a process of correcting the distortion of a cropped image obtained by cropping the specified region.

2. The image processing system as described in claim 1, The selection signal is a tally signal that designates the specified area as an image, i.e., a program image, or a candidate image of the program image for distribution or broadcast.

3. The image processing system as described in claim 1 or 2, The image processing system further includes a third acquisition unit for acquiring control signals, wherein the control signals are signals that instruct at least one of panning, tilting, and zooming on any one of the plurality of regions. The processing unit generates a cropped image corresponding to the region based on the control signal.

4. The image processing system as described in claim 1 or 2, The processing unit generates the plurality of cropped images at a resolution lower than that of the original image.

5. The image processing system as described in claim 1 or 2, The image processing system also includes a camera, which captures the original image and sends the image signal containing the image data of the captured original image.

6. An image processing method, wherein in the image processing method, Acquire the image signal containing the image data of the original image. Acquire a selection signal, which is a signal indicating that a specified region is selected from among multiple regions contained in the original image. Multiple cropped images are generated by cropping the multiple regions from the original image. Output the multiple cropped images. Distortion correction processing is performed based on the selection signal. The distortion correction processing is a process of correcting the distortion of the cropped image obtained by cropping the specified region.

Images