Image controller, image processing system, and image correction method

Abstract

Embodiments of the present application provide an image controller, an image processing system and an image correction method. A first controller obtains a first image from an image capturing device. The first controller converts the first image into a second image according to a conversion job. The conversion job includes a distortion correction, and the distortion correction is used to correct a deformation of one or more objects in the first image. A second controller detects the objects in the second image to generate a detection result. The first controller corrects the conversion job according to the detection result. In this way, the visual experience can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to an image processing technique, and in particular, to an image controller, an image processing system and an image correction method. BACKGROUND

[0002] In the prior art, although a camera equipped with a wide-angle lens or a fisheye lens can capture an image with a wide field of view (FoV), the image edge can be curved and form an unnatural appearance. The distortion of the wide-angle or fisheye image can make its content difficult to identify and can also make the user's eyes feel uncomfortable.

[0003] On the other hand, such cameras are usually equipped in products such as rearview mirrors, IP cameras, surveillance systems, Internet of Things cameras and machine vision related products. In some application scenarios, the objects in the image are the targets that the viewer wants to track. However, there can be more than one object in the image and the objects can move, but such products usually cannot provide appropriate images in response to the movement or number of objects. SUMMARY

[0004] The present application is directed to an image controller, an image processing system and an image correction method, which can simply and effectively correct a distorted image and can improve the identification of a specific tracking target in the image.

[0005] According to an embodiment of the present application, an image correction method includes (but is not limited to) the following steps: obtaining a first image from an image capturing device. Converting the first image into a second image according to a conversion operation. The conversion operation includes a distortion correction, and the distortion correction is used to correct the deformation of one or more target objects in the first image. Detecting the target objects in the second image to generate a detection result. Correcting the conversion operation according to the detection result.

[0006] According to an embodiment of the present application, an image processing system includes (but is not limited to) an image capturing device, a first controller and a second controller. The image capturing device includes a lens and an image sensor. A first image is captured through the lens and the image sensor. The first controller is coupled to the image capturing device and is used to convert the first image into a second image according to a conversion operation. The conversion operation includes a distortion correction. The distortion correction is used to correct the deformation of one or more target objects in the first image. The second controller is coupled to the first controller and is used to detect those target objects in the second image to generate a detection result. The first controller is also used to correct the conversion operation according to the detection result.

[0007] The image controller of this invention includes (but is not limited to) a memory and a processor. The memory stores program code. The processor is coupled to the memory. The processor is configured to load and execute the program code to obtain a first image, convert the first image into a second image according to a conversion job, detect one or more targets in the second image to generate detection results, and correct the conversion job based on the detection results. The conversion job includes deformation correction, and the deformation correction is used to correct the deformation of one or more targets in the first image. Attached Figure Description

[0008] The accompanying drawings are included to further illustrate the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0009] Figure 1 This is a block diagram of components of an image processing system according to an embodiment of the present invention;

[0010] Figure 2 This is a flowchart of an image correction method according to an embodiment of the present invention;

[0011] Figure 3A This is a schematic diagram of dewarp according to an embodiment of the present invention;

[0012] Figure 3B This is a schematic diagram of viewing angle adjustment according to an embodiment of the present invention;

[0013] Figure 3C This is a schematic diagram of zoom adjustment according to an embodiment of the present invention;

[0014] Figure 3D This is a schematic diagram of a shift according to an embodiment of the present invention;

[0015] Figure 3E This is a schematic diagram illustrating the adjustment of the vertical viewing angle according to an embodiment of the present invention;

[0016] Figure 3F This is a schematic diagram illustrating the adjustment of the left and right viewing angles according to an embodiment of the present invention;

[0017] Figure 3G This is a schematic diagram of planar viewing angle adjustment according to an embodiment of the present invention;

[0018] Figure 3H This is a schematic diagram of the arrangement of an image capture device and the captured image according to an embodiment of the present invention;

[0019] Figure 3I This is a schematic diagram of the arrangement of an image capture device and the captured image according to an embodiment of the present invention;

[0020] Figure 3J FIG. 1 is a schematic diagram of an image capture device and a captured image according to an embodiment of the present application;

[0021] Figure 4 FIG. 2 is a schematic diagram of fisheye image unwrapping according to an embodiment of the present application;

[0022] Figure 5 FIG. 3 is a schematic diagram of target arrangement according to an embodiment of the present application;

[0023] Figure 6 FIG. 4 is a schematic diagram of target arrangement for multiple modes according to an embodiment of the present application;

[0024] Figure 7A FIG. 5 is a schematic diagram of target arrangement for a mode according to an embodiment of the present application;

[0025] Figure 7B FIG. 6 is a schematic diagram of target arrangement for a mode according to an embodiment of the present application;

[0026] Figure 7C FIG. 7 is a schematic diagram of target arrangement for a mode according to an embodiment of the present application;

[0027] Figure 8 FIG. 8 is a schematic diagram of coordinate system conversion for a mode according to an embodiment of the present application;

[0028] Figure 9 FIG. 9 is a schematic diagram of a second image under a viewing angle according to an embodiment of the present application;

[0029] Figure 10 FIG. 10 is a schematic diagram of a second image under a rotated viewing angle according to an embodiment of the present application;

[0030] Figure 11 FIG. 11 is a schematic diagram of a corrected second image according to an embodiment of the present application;

[0031] Figure 12A FIG. 12 is a schematic diagram of a second image for a meeting scenario according to an embodiment of the present application;

[0032] Figure 12B FIG. 13 is a schematic diagram of a second image for a meeting scenario according to another embodiment of the present application;

[0033] Figure 12C FIG. 14 is a schematic diagram of a corrected second image according to another embodiment of the present application;

[0034] Figure 13 FIG. 15 is a schematic diagram of a multi-target window image according to an embodiment of the present application;

[0035] Figure 14A FIG. 16 is a schematic diagram of a second image for a multi-target window according to an embodiment of the present application;

[0036] Figure 14B is a schematic view of a modified second image according to an embodiment of the present application.

[0037] BRIEF DESCRIPTION OF DRAWINGS

[0038] 1: image processing system;

[0039] 10: image capturing device;

[0040] 11: lens;

[0041] 13: image sensor;

[0042] 30: first controller;

[0043] 31: memory;

[0044] 35: processor;

[0045] 50: second controller;

[0046] 51: memory;

[0047] 55: processor;

[0048] S210-S270: steps;

[0049] FIM1-FIM2: first image;

[0050] SIM1-SIM28: second image;

[0051] FOV1-FOV7: field of view;

[0052] SR1, SR2: scale;

[0053] SH1, SH2: direction;

[0054] TI1: upward adjustment;

[0055] TI2: downward adjustment;

[0056] PA1: rightward adjustment;

[0057] PA2: leftward adjustment;

[0058] RO1: clockwise rotation;

[0059] RO2: counterclockwise rotation;

[0060] x, y, z: axis;

[0061] P1, P2, P3, P4: person;

[0062] IMe: fisheye image;

[0063] IMo, IMo1, IMo2: outer ring image;

[0064] TW1~TW6: window;

[0065] M1~M15: mode;

[0066] CS1, CS2: coordinate system;

[0067] (x o ,y o ), (x t ,y t ): coordinate;

[0068] T1~T6: target object;

[0069] θ1, θ2, θ3: angle;

[0070] BB1, BB2: bounding box. DETAILED DESCRIPTION

[0071] Reference will now be made in detail to the exemplary embodiments of the present application, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or like parts.

[0072] Figure 1 is a component block diagram of an image processing system 1 according to an embodiment of the present application. Please refer to Figure 1 , the image processing system 1 includes (but not limited to) an image capturing device 10, a first controller 30 and a second controller 50.

[0073] The image capturing device 10 can be a camera, a video camera, a monitor or a similar functional device. The image capturing device 10 can include (but not limited to) a lens 11 and an image sensor 15 (e.g., a Charge Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS), etc.). In an embodiment, an image can be captured by the lens 11 and the image sensor 15. For example, light rays are imaged on the image sensor 15 through the lens 11.

[0074] In some embodiments, the specifications (e.g., taking aperture, magnification, focal length, taking visual angle, size of the image sensor 15, etc.) of the image capturing device 10 and the number thereof can be adjusted according to actual needs. For example, the lens 11 is a fisheye or a wide-angle lens, and accordingly a fisheye image or a wide-angle image is generated.

[0075] The first controller 30 can be coupled to the image capture device 10 through a camera interface, an I2C, and / or other transmission interfaces. The first controller 30 includes, but is not limited to, a memory 31 and a processor 35. The memory 31 can be any type of fixed or removable random access memory (RAM), read only memory (ROM), flash memory, conventional hard disk drive (HDD), solid-state drive (SSD), or similar component. In an embodiment, the memory 31 is used to store program codes, software modules, configuration settings, data, or files. The processor 35 can be an image processor or a graphic processing unit (GPU), or other programmable general purpose or special purpose microprocessors (Microprocessor), digital signal processors (DSP), programmable controllers, field programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), or other similar components or combinations thereof. In an embodiment, the processor 35 is used to perform all or part of the operations of the first controller 30, and can load and execute various program codes, software modules, files, and data stored in the memory 31.

[0076] The second controller 50 can be coupled to the first controller 30 through a camera interface (e.g., a mobile industry processor interface (MIPI)), an I2C, a USB, and / or other transmission interfaces. The second controller 50 includes, but is not limited to, a memory 51 and a processor 55. The implementation and function of the memory 51 can refer to the description of the memory 31, which will not be repeated here. The implementation and function of the processor 55 can refer to the description of the processor 35, which will not be repeated here. In an embodiment, the processor 55 is used to perform all or part of the operations of the second controller 50, and can load and execute various program codes, software modules, files, and data stored in the memory 51.

[0077] In one embodiment, the image capturing device 10, the first controller 30 and the second controller 50 can be integrated into a standalone device. For example, the image processing system 1 is a camera system, in which the first controller 30 can be a fisheye controller, a wide-angle lens controller or other image-related controller, and the second controller 50 is a microcontroller or a SoC. In another embodiment, the image capturing device 10 and the first controller 30 can be integrated into a module, and the second controller 50 is, for example, a computer system (e.g., a desktop computer, a laptop computer, a server, a smartphone or a tablet computer) or is disposed in one of the parts thereof. In yet another embodiment, the first controller 30 and the second controller 50 can be integrated into an image controller or a suitable controller module, and can be coupled with the image capturing device 10.

[0078] In the following, the method according to the embodiments of the present application will be described in conjunction with the devices, components and modules in the image processing system 1. The various flows of the method can be adjusted accordingly depending on the implementation, and are not limited thereto.

[0079] Figure 2 is a flowchart of an image correction method according to an embodiment of the present application. Please refer to Figure 2 The first controller 30 obtains a first image from the image capturing device 10 (step S210). Specifically, the first image is an image of one or more target objects captured by the image capturing device 10 or other external image capturing device. In one embodiment, the target objects are, for example, human bodies. In some embodiments, the first image can also be for the upper body (e.g., the waist, the shoulders or above the chest) of a human. In other embodiments, the target objects can also be various types of living or non-living objects. The first controller 30 can obtain the first image captured by the image capturing device 10 via a camera interface and / or an I2C.

[0080] The first controller 30 can convert the first image into a second image according to a conversion operation (step S230). Specifically, in one embodiment, the conversion operation includes a distortion correction. The distortion correction is used to correct the distortion of one or more target objects in the first image. In another embodiment, the conversion operation includes a position adjustment. The position adjustment is used to adjust the position of one or more target objects in the first image. In yet another embodiment, the conversion operation includes a distortion correction and a position adjustment. That is, the appearance and / or position of the target objects in the first image can be different from the same target objects in the second image.

[0081] For example, Figure 3A is a schematic diagram of a dewarping according to an embodiment of the present application. Please refer to Figure 3AIn this embodiment, the transformation correction is, for example, a de-warping process, in which the first image FIM1 is, for example, an image obtained by a fisheye lens, which is, for example, a warped image. The first controller 30 can perform a conversion operation, which is, for example, a de-warping unwrapping, on the first image FIM1 to generate a second image SIM1, which is closer to a real image. In this way, an image of the target object with a better scale or a normal scale can be generated.

[0082] In some application scenarios, the image capturing angle can be adjusted to adapt to the size (e.g., resolution 1920x1080 or 480x272) or scale (e.g., 16:9 or 4:3) of the display device. Figure 3B is a schematic diagram of the angle adjustment according to an embodiment of the present application. Please refer to Figure 3B In this embodiment, the position adjustment included in the conversion operation is, for example, an angle adjustment, and the image capturing angle of the lens 11 of the image capturing device 10 is, for example, 180 degrees. In this embodiment, the first controller 30 can change or adjust the image capturing angle of the first image FIM1 to an angle FOV1 (e.g., 140 degrees) (i.e., a conversion operation) to generate a second image SIM2. Alternatively, the first controller 30 can change or adjust the image capturing angle of the first image FIM1 to an angle FOV2 (e.g., 110 degrees) (i.e., a conversion operation) to generate a second image SIM3. In this way, the image capturing angle can be directed towards the target object at a specific position in front of the lens 11.

[0083] In some application scenarios, the image processing application has a zooming image requirement. For example, if the image is not sufficient for image recognition, the image needs to be zoomed in. In addition, zooming in the image will cause the image capturing angle to be relatively reduced. If the original size of the image is desired to be viewed, the image capturing angle can be restored by zooming out the image. Figure 3C is a schematic diagram of the zoom adjustment according to an embodiment of the present application. Please refer to Figure 3C In this embodiment, the conversion operation can also include a zoom adjustment. In this embodiment, the first image FIM1 can be zoomed in according to a zoom ratio SR1 (e.g., 120%) (i.e., a conversion operation) to generate a second image SIM5. Alternatively, the first image FIM1 can be zoomed out according to a zoom ratio SR2 (e.g., 80%) (i.e., a conversion operation) to generate a second image SIM6. In this way, the target object can be zoomed in or out.

[0084] In some application scenarios, after the image is zoomed in, the area beyond the viewable range in the image can be viewed by shifting. Figure 3D is a schematic diagram of the shifting according to an embodiment of the present application. Please refer to Figure 3DIn this embodiment, the position adjustment included in the conversion job is, for example, a shift of the image. The dashed-line frame in the figure represents the visible range of the first image, and the solid-line frame represents the visible range of the second image. The first controller 30 can, for example, notify the image capturing apparatus 10 that the first image FIM1 (i.e., the conversion job) can be shifted according to the direction SH1 (to the right upper side) to generate the second image SIM7. Alternatively, the first controller 30 can, for example, notify the image capturing apparatus 10 that the first image FIM1 (i.e., the conversion job) can be shifted according to the direction SH2 (to the left lower side) to generate the second image SIM8. Thereby, the position of the target object in the image can be changed.

[0085] In some application situations, when the image of interest is located above or below the image capturing apparatus 10, the angle can be adjusted by up-down angle adjustment (or tilt adjustment) to obtain a better image capturing angle. For example, when the image capturing apparatus 10 is integrated in an electronic doorbell and installed on a wall, the height of the image capturing apparatus 10 can be higher or lower than the standing height of a person, and thus the image capturing angle can be changed by up-down angle adjustment. Figure 3E is a schematic diagram of up-down angle adjustment according to an embodiment of the present application. Please refer to Figure 3E In this embodiment, the position adjustment included in the conversion job is, for example, up-down angle adjustment. The image capturing apparatus 10 is set upright and faces the y axis. The first controller 30 can, for example, notify the image capturing apparatus 10 that the angle of the first image FIM1 can be adjusted upward according to the axis x (i.e., the conversion job) to generate the second image SIM9. Alternatively, the first controller 30 can, for example, notify the image capturing apparatus 10 that the angle of the first image FIM1 can be adjusted downward according to the axis x (i.e., the conversion job) to generate the second image SIM10. Thereby, the position of the target object in the image can be changed.

[0086] In some application situations, when the target object of interest is located to the left or right of the image capturing apparatus 10, the angle can be adjusted by left-right angle adjustment (or pan adjustment) to obtain a better image capturing angle. For example, when the image capturing apparatus 10 is integrated in an electronic doorbell and installed on a wall, the lens 11 can not face the visitor directly, and thus the image capturing angle can be adjusted to face the visitor by left-right angle adjustment. Figure 3F is a schematic diagram of left-right angle adjustment according to an embodiment of the present application. Please refer to Figure 3F, the position adjustment included in the conversion job is, for example, a left-right view angle adjustment (or a left-right rotation angle adjustment (pan)). The image capturing device 10 is set upright and faces the y-axis. The first controller 30 can, for example, inform the image capturing device 10 that the view angle of the first image FIM1 can be adjusted rightward PA1 according to the z-axis (i.e., a conversion job) to generate a second image SIM11. Alternatively, the first controller 30 can, for example, inform the image capturing device 10 that the view angle of the first image FIM1 can be adjusted leftward PA2 according to the z-axis (i.e., a conversion job) to generate a second image SIM12. Thereby, the position of the target object in the image can be changed.

[0087] Figure 3G is a schematic diagram of a planar view angle adjustment according to an embodiment of the present application. Please refer to Figure 3G , the position adjustment included in the conversion job is, for example, a planar view angle adjustment (or a rotation). The image capturing device 10 is set flat and faces the y-axis. The first controller 30 can, for example, inform the image capturing device 10 that the view angle of the first image FIM1 can be rotated clockwise RO1 according to the y-axis (i.e., a conversion job) to generate a second image SIM13. Alternatively, the first controller 30 can, for example, inform the image capturing device 10 that the view angle of the first image FIM1 can be rotated counterclockwise RO2 according to the y-axis (i.e., a conversion job) to generate a second image SIM14. Thereby, the position of the target object in the image can be changed.

[0088] Regarding the application scenario of the planar view angle adjustment, Figure 3H is a schematic diagram of the arrangement of the image capturing device 10 and the captured images according to an embodiment of the present application. Please refer to Figure 3H , the image capturing device 10 is set flat. Assuming that the lens 11 is a fisheye lens, light passes through the lens 11 and projects on the image sensor 13 to generate a fisheye image IMe. According to the application requirement, the first controller 30 can, for example, inform the image capturing device 10 that only the outer ring image IMo corresponding to the outer ring of the lens 11 can be captured.

[0089] Figure 3I is a schematic diagram of the arrangement of the image capturing device 10 and the captured images according to an embodiment of the present application. Please refer to Figure 3I , as shown in the upper half of the figure, assume that the image capturing device 10 is set flat between the persons P1, P2, P3, and P4. If only the outer ring of the lens 11 is captured and divided into two halves, the outer ring images IMo1 and IMo2 can be generated. The outer ring image IMo1 captures the persons P1 and P2, and the outer ring image IMo2 captures the persons P3 and P4.

[0090] Figure 3J is a schematic diagram of the arrangement of the image capturing device 10 and the captured images according to an embodiment of the present application. Please refer to Figure 3JAssume that the first controller 30 adjusts the planar perspective of the outer ring images IMo1 and IMo2 according to the direction of the arrows shown in the left figure. In the figure, the light-colored lines representing figures P1~P4 indicate their original positions, and the dark-colored lines representing figures P1~P4 indicate their positions after the planar perspective adjustment. The outer ring image IMo1 will shift figures P1 and P2 to the left, and the outer ring image IMo2 will shift figures P3 and P4 to the right.

[0091] Figure 4 This is a schematic diagram of a fisheye image unfolding according to an embodiment of the present invention. Please refer to... Figure 4 Assume the first image FIM2 is a fisheye image, and the people in the image are distorted. The second image SIM15 is an image corrected by the first controller 30, and the proportions of the people in the image are normal.

[0092] In one embodiment, the conversion operation further includes target arrangement (or window arrangement, multi-segment window). The second image includes, for example, multiple windows. Furthermore, it is assumed that the second image includes one or more targets. For example, a first target, a second target, a third target, and / or a fourth target. Target arrangement is used to adjust the target window (i.e., one of those windows) of the first target among these targets in the second image. This embodiment may divide the second image into multiple windows, crop the distortion-corrected image, and arrange the cropped portion of the image within specific windows. In a preferred embodiment, this embodiment utilizes an application to divide the second image into multiple windows.

[0093] For example, Figure 5 This is a schematic diagram of target arrangement according to an embodiment of the present invention. Please refer to... Figure 5 The first controller has 30 pairs. Figure 4 After distortion correction is performed on the first image FIM2, the scaled and / or cropped images are arranged in different windows TW1 and TW2, for example. The images arranged in different windows TW1 and TW2 are, for example, images of people participating in the meeting or images of participants after specific screening. Images of participants after specific screening are, for example, participants speaking in the meeting.

[0094] Figure 6 This is a schematic diagram illustrating the target arrangement of various modes M1 to M15 according to an embodiment of the present invention. Please refer to... Figure 6The various modes Ml - Ml 5 can include one window TWl, two windows TWl, TW2, three windows TWl - TW3, four windows TWl - TW4, five windows TWl - TW5, or six windows TWl - TW6. The split line in each mode M2 - Ml 5 is the window range. TWl - TW6 are used to represent the number of different windows. Further, even if the number of windows is the same, the position, size, and / or shape of the windows in different targets can be different. For example, the window TWl of mode M6 is larger than the window TWl of mode 2.

[0095] It should be noted that there can be other variations in the number, size, and shape of the windows, and embodiments of the present application are not limited to the same. Further, the symbols "TWl", "TW2", "TW3", "TW4", "TW5", "TW6" are merely used as designations.

[0096] Three modes are used as examples below:

[0097] Figure 7A is a schematic diagram of a target arrangement of modes according to an embodiment of the present application. Referring to Figure 4 , Figure 6 and Figure 7A , if the first controller 30 selects mode Ml in Figure 6 , then the first image FIM2 in Figure 4 can be converted to the second image SIM16 in Figure 7A , where the second image SIM16 in Figure 7A is, for example, a single window mode.

[0098] Figure 7B is a schematic diagram of a target arrangement of modes according to an embodiment of the present application. Referring to Figure 4 , Figure 6 and Figure 7B , if the first controller 30 selects mode M5 in Figure 6 , then the first image FIM2 in Figure 4 can be converted to the second image SIM17 in Figure 7B . The second image SIM17 in Figure 7B is, for example, a two-split window. Window TWl is for person P3, and window TW2 is for the four persons PI, P2, P3, P4.

[0099] Figure 7C is a schematic diagram of a target arrangement of modes according to an embodiment of the present application. Referring to Figure 4 , Figure 6 and Figure 7C , if the first controller 30 selects mode Ml 1 in Figure 6 , then the first image FIM2 in Figure 4 can be converted to the second image SIM18 inFigure 7C The second image SIM18 is, for example, a three-part window. The window TW1 is for the four persons P1, P2, P3, P4, the window TW2 is for the person P3, and the window TW3 is for the person P2.

[0100] Referring back to Figure 2 The second controller 50 of the present embodiment can detect one or more objects in the second image to generate a detection result (step S250). Specifically, the second controller 50 can obtain the second image converted by the first controller 30 via the USB and / or I2C interface.

[0101] In an embodiment, the second controller 50 can perform object detection on the second image. Object detection is, for example, determining a bounding box or pinot (possibly located at the contour, center, or any location on the object) in the second image that corresponds to an object (e.g., a person, an animal, a non-living object, or a part thereof), and thereby recognizing the type of the object (e.g., human, male or female, dog or cat, table or chair, etc.). The detection result includes the bounding box (or pinot) of the object and / or the type of the object. The object detection described in the present disclosure can also be determining a Region of Interest (ROI) or a bounding rectangle in the second image that corresponds to the object, without any limitation herein.

[0102] In an embodiment, the second controller 50 can apply, for example, a neural network based algorithm (e.g., YOLO, Region Based Convolutional Neural Networks (R-CNN), or Fast R-CNN) or a feature matching based algorithm (e.g., Histogram of Oriented Gradient (HOG), Harr, or Speeded Up Robust Features (SURF)) to perform object detection.

[0103] It should be noted that the present embodiments do not limit the algorithm used for object detection. In addition, in some embodiments, the second controller 50 can specify a particular type of object.

[0104] In an embodiment, the second controller 50 determines the position of the target object in the second image. That is, the detection result includes the position of the target object. For example, whether the target object is in the middle of the second image. For another example, whether the target object appears in the second image. In some embodiments, the second controller 50 can define a reference axis (e.g., a horizontal axis or a vertical axis) in the second image and determine the angle of the target object in the second image relative to the reference axis.

[0105] In an embodiment, the second controller 50 can also determine whether the target object moves in the second image. That is, the detection result includes the motion of the target object. For example, the second controller 50 can determine the correlation and change in position or pose of the same target object in consecutive image frames in the second image through object tracking. The consecutive image frames represent those consecutive image frames of a video or a video stream. And the object tracking is, for example, to determine the position, movement, direction, and other motion correlation of the same target object (whose position can be determined by a bounding box or a representative point) in adjacent second images, thereby locating the moving target object. In an embodiment, the second controller 50 can apply, for example, an optical flow method, a Simple Online And Realtime Tracking (SORT) method, a Deep SORT method, a Joint Detection and Embedding (JDE) model, or other tracking algorithms to implement object tracking. It should be noted that the embodiments of the present application do not limit the algorithms used for object tracking.

[0106] As described above, in a preferred embodiment, the second controller 50 can determine the correlation and change in position or pose of the same target object in consecutive image frames in the second image through object tracking. Preferably, when the detection result generated by the second controller 50 detecting the target object in the second image is that the target object in the second image does not move, the embodiment can only perform conversion operations such as dewarping on the target object corresponding to the position of the bounding box, the region of interest (ROI), or the bounding rectangle, so that the target object in the bounding box, for example, has a better or normal proportion of the target object image. In other words, when the detection result is that the position of the at least one target object in the second image does not change, the embodiment can not need to perform conversion operations such as dewarping on the entire image, but only correct the deformation of the range corresponding to the position in the first image, thereby improving the efficiency of image processing operations.

[0107] In contrast, when the same target object in the consecutive second images in the front and back image frames (or bounding box selections) changes in position or pose, the second controller 50 detects the detection result of the target object in the second images as the movement of the target object in the second images. The present embodiment corrects the conversion job by the detection result, so that the overall image performs a conversion job such as dewarping, and then detects the target object again.

[0108] In an embodiment, the second controller 50 determines the integrity of the target object in the second images. That is, the detection result includes the integrity of the target object. For example, the second controller 50 can recognize the key points (e.g., eyes, nose, or mouth) of the target object in the second images, and confirm whether the corresponding parts are complete or the number is correct.

[0109] It should be noted that in some embodiments, the first controller 30 also detects one or more target objects in the second images to generate a detection result. For example, the first controller 30 performs object detection, object tracking, or integrity detection on the second images, the same or similar content of which can be referred to the foregoing description and will not be repeated here.

[0110] The first controller 30 can correct the conversion job according to the detection result (step S270). Specifically, the target object and / or the image capture device 10 can change the position. If the conversion job remains unchanged, the position of the target object in the second images can not be centered or part of the target object is cut off and not complete, thereby affecting the viewing experience. In an embodiment, the second controller 50 can return the detection result to the first controller 30, and determine whether the conversion job needs to be adjusted accordingly.

[0111] In an embodiment, the position of the target object in the detection result is in a bounding box format. The bounding box format includes the coordinates of the horizontal and vertical axes of the bounding box in the second images and the size (e.g., width and height) of the second images.

[0112] In another embodiment, the position of the target object in the detection result is in a representative point format. The representative point format includes the coordinates of the horizontal and vertical axes of the representative point in the second images and the scaling factor.

[0113] In an embodiment, the coordinate system defined by the first controller 30 and the second controller 50 for the second images can be different. Figure 8 FIG. 1 is a schematic diagram of coordinate system conversion according to an embodiment of the mode of the present application. Please refer to Figure 8The second controller 50 positions the pixels in the second image in a coordinate system CS1, such as image coordinates. The coordinate system CS1 has its origin at the upper left corner. The first controller 30 positions the pixels in the second image in a coordinate system CS2, such as a polar coordinate system. The coordinate system CS2 has its origin at the center point. In other preferred embodiments, either the first controller 30 or the second controller 50 can position the pixels in the second image in an appropriate coordinate system, which is not limited herein.

[0114] If the detection result includes the target position of the fourth target in the second image, the first controller 30 or the second controller 50 can convert the coordinates (x o ,y o ) of the target position of the fourth target from the coordinate system CS1 to the coordinates (x t ,y t ) of the coordinate system CS2, with the conversion formulas being:

[0115] x t =x o -w / 2…(1)

[0116] y t =y o -h / 2…(2)

[0117] x o is the coordinate of the fourth target on the horizontal axis of the coordinate system CS1, x t is the coordinate of the fourth target on the horizontal axis of the coordinate system CS2, y o is the coordinate of the fourth target on the vertical axis of the coordinate system CS1, y t is the coordinate of the fourth target on the vertical axis of the coordinate system CS2, w is the width of the second image, and h is the height of the second image.

[0118] It should be noted that if the first controller 30 and the second controller 50 use the same coordinate system, the coordinate conversion can be ignored.

[0119] It is worth noting that the target object can not be centered in the second image or the window of the second image. For example, Figure 9 is a schematic diagram of the second image SIM19 under the imaging view angle FOV3 according to an embodiment of the present application. Please refer to Figure 9, assume that the maximum view angle FOV4 of the image capturing device 10 is 180 degrees. That is, the first image includes a 180-degree field of view. The view angle FOV3 set by the conversion job is 140 degrees. Assume that an imaginary line extending straight forward from the location of the image capturing device 10 is the reference axis (i.e., the vertical center line of the image capturing angle FOV3). The target object T1 is located straight forward of the image capturing device 10, i.e., the offset angle of the target object T1 with respect to the reference axis is zero. Therefore, the target object T1 is located in the middle of the second image SIM19.

[0120] Figure 10 is a schematic diagram of a second image SIM20 of a rotated view angle according to an embodiment of the present application. Please refer to Figure 10 , unlike Figure 9 , the offset angle of the target object T2 with respect to the reference axis is θ1 (e.g., 45 degrees). Therefore, the target object T2 is located on the left side of the second image SIM20.

[0121] In an embodiment, the first controller 30 can align the target object according to the offset angle. The detection result is the offset angle of a first target object in the second image with respect to the reference axis. The reference axis is the horizontal or vertical center line of the original image capturing angle of the second image. The first controller 30 can set the conversion job to rotate the image capturing angle of the first image to reduce the offset angle according to the offset angle. For example, the rotated view angle includes the up-down view angle adjustment previously described, Figure 3E the left-right view angle adjustment previously described, and Figure 3F the planar view angle adjustment shown in FIG. 6. Figure 3G

[0122] In an embodiment, the first controller 30 can convert the coordinates of the second image according to the ratio of the image capturing angle of the first image to the length (e.g., the width or height) of the first image in the direction corresponding to the axis of the first image capturing angle rotation. If the coordinate system CS2 is used, Figure 6 , the image capturing angle rotations in different axes are respectively:

[0123] Pan(x t )=x t ×fov H / w…(3)

[0124] Tilt(y t )=y t ×fov V / h…(4)

[0125] Rotate(x t )=x t ×fov H / w…(5)

[0126] fov​H fov represents the horizontal viewing angle of the second image. V This defines the vertical viewing angle of the second image. `Pan()` is a function for adjusting the vertical viewing angle. For example, the left and right rotation directions correspond to the x-axis, so it adjusts the horizontal viewing angle and the width of the second image. `Tilt()` is a function for adjusting the horizontal viewing angle. For example, the vertical rotation directions correspond to the y-axis, so it adjusts the vertical viewing angle and the height of the second image. `Rotate()` is a function for adjusting the planar viewing angle. Figure 3J As shown, the outer ring images IMo1 and IMo2 exhibit left and right shift effects, respectively. Therefore, the horizontal viewing angle and the width of the second image are considered.

[0127] In one embodiment, the detection result includes the image viewing angle of the rotated second image exceeding the (maximum) viewing angle of the image capture device 10. Figure 10 For example, after the FOV4 of the second image SIM21 is rotated, its left boundary exceeds the maximum FOV3 of the image capture device 10. Therefore, even if the target object T2 is located in the middle of the second image SIM21, a black block is generated on the left side of the second image SIM21 because a visible image cannot be obtained.

[0128] The first controller 30 can set the portion of the rotating second image's viewing angle (hereinafter referred to as the first viewing angle) that exceeds the viewing angle of the image capture device 10 (hereinafter referred to as the second viewing angle) to be limited to the edge of the second viewing angle. For example, if using Figure 8 In coordinate system CS2, the image viewpoint edge corrections are as follows:

[0129] If the coordinate of the horizontal axis rotated by the image-taking angle is x t2 If the coordinate x is between the left and right edges of the second image-taking viewpoint, then... t2 constant;

[0130] If the coordinate of the horizontal axis rotated by the image-taking angle is x t2 If the coordinates of the left edge of the second imaging viewpoint are less than the coordinates of the left edge, then the coordinate x t2 Corrected to the coordinates of the left edge of the second image-taking viewpoint;

[0131] If the coordinate of the horizontal axis rotated by the image-taking angle is x t2 If the coordinates are greater than the right edge of the second imaging viewpoint, then the coordinate x t2 Corrected to the coordinates of the right edge of the second imaging viewpoint;

[0132] If the coordinates of the vertical axis rotated by the image-taking angle are y t2 If the coordinate y is between the upper and lower edges of the second image-taking viewpoint, then... t2 constant;

[0133] if the coordinate y of the vertical axis after the image capturing angle rotation is less than the coordinate of the lower edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the lower edge of the second image capturing angle. t2 if the coordinate y of the vertical axis after the image capturing angle rotation is less than the coordinate of the lower edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the lower edge of the second image capturing angle. t2 if the coordinate y of the vertical axis after the image capturing angle rotation is less than the coordinate of the lower edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the lower edge of the second image capturing angle.

[0134] if the coordinate y of the vertical axis after the image capturing angle rotation is greater than the coordinate of the upper edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the upper edge of the second image capturing angle. t2 if the coordinate y of the vertical axis after the image capturing angle rotation is greater than the coordinate of the upper edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the upper edge of the second image capturing angle. t2 if the coordinate y of the vertical axis after the image capturing angle rotation is greater than the coordinate of the upper edge of the second image capturing angle, the coordinate y is corrected to the coordinate of the upper edge of the second image capturing angle.

[0135] For example, Figure 11 is a schematic diagram of a second image SIM 20, SIM 22 according to an embodiment of the present application. Please refer to Figure 11 , and Figure 10 The difference is that the first image capturing angle of the second image SIM 22 is corrected, so the target T2 is close to the middle of the second image (for example, the angle θ2 is 20 degrees).

[0136] Figure 12A is a schematic diagram of a second image SIM 23 of a conference scenario according to an embodiment of the present application. Please refer to Figure 12A In the application scenario of the conference mode, it is assumed that the lens 10 is a fisheye lens and can obtain a first image of 360 degrees, and the image is spliced in a manner of 180 degrees up and down. Since the targets T3, T4 are located in the upper 180-degree angle of view, and the targets T5, T6 are located in the lower 180-degree angle of view, the targets T3-T6 are located at appropriate positions in the second image.

[0137] Figure 12B is a schematic diagram of a second image SIM 24 of a conference scenario according to another embodiment of the present application. Please refer to Figure 12B , and Figure 12A The difference is that the targets T3-T6 of the present embodiment are located at non-ideal positions or are in a moving state. Therefore, the parts of the targets T3, T5 in the second image SIM 24 are cut (i.e., incomplete).

[0138] Figure 12C is a schematic diagram of a corrected second image SIM 25 according to another embodiment of the present application. Please refer to Figure 12C If the detection result of the second controller 50 is the completeness of the target, after determining the completeness of the target, the first controller 30 can be triggered to rotate the image capturing angle of the first image (for example, the rotation angle θ3). In this way, compared with Figure 12B , the targets T3-T6 can be completely presented in the second image SIM 25.

[0139] In one embodiment, the detection result includes a size ratio of the second target in the target object in the second image. The image processing system of this embodiment can set the conversion job according to the size ratio as changing the scaling factor of all or part of the second image to maintain the proportion of the second target in the second image. For example, the size of the second target in the window TW1 of the mode M1 can be larger than the size of the second target in the window TW1 of the mode M15 to improve the visual experience. Figure 6

[0140] In one embodiment, if the detection result is to locate the second target by the bounding box, the image processing system of this embodiment can set the scaling factor as the minimum of the width ratio, the height ratio, and the maximum ratio. The size ratio includes the height ratio and the width ratio. The width ratio is the ratio of the width of the second image to the width of the bounding box of the second target, and the height ratio is the ratio of the height of the second image to the height of the bounding box.

[0141] In another embodiment, if the detection result is to locate the second target by the representative point, the image processing system of this embodiment can set the scaling factor as the reference factor. That is, directly magnify the second target by the specified reference factor.

[0142] In one embodiment, the conversion job is arranged for the target. The image processing system of this embodiment will determine the target window according to the bounding box or the representative point in the original window. The image processing system of this embodiment can set the reference axis of the evaluation offset as the center axis of the bounding box of the third target or the extension line of the representative point in the plurality of target objects, and rotate the first image capturing view angle to align the third target with the target window. That is, the center of the view angle is directed towards the third target. The view angle in functions (3)-(4) can be replaced by the view angle for the target window. In addition, if the rotated view angle exceeds the bounding box, the first controller 30 can correct the view angle used for cropping the image, and limit the part exceeding the view angle to the edge of the bounding box.

[0143] In one embodiment, the image processing system can determine the size ratio of the target window of the third target in the second image, and set the conversion job according to the size ratio as changing the scaling factor of the third target. For example, the image processing system can set the scaling factor as the minimum of the second width ratio, the second height ratio, and the maximum ratio. The size ratio includes the second height ratio and the second width ratio. The second width ratio is the ratio of the width of the target window to the width of the bounding box of the third target, and the second height ratio is the ratio of the height of the target window to the height of the bounding box.

[0144] For example, Figure 13 is a schematic diagram of a multi-target window image according to one embodiment of the present application. Please refer to Figure 13 For example, Figure 6 ​The mode M11 is taken as an example. The window TW1 is the original window, and the windows TW2 and TW3 are the target windows. The original window is the image within the maximum view angle FOV5 of the image capturing device 10 or the default view angle of the second image in the single window mode. The other two target windows are the images within the view angles FOV6 and FOV7, respectively. The second controller 50 obtains the bounding boxes BB1 and BB2 according to the detection result, and arranges the images in the two BB1 and BB2 in the windows TW2 and TW3, respectively. Compared with the view angle FOV5, the view angles FOV6 and FOV7 are rotated, and the left and right boundaries of the view angles FOV6 and FOV7 are the left and right boundaries of the bounding boxes BB1 and BB2. Therefore, the face in each target window can be automatically centered, and the face in the target window can be automatically scaled to a proper size. In addition, if the face in the target window moves, the face in the target window can still be centered and the size of the face can still be maintained according to the detection result of the second controller 50.

[0145] Figure 14A FIG. 27 is a schematic diagram of the second image SIM27 with multiple target windows according to an embodiment of the present application. Please refer to FIG. 26 and FIG. 27. Figure 14A Figure 6 The mode M4 is taken as an example. Before the conversion operation is corrected, the faces of the target objects T3-T6 in the second image SIM27 are cut by the windows, and the visual effect is not good.

[0146] Figure 14B FIG. 28 is a schematic diagram of the corrected second image SIM28 according to an embodiment of the present application. Please refer to FIG. 27 and FIG. 28. Figure 14B The first controller 30 can correct the conversion operation according to the detection result that the target objects T3-T6 deviate from the center of the window. Compared with Figure 14A After the conversion operation is corrected, the faces of the target objects T3-T6 in the second image SIM28 are centered in the target windows, and the second image SIM28 presents the faces with proper sizes, so as to improve the visual effect. In addition, no matter where the target objects T3-T6 are located in the conference room or how the target objects T3-T6 move, the second image SIM28 can present the four-split screen with proper sizes (i.e., the faces are centered and the sizes of the faces are consistent).

[0147] In summary, in the image controller, the image processing system and the image correction method according to the embodiments of the present application, the conversion operation related to the deformation correction and / or the target arrangement is corrected based on the detection result of the image recognition. The embodiments of the present application can orient the view angle towards the target object and change the size of the target object in the image. In this way, the target object can be automatically centered in the image or the specified target window, and the size of the target object in the image can be automatically adjusted, so as to improve the visual effect. Even if the target object moves, the target object can still be centered and the size of the target object can still be maintained through the detection result and the operation correction.

[0148] ​It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can still be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. An image correction method, characterized in that, include: The first image is obtained through the first controller; The first controller converts the first image into a second image according to a conversion job, wherein the conversion job includes at least deformation correction, and the deformation correction is used to correct the deformation of at least one target object in the first image; The second controller detects the at least one target in the second image to generate a detection result, wherein the detection result includes the offset angle of the first target among the at least one target in the second image relative to a reference axis; as well as The first controller corrects the conversion job based on the detection result, and the corrected conversion job includes: The modified conversion operation, mediated by the first controller, further includes position adjustment to adjust the position of the at least one target object in the second image, wherein the position adjustment includes: The modified conversion operation, based on the offset angle, is to rotate the first image capture angle of the first image to reduce the offset angle.

2. The image correction method according to claim 1, characterized in that, The conversion operation further includes target arrangement, the second image includes multiple windows, the target arrangement is used to adjust the target window of the first target in the second image, the target window is one of the windows, and the step of setting the conversion operation to rotate the first imaging viewpoint of the first image according to the offset angle includes: Rotate the first image-taking viewpoint to align the first target with the target window.

3. The image correction method according to claim 1, characterized in that, The detection result includes a first imaging angle that has been rotated beyond a second imaging angle of the image capture device, and the position adjustment also includes: The portion of the rotated first imaging angle that extends beyond the second imaging angle is limited to the edge of the second imaging angle.

4. The image correction method according to claim 1, characterized in that, The step of setting the conversion operation to rotate the first imaging angle of the first image based on the offset angle includes: The coordinates of the first image are transformed according to the ratio of the length of the first image to the length of the first image along the axis corresponding to the direction of rotation of the first image viewpoint.

5. The image correction method according to claim 1, characterized in that, The detection result includes the size ratio of the second target in the at least one target in the second image, and the conversion operation is set to change the scaling factor from the first image to the second image based on the size ratio.

6. The image correction method according to claim 5, characterized in that, The step of setting the conversion operation to change the scaling factor from the first image to the second image according to the size ratio includes: The scaling factor is set to the minimum of the width ratio, height ratio, and maximum ratio, wherein the second target is positioned by a bounding box, the size ratio includes the height ratio and the width ratio, the width ratio is the ratio of the width of the second image to the width of the bounding box of the second target, and the height ratio is the ratio of the height of the second image to the height of the bounding box.

7. The image correction method according to claim 5, characterized in that, The step of setting the conversion operation to change the scaling factor from the first image to the second image according to the size ratio includes: The scaling factor is set as a reference scaling factor, wherein the second target is located as a representative point.

8. The image correction method according to claim 2, characterized in that, The detection result includes the size ratio of the target window of the third target among the at least one target in the second image, wherein the target window is one of the windows, and the step of correcting the conversion operation based on the detection result includes: The conversion operation is set to change the scaling factor of the third target based on the size ratio.

9. The image correction method according to claim 2, characterized in that, The detection result includes that the position of the at least one target object in the second image has not changed, while the deformation correction only corrects the deformation of the range corresponding to the position in the first image.

10. The image correction method according to claim 1, characterized in that, The second controller locates pixels in the second image using a first coordinate system, and the first controller locates pixels in the second image using a second coordinate system. The detection result includes the target position of the fourth target among the at least one target object in the second image, and the step of correcting the conversion operation based on the detection result further includes: The coordinates of the target location are converted from the first coordinate system to the second coordinate system, wherein the second coordinate system has the center point as the origin and the first coordinate system has the upper left corner as the origin.

11. An image processing system, characterized in that, include: An image capturing device includes a lens and an image sensor, and is used to capture a first image through the lens and the image sensor; A first controller is coupled to the image capture device and is used to convert the first image into a second image according to a conversion job, wherein the conversion job includes deformation correction and the deformation correction is used to correct the deformation of at least one object in the first image; as well as A second controller, coupled to the first controller, is used to detect the at least one target in the second image to generate a detection result, wherein the detection result includes the offset angle of a first target among the at least one target in the second image relative to a reference axis. The first controller is further used to correct the conversion operation based on the detection result, wherein the corrected conversion operation includes: The first controller causes the modified conversion operation to further include position adjustment to adjust the position of the at least one target object in the second image, wherein the position adjustment includes: The modified conversion operation, based on the offset angle, is to rotate the first image capture angle of the first image to reduce the offset angle.

12. The image processing system according to claim 11, characterized in that, The detection result includes a first rotating image-capturing angle that exceeds a second image-capturing angle of the image capture device, and the first controller is further configured to limit the portion of the first rotating image-capturing angle that exceeds the second image-capturing angle to the edge of the second image-capturing angle.

13. The image processing system according to claim 12, characterized in that, The conversion operation further includes target arrangement, the second image includes multiple windows, the target arrangement is used to adjust the target window of the first target in the second image, the target window is one of the windows, and the first controller is further used to: Rotate the first image-capturing viewpoint to align the first target with the target window; The detection result includes the fact that the position of the at least one target object in the second image has not changed, while the deformation correction only corrects the deformation of the range corresponding to the position in the first image.

14. The image processing system according to claim 12, characterized in that, The first controller is also used to: The coordinates of the first image are transformed according to the ratio of the length of the first image to the length of the first image along the axis corresponding to the direction of rotation of the first image viewpoint.

15. The image processing system according to claim 11, characterized in that, The detection result includes the size ratio of the second target in the at least one target in the second image, and the conversion operation is set to change the scaling factor from the first image to the second image based on the size ratio; The scaling factor is set to the smallest of the width ratio, height ratio, and maximum ratio. The second target is positioned by a bounding box. The size ratio includes the height ratio and the width ratio. The width ratio is the ratio of the width of the second image to the width of the bounding box of the second target, and the height ratio is the ratio of the height of the second image to the height of the bounding box. The scaling factor is set as a reference scaling factor, and the second target is located using a representative point.

16. The image processing system according to claim 13, characterized in that, The detection result includes the size ratio of the target window of the third target among the at least one target in the second image, wherein the target window is one of the windows, and the first controller is further configured to: The conversion operation is set to change the scaling factor of the third target based on the size ratio.

17. The image processing system according to claim 11, characterized in that, The second controller locates pixels in the second image using a first coordinate system, and the first controller locates pixels in the second image using a second coordinate system. The detection result includes the target position of the fourth target among the at least one target in the second image, and the first controller or the second controller is further configured to: The coordinates of the target location are converted from the first coordinate system to the second coordinate system, wherein the second coordinate system has the center point as the origin and the first coordinate system has the upper left corner as the origin.

18. An image controller, characterized in that, A device suitable for coupling with an image capture device, the image capture device including a lens and an image sensor, the image capture device capturing a first image through the lens and the image sensor, the image controller including: Memory, used to store program code; and A processor, coupled to the memory, is configured to load and execute the program code to: Obtain the first image; The first image is converted into a second image according to a conversion job, wherein the conversion job includes deformation correction, and the deformation correction is used to correct the deformation of at least one target object in the first image; Detect the at least one target in the second image to generate a detection result, wherein the detection result includes the offset angle of the first target among the at least one target in the second image relative to a reference axis; The conversion job is corrected based on the detection results, and the corrected conversion job includes: The modified conversion operation further includes position adjustment to adjust the position of the at least one target object in the second image, wherein the position adjustment includes: The image controller sets the corrected conversion operation based on the offset angle to rotate the first image's first viewing angle to reduce the offset angle.

19. The image controller according to claim 18, characterized in that, The detection result includes a first rotating image-capturing angle exceeding a second image-capturing angle of the image capture device, and the image controller is further configured to limit the portion of the first rotating image-capturing angle exceeding the second image-capturing angle to the edge of the second image-capturing angle.

20. The image controller according to claim 19, characterized in that, The conversion operation further includes target arrangement, the second image includes multiple windows, the target arrangement is used to adjust the target window of the first target in the second image, the target window is one of the windows, and the image controller is further used to: Rotate the first image-capturing viewpoint to align the first target with the target window; The detection result includes the size ratio of the target window of the third target among the at least one target object in the second image, wherein the target window is one of the windows, and the image controller is further used to: The conversion operation is set to change the scaling factor of the third target based on the size ratio.

21. The image controller according to claim 19, characterized in that, The coordinates of the first image are also transformed according to the ratio of the length of the first image to the length of the first image in the axial direction corresponding to the direction of rotating the first image perspective.

Images