Method and system for partially magnifying image

Abstract

A method for magnifying a local image includes the following steps: selecting a target region in a first image, wherein the target region contains at least one feature point; adjusting a specified optical magnification according to the position of the target region in the first image so that the target region is fully displayed in a second image; determining the corresponding position of the at least one feature point in the second image; and extracting an image of the corresponding target region from the second image based on the corresponding position of the at least one feature point in the second image.

Description

Technical Field

[0001] This invention relates to an electronic device and its operating method, and more particularly to a method and system for magnifying a local image. Prior Technology

[0002] A document camera (also known as a document projector) is a special type of camera that can display objects (mostly physical documents and images) placed on a specific surface in real time on the screen.

[0003] When manually performing PTZ (pan, tilt, zoom) operations on existing document cameras, one must first zoom in to a specific magnification based on experience, and then gradually adjust the pan / tilt to the desired position. The PTZ operation is cumbersome and usually requires multiple pan / tilt operations to achieve the desired result.

[0004] To address the cumbersome nature of manual operations, various fields have racked their brains to find solutions, but a suitable method has yet to be developed. Therefore, how to more efficiently and automatically magnify images is a crucial research topic and a goal that urgently needs improvement in related fields. Summary of the Invention

[0005] This invention proposes a system for locally magnified images, which improves upon the problems of prior art.

[0006] In some embodiments of the present invention, the method for magnifying a local image proposed in the present invention includes the following steps: selecting a target region in a first image, wherein the target region contains at least one feature point; adjusting a specified optical magnification according to the position of the target region in the first image so that the target region is fully displayed in a second image; determining the corresponding position of at least one feature point in the second image; and extracting an image of the corresponding target region from the second image based on the corresponding position of at least one feature point in the second image.

[0007] In some embodiments of the present invention, the system for partial magnification of images proposed in the present invention includes a photographic device and a computer device, the computer device being electrically connected to the photographic device. The photographic device acquires an initial image, the computer device selects a target area in the initial image, and adjusts the optical magnification of the photographic device accordingly, so that the target area is fully presented in the magnified image acquired by the photographic device based on the optical magnification. The computer device determines the corresponding position of at least one feature point in the target area in the magnified image, and extracts the image of the corresponding target area from the magnified image accordingly.

[0008] In summary, the technical solution of this invention has significant advantages and beneficial effects compared with the prior art. The method and system for locally magnified images of this invention provide a convenient and automated PTZ mode, and can achieve better display effects for locally magnified images.

[0009] The above description will be given in detail below with reference to the embodiments, and the technical solution of the present invention will be further explained. Simple Explanation of the Diagram

[0010] To make the above and other objects, features, advantages and embodiments of the present invention more apparent and understandable, the accompanying drawings are described below: Figure 1 is a block diagram of a system for partially magnified images according to one of the embodiments of the present invention; Figure 2 is a flowchart of a method for partially magnifying an image according to one embodiment of the present invention; and Figures 3 to 8 are schematic diagrams of a method for partially magnifying an image according to some embodiments of the present invention. Implementation

[0011] To make the description of this invention more detailed and complete, reference can be made to the accompanying drawings and the various embodiments described below, in which the same numbers represent the same or similar elements. Furthermore, well-known elements and steps are not described in the embodiments to avoid unnecessarily limiting the invention.

[0012] Figure 1 is a block diagram of a partial magnification image system 100 according to some embodiments of the present invention. As shown in Figure 1, the partial magnification image system 100 may include a photographic device 110, a computer device 120, an input device 130, and a display device 150. Architecturally, the photographic device 110, the input device 130, and the display device 150 are electrically connected to the computer device 120.

[0013] In practice, for example, computer device 120 may be a personal computer, laptop computer, tablet computer, smartphone or other computer circuitry; photography device 110 may be a document camera, document camera, webcam or other camera; input device 130 may be a mouse, keyboard, touch device or other human-computer input interface device; and display device 150 may be a display screen, monitor or other display device.

[0014] In use, the photographic device 110 acquires an initial image, and the computer device 120 selects a target area from the initial image. Based on this selection, the computer device 110 adjusts the optical magnification to ensure the target area is fully displayed in the magnified image acquired by the photographic device 110 based on the optical magnification. The computer device 120 determines the corresponding positions of feature points in the target area within the magnified image and extracts the image of the corresponding target area from the magnified image. For example, these feature points can be points where the image grayscale value changes drastically or points with significant curvature at the image edge (i.e., the intersection of two edges). The display device 150 can display the image of the corresponding target area. Thus, the partial magnification image system 100 provides a convenient and automated PTZ mode and achieves better display effects for partial magnification images.

[0015] Regarding the target area selection mechanism described above, in some embodiments of the present invention, the user can input selection information through the input device 130, and the computer device 120 can select the target area based on the selection information input by the input device 130. Alternatively, the computer device 120 can detect the target area based on object recognition. Both the aforementioned selection and object recognition methods are more convenient than the traditional manual PTZ method.

[0016] Regarding the specific mechanism for determining the corresponding positions of feature points in the target area within the magnified image, in some embodiments of the present invention, the computer device 120 detects a plurality of matching feature points in the magnified image that correspond to the plurality of feature points in the target area of ​​the initial image. Based on the coordinate changes from the plurality of feature points to the plurality of matching feature points, the computer device 120 calculates the corresponding coordinates and size of the target area in the magnified image, and thereby crops the corresponding target area from the magnified image. Thus, the system 100 for local magnification of images uses feature matching to find a specified area in the magnified image, thereby performing more precise local magnification.

[0017] Alternatively, regarding the specific mechanism for determining the corresponding positions of feature points in the target region within the magnified image, in some embodiments of the present invention, the computer device 120 calculates the corresponding coordinates of the target region in the magnified image based on the aforementioned optical magnification, obtains the coordinate positions of a plurality of vertices from the corresponding coordinates of the target region in the magnified image, and then crops the image of the corresponding target region from the magnified image accordingly. In this way, the system 100 for locally magnified images directly performs coordinate transformation, saving computational resources.

[0018] After extracting the image of the corresponding target area from the magnified image, in some embodiments of the present invention, the computer device 120 performs image processing on the image of the target area. Image processing may include, for example, at least one of super-resolution and automatic focusing. In practice, for example, the computer device 120 uses super-resolution to enhance the details of the image, or uses automatic focusing to highlight objects in the image. Thus, the system 100 for locally magnified images can improve image details and display quality.

[0019] To provide a more detailed explanation of the operation of the system 100 for magnifying a local image, please refer to Figures 1 and 2. Figure 2 is a flowchart of a method 200 for magnifying a local image according to some embodiments of the present invention. It should be understood that, unless otherwise specified, the order of the steps mentioned in the embodiment of Figure 2 can be adjusted according to actual needs, and they can even be performed simultaneously or partially simultaneously.

[0020] The method 200 for magnifying a local image can take the form of a computer program product on a non-transitory computer-readable recording medium, which has a plurality of computer-readable instructions contained in the medium. Suitable recording media may include any of the following: non-volatile memory, such as read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electronically eraseable programmable read-only memory (EEPROM); volatile memory, such as static RAM (SRAM), dynamic RAM (SRAM), double data rate random access memory (DDR-RAM); optical storage devices, such as read-only optical discs (CD-ROM), read-only bit multi-function discs (DVD-ROM); magnetic storage devices, such as hard disk drives and floppy disk drives.

[0021] In steps S201 and S202, a target area is selected from the first image. In practice, for example, the first image may be the initial image acquired by the photographic device 110.

[0022] Specifically, regarding the target area selection mechanism mentioned above, in some embodiments of the present invention, in step S201, the target area is selected by drawing a box. Alternatively, in step S202, the target area is detected based on object recognition. In practice, for example, the user can directly select the area to be magnified using the input device 130 (e.g., a mouse); or, after the computer device 120 detects the object through object recognition, the user can directly click on the object to be magnified using the input device 130.

[0023] In step S203, based on the position of the target area in the first image, the specified optical magnification is adjusted so that the target area is fully displayed in the second image. In some embodiments of the present invention, the computer device 120 finds a suitable optical magnification based on the coordinates of the specified area and controls the photographic device 110 to perform optical magnification to obtain a second image (e.g., a magnified image). In practice, for example, the closer to the center of the image, the larger the optical magnification, and the better the image magnification effect. Thus, the method 200 for locally magnifying images can dynamically determine the optical magnification of the camera, making locally magnified images clearer.

[0024] In step S204, the corresponding positions of the feature points contained in the target region in the second image are determined. Specifically, regarding the feature matching in step S204, in some embodiments of the present invention, at least one feature point in the second image is detected, at least one feature point in the first image and at least one feature point in the second image are matched, and the corresponding coordinates of the at least one feature point in the first image in the second image are calculated, thereby determining the corresponding positions of the feature points contained in the target region in the second image.

[0025] Furthermore, in step S204, based on the corresponding positions of the feature points in the second image, the image of the corresponding target area is extracted from the second image.

[0026] Next, in step S205, image processing is performed on the image of the target area, including super-resolution and automatic focusing.

[0027] Regarding super-resolution, in practice, for example, computer device 120 uses one or more low-resolution images and, through machine learning or image reconstruction methods, generates a high-resolution image with high quality and rich detail information, thereby enhancing the original image details.

[0028] Regarding automatic focusing, in practice, for example, the computer device 120 identifies the objects in the image through object recognition, reverses the grayscale of the rest of the image, and retains the image of the identified objects to reduce interference from other objects in the image.

[0029] On the other hand, in some embodiments of the present invention, before or after step S205, the method 200 for locally magnifying an image may selectively digitally magnify the image of the target area so that the user can view it.

[0030] To further illustrate the method 200 for magnifying a partial image, please refer to Figures 1-8, which are schematic diagrams of a method 200 for magnifying a partial image according to some embodiments of the present invention.

[0031] As shown in Figure 3, in step S201, the user can input selection information (e.g., using the mouse to select the area to be enlarged) through the input device 130. The computer device 120 selects the target area 310 in the first image 300 based on the selection information input by the input device 130. The display device 150 can display the first image 300 and the selected target area 310.

[0032] On the other hand, as shown in Figure 4, in step S202, the computer device 120 uses image processing technology in deep learning applications to analyze important elements or features in the image, automatically identify objects 410, 420, and 430 within the image, and mark them for the user to select. The display device 150 can display the first image 300 and the marked objects 410, 420, and 430. The user can directly select the object to be magnified through the input device 130.

[0033] To simplify the explanation, the following steps S203 in Figures 5 and 6 and steps S204 in Figures 7 and 8 will be explained based on the first image 300 and its target area 310 in Figure 3.

[0034] For example, assuming the camera device 110 has a single lens and its position remains unchanged, as shown in Figure 5, in step S203, the computer device 120 calculates the most suitable optical magnification for a specified area based on the field of view (FOV) of the camera device 110 at different optical magnifications, and controls the camera device 110 to perform optical magnification accordingly. In practice, for example, area 510 has a field of view of 1x optical magnification, area 520 has a field of view of 2x optical magnification, area 530 has a field of view of 3x optical magnification, and area 540 has a field of view of 4x optical magnification. The closer the specified object (e.g., target area 310 in Figure 3) is to the center of the first image 300, the greater the optical magnification can be.

[0035] When a designated object spans different areas, the computer device 120 primarily uses the lowest magnification to reduce the distortion rate of the magnified image from the photography device 110. In practice, for example, if the laptop displayed in the first image 300 of Figure 5 is the designated object (e.g., the target area), and it spans 2-4 times the visible range—that is, the laptop in the first image 300 spans areas 520, 530, and 540—only partially displays the laptop in area 530 (corresponding to 3x optical magnification) and area 540 (corresponding to 4x optical magnification). Only in area 520 (corresponding to 2x optical magnification) can the entire laptop be displayed. Therefore, the computer device 120 selects the lowest optical magnification of 2x from the aforementioned 2-4x optical magnification range, and thus sets the photography device 110 to 2x optical magnification.

[0036] As shown in Figure 6, in step S203, if the computer device 120 selects the relatively lowest optical magnification of 2x for the photography device 110, it controls the photography device 110 to perform optical magnification to obtain a second image 600, the range of which roughly corresponds to the area 520 in Figure 5.

[0037] As shown in Figure 7, in step S204, the computer device 120 can detect the scale-invariant features before and after image magnification through feature matching technology, compare the features in the magnified second image 600 to find the corresponding coordinates of the target area 310, and thus crop to achieve more accurate local magnification.

[0038] Regarding feature extraction and matching, in step S204, in some embodiments of the present invention, the computer device 120 performs Scale-Invariant Feature Transform (SIFT), a feature extraction algorithm for computer vision used to detect and describe local features in an image. The computer device 120 performs Fast Library for Approximate Nearest Neighbors (FLANN) to find relatively good matches more quickly. The computer device 120 performs FLANN to find two closest matching feature points for each feature point 710. If the matching feature point 720 has a significantly higher matching degree with feature point 710 than the other matching feature point, then matching feature point 720 is a valid matching feature point.

[0039] Next, in step S204, in some embodiments of the present invention, the computer device 120 removes outliers. In practice, for example, the computer device 120 uses standard deviation to filter outliers, thereby removing misclassified matching feature points.

[0040] Then, in step S204, in some embodiments of the present invention, the computer device 120 crops the image according to the matching coordinates. In practice, for example, the computer device 120 calculates the corresponding coordinates and size of the target region 310 in the second image 600 based on the coordinate changes of the feature points 710 and the matching feature points 720 of the target region 310 to form an enlarged target region 610, thereby cropping the image occupied by the enlarged target region 610.

[0041] In short, regarding the feature matching and cropping image in step S204, in some embodiments of the present invention, the computer device 120 detects a plurality of matching feature points 720 in the second image 600 that match the plurality of feature points 710 in the target area 310 of the first image 300. The computer device 120 calculates the corresponding coordinates and size of the target area 310 in the second image 600 based on the coordinate changes from the plurality of feature points 710 to the plurality of matching feature points 720, and cropes the image corresponding to the target area 310 from the second image 600 (i.e., the image 800 occupied by the enlarged target area 610 shown in Figure 8).

[0042] Alternatively, in step S204, the computer device 120 may perform coordinate transformation to replace the above-described feature matching. In some embodiments of the present invention, the computer device 120 calculates the corresponding coordinates of the target region 310 in the second image 600 based on the specified optical magnification (e.g., optical magnification), and obtains the coordinate positions of a plurality of vertices from the corresponding coordinates of the target region 310 in the second image 600. In practice, for example, the computer device 120 uses a plurality of vertices (e.g., four) of the boundary range (e.g., rectangle) of the target region 310 as feature points, and uses them to crop the image corresponding to the target region 310 from the second image 600 (i.e., the image 800 occupied by the magnified target region 610 shown in Figure 8).

[0043] In summary, the technical solution of the present invention has significant advantages and beneficial effects compared with the prior art. The method 200 and system 100 for local image magnification of the present invention provide a convenient and automated PTZ mode, and can achieve better display effects for local magnified images, avoiding the traditional cumbersome manual PTZ operation and eliminating the need for moving the photographic device 110.

[0044] Although the present invention has been disclosed above by way of embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

[0045] To make the above and other objects, features, advantages and embodiments of the present invention more apparent and understandable, the appended symbols are explained as follows. 100: Systems for Locally Magnified Images 110: Photographic equipment 120: Computer device 130: Input device 150: Display device 200: Methods for Local Image Magnification S201~S205: Steps 300: First Image 310: Target Area 410, 420, 430: Object 510, 520, 530, 540: Areas 600: Second Image 610: Zoom in on the target area 710: Feature Points 720: Matching Feature Points 800: Image

Claims

1. A method for partially magnifying an image, comprising the following steps: selecting a target region in a first image, wherein the target region includes at least one feature point; adjusting a specified optical magnification according to the position of the target region in the first image, and obtaining a second image according to a viewing range corresponding to the specified optical magnification, so that the target region is fully displayed in the second image; determining the corresponding position of the at least one feature point in the second image; and extracting an image corresponding to the target region from the second image according to the corresponding position of the at least one feature point in the second image.

2. The method for magnifying a local image as described in claim 1, wherein the step of selecting the target region includes: selecting the target region by drawing a box or detecting the target region based on object recognition.

3. The method for magnifying a local image as described in claim 1, wherein the step of determining the corresponding position of the at least one feature point in the second image comprises: detecting at least one feature point in the second image; matching the at least one feature point in the first image and the at least one feature point in the second image; and calculating a corresponding coordinate of the at least one feature point in the first image in the second image.

4. The method for magnifying a local image as described in claim 1 further includes: digitally magnifying the image of the target region.

5. The method for magnifying a local image as described in claim 1 further includes: performing image processing on the image of the target region, the image processing including super-resolution and automatic focusing.

6. A system for magnifying a partial image, comprising: a photographic device for acquiring an initial image; and a computer device electrically connected to the photographic device, wherein the computer device selects a target area in the initial image, adjusts an optical magnification of the photographic device accordingly, acquires a magnified image based on a viewing range corresponding to the optical magnification, such that the target area is fully presented in the magnified image, and the computer device determines the corresponding position of at least one feature point in the target area in the magnified image, thereby extracting an image corresponding to the target area from the magnified image.

7. The system for magnifying a partial image as described in claim 6 further includes: an input device electrically connected to the computer device, the computer device selecting the target area based on selection information input by the input device, or the computer device detecting the target area based on object recognition.

8. The system for magnifying a partial image as described in claim 6, wherein the computer device detects a plurality of matching feature points in the magnified image that match the feature points based on a plurality of feature points in the target region of the initial image, and the computer device calculates the corresponding coordinates and size of the target region in the magnified image based on the coordinate changes of the feature points to the matching feature points, thereby cropping the image corresponding to the target region from the magnified image.

9. The system for partial magnification of an image as described in claim 6, wherein the computer device calculates the corresponding coordinates of the target region in the magnified image based on the optical magnification, obtains the coordinate positions of a plurality of vertices from the corresponding coordinates of the target region in the magnified image, and thereby crops the image corresponding to the target region from the magnified image.

10. A system for magnifying a local image as described in claim 6, wherein the computer device performs image processing on the image of the target area, the image processing including at least one of super-resolution and automatic focusing.

Images