Tilted image correction

Abstract

Techniques for tilted image correction are described. The technique includes receiving a raw image depicting a squint view of a circular object (402) and pre-processing the raw image into an edge image (404). The technique further includes generating, by a machine learning model, a heat map containing an ellipse formed by a squint view of the circular object (406) based on the edge image. The technique further includes calculating ellipse parameters describing an ellipse of the heat map (408). The technique further includes performing an affine transformation on the original image using the ellipse parameter to generate a corrected image, wherein the corrected image converts an ellipse to a circle (410).

Description

Background Art

[0001] The present disclosure relates to automatic image modification and, more particularly, to tilted image correction.

[0002] Images of circular objects taken from an angle (e.g., tilted images) may cause the circular objects to appear oval. Tilt images are common given the variety of situations in which a camera (and / or associated user) may not be able to accurately capture an image of a circular object from a vantage point directly in front of the circular object. Furthermore, humans are able to easily infer information about circular objects in tilted images from their oval appearance. Thus, tilted images have little effect on human understanding. However, image recognition algorithms may experience increased error rates when classifying, characterizing, or otherwise inferring information about circular objects that appear oval in tilted images. Summary of the invention

[0003] Aspects of the present disclosure relate to a computer-implemented method that includes receiving an original image depicting an oblique view of a circular object. The method also includes preprocessing the original image into an edge image. The method also includes generating a heat map through a machine learning model based on the edge image, the heat map containing ellipses formed by the oblique views of the circular object. The method also includes calculating ellipse parameters of the ellipse describing the heat map. The method also includes performing an affine transformation on the original image using the ellipse parameters to generate a corrected image, wherein the corrected image converts the ellipse into a circle.

[0004] Other aspects of the present disclosure relate to systems and computer program products configured to perform the above methods.This summary is not intended to describe every aspect, every implementation, and / or every embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The accompanying drawings included in this application are incorporated into the specification and constitute a part thereof. They illustrate embodiments of the present disclosure and are used to explain the principles of the present disclosure together with the specification. The accompanying drawings are only used to illustrate certain embodiments and are not intended to limit the present disclosure.

[0006] Figure 1 A block diagram of an example computing environment implementing image correction software is shown according to some embodiments of the present disclosure.

[0007] Figure 2 An oblique image causing a circular object to appear elliptical and a corrected image modifying the circular object in the oblique image into a circle according to an embodiment of the present disclosure are shown.

[0008] Figure 3A , 3B , 3C, 3D, 3E, 3F, 3G, 3H, 3I, 3J, 3K and 3L show various intermediate images for generating a corrected image from a tilted image according to an embodiment of the present disclosure.

[0009] Figure 4 A flow chart of an example method for implementing image correction software according to some embodiments of the present disclosure is shown.

[0010] Figure 5 A flow chart of an example method for generating a heat map corresponding to a tilted image according to some embodiments of the present disclosure is shown.

[0011] Figure 6 A flow chart of an example method of determining ellipse parameters from a heat map of an oblique image according to some embodiments of the present disclosure is shown.

[0012] Figure 7 A flow chart of an example method for downloading, deploying, metering and billing for use of tilted image correction code according to some embodiments of the present disclosure is shown.

[0013] Figure 8 A block diagram of an example computing environment is shown in accordance with some embodiments of the present disclosure.

[0014] Although the present disclosure may be subjected to various modifications and alternative forms, its specific contents have been shown in the accompanying drawings in an exemplary form and will be described in detail. However, it should be understood that the present disclosure is not intended to be limited to the specific embodiments described. On the contrary, the present disclosure is intended to cover all modifications, equivalents and alternatives that meet the spirit and scope of the present disclosure. DETAILED DESCRIPTION

[0015] Various aspects of the present disclosure are directed to automatic image correction, more specifically, tilt image correction. Although the embodiments of the present disclosure are not limited to such applications, the embodiments of the present disclosure can be better understood in conjunction with the above content.

[0016] As previously mentioned, image recognition algorithms may experience increased error rates when classifying, characterizing, predicting, or otherwise inferring information about circular objects that appear oval in oblique images. While this challenge permeates throughout field image classification technology, it is particularly problematic in applications designed to gather information from oblique images of circular objects and perform actions based on the gathered information.

[0017] One example application involves automatic reading of analog meters. Analog meters can be used to measure pressure, temperature, flow, volume, and / or other information. A camera can be placed near the analog meter, and images of the camera are collected and automatically evaluated to collect information about the analog meter from the image (e.g., pressure readings, temperature readings, flow readings, etc.). In many cases, the camera may not be placed directly in front of the analog meter, so the image collected from the camera is an oblique image, in which the circular analog meter appears elliptical. This may occur when the camera cannot be placed in front of the analog meter, and / or a single camera is placed to collect information from multiple analog meters within the camera's field of view. In these cases, due to the elliptical appearance of the circular analog meter, image analysis software that evaluates the circular analog meter from the oblique image is prone to errors. Although analog meters are discussed above, aspects of the present disclosure are related to any approximately circular object with automatic image analysis applications (e.g., clocks, motion analysis, targets, road signs, etc.).

[0018] Various aspects of the present disclosure are directed to overcoming the above challenges by converting an oblique image of a circular object that appears elliptical to a corrected image in which the circular object appears circular. Various aspects of the present disclosure can accurately convert an oblique image to a corrected image by: (i) generating a heat map of an ellipse formed by a circular object in the oblique image; (ii) determining ellipse parameters based on the heat map; and (iii) performing an affine transformation on the oblique image using the ellipse parameters to convert the oblique image of the circular object that appears elliptical to a corrected image in which the circular object appears circular.

[0019] Advantageously, aspects of the present disclosure can improve the accuracy of image analysis involving oblique images of circular objects. More specifically, aspects of the present disclosure advantageously generate heat maps using a fully convolutional neural network (FCNN), which is well suited for generating heat maps of ellipses from oblique images of circular objects. Another example advantage is that the heat map eliminates most of the irrelevant pixels (e.g., outliers) in the oblique image that are not related to the circular object. Therefore, the heat map can determine the ellipse parameters in a single calculation pass (whereas other less robust techniques may require multiple calculation passes, which is time consuming and inefficient). Another example advantage is that the present disclosure can be used to correct any circular object in an oblique image (e.g., it does not have to be limited to one application, such as an analog meter).

[0020] Now referring to the accompanying drawings, Figure 1 A block diagram of an example computing environment 100 implementing tilted image correction software 104 is shown according to some embodiments of the present disclosure. The example computing environment 100 includes a server 102 communicatively coupled to a device 106 via a network 126.

[0021] The server 102 may be any computing configuration of hardware and / or software capable of executing the tilt image correction software 104. In some embodiments, the server 102 may be any server, computer, mainframe, or other combination of computer hardware capable of executing software. In some embodiments, the server 102 may be a virtual machine (VM), container instance, or other virtualized combination of discrete physical hardware resources.

[0022] Device 106 may be any camera, smartphone, tablet, wearable device, or other device having camera 108. Device 106 may use camera 108 to generate oblique image 110. Oblique image 110 includes a circular object that appears elliptical in oblique image 110 due to the angle formed between camera 108 and the circular object when oblique image 110 is generated.

[0023] The network 126 may be a local area network (LAN), a wide area network (WAN), an intranet, the Internet, or any other network 126 or group of networks 126 capable of connecting the above components together continuously, semi-continuously, or intermittently (directly or indirectly).

[0024] The server 102 may run the oblique image correction software 104. The oblique image correction software 104 may be configured to convert the oblique image 110 into a corrected image 124, wherein the corrected image 124 makes the circular object appear circular (rather than elliptical). The oblique image correction software 104 may be configured to modify the oblique image 110 to generate a modified image 112. For example, the modified image 112 may be cropped, resized, converted to a grayscale image, transformed into an edge image, and / or blurred. The modified image 112 may be provided to the FCNN 114, wherein the FCNN 114 is trained (e.g., based on a data set containing real and / or synthetic data) to generate a heat map of ellipses in oblique images of circular objects. The FCNN 114 may output a heat map 116, wherein the heat map 116 includes an ellipse 118 formed by the circular object in the oblique image 110. An advantage of the FCNN is that it is able to generate a heat map with relatively few abnormal pixels, thereby improving the accuracy and efficiency of the heat map generation process. Although FCNN 114 is discussed above, other types of machine learning models (whether currently known or subsequently developed) may also replace FCNN 114.

[0025] The tilt image correction software 104 can calculate ellipse parameters 122 based on the ellipse 118 of the heat map 116. The ellipse parameters 122 can include, for example, the ellipse center, axis lengths (e.g., half lengths of the major axis and / or minor axis), the angle of the major axis (e.g., the major axis), and / or other ellipse parameters (e.g., focus, eccentricity, axis intersection (e.g., vertex), angle of the minor axis (e.g., the minor axis), etc.). The ellipse parameters 122 can be provided to the affine transformer 120 along with the modified image 112. The affine transformer 120 can apply an affine transformation to the modified image 112 using the ellipse parameters 122 to generate a corrected image 124. In the corrected image 124, the circular object appears to be a circle (rather than an ellipse) due to the affine transformation performed by the affine transformer 120.

[0026] Figure 1 It is for illustrative purposes only and should not be construed as limiting. Figure 1 The components shown may be more, fewer and / or different components. In addition, the components shown may be split into multiple discrete components, and / or multiple discrete components may be combined into a single component, but these operations are all within the spirit and scope of the present disclosure. For example, the tilt image correction software 104 may be implemented in whole or in part on the device 106 instead of or in addition to being implemented on the server 102.

[0027] Figure 2 The oblique image 200 that makes a circular object appear elliptical and the corrected image 202 that modifies the circular object in the oblique image into a circular shape according to an embodiment of the present invention are shown. Figure 1 The tilted image 110 in FIG. 1 is consistent with the corrected image 202. Figure 1 Advantageously, the corrected image 202 may improve accuracy when evaluated using image analysis.

[0028] Figure 3A , 3B , 3C, 3D, 3E, 3F, 3G, 3H, 3I, 3J, 3K and 3L show various intermediate images according to embodiments of the present disclosure for generating a corrected image from a tilted image. Figures 3A-3K A summary discussion was given, and Figure 4-6 We will discuss the generation of Figures 3A-3K operation.

[0029] Figure 3A An oblique image is shown. As shown in the figure, Figure 3A The circular analog instruments in the photo were photographed at an angle, making them appear oval. Figure 3A The oblique image shown can be compared with Figure 1The oblique image 110 and / or Figure 2 The tilted image 200 in FIG.

[0030] Figure 3B Cropped and resized oblique images are shown. Cropping and / or resizing the oblique images can improve the accuracy of the rectified images (and thus the accuracy of image analysis performed on the rectified images) by converting the oblique images to a size and format consistent with the size and format of the FCNN trained to generate a heat map of an ellipse formed by the oblique images of a circular object.

[0031] Figure 3C A cropped and resized oblique image is shown, which has been converted to a grayscale image. Although shown in black and white for simplicity, in some embodiments, Figure 3A (original oblique image), Figure 3B (cropped and resized image) and Figure 3L (Correction images) are all color images. In these embodiments, Figure 3C is the grayscale image corresponding to the color image.

[0032] Figure 3D FIG. 4 shows an example of applying an edge detector (eg, a Laplacian image generated by applying a Laplacian operator, a Canny image generated by applying a Canny operator, a Sobel image generated by applying a Sobel operator, a Deriche image generated by applying a Deriche operator, a Prewitt image generated by applying a Prewitt operator, etc.) to a pixel. Figure 3C The edge image generated by the grayscale image. Figure 3E The edge blur is shown Figure 3D edge image.

[0033] Figure 3F It shows that by Figure 3E The blurred edge image in is fed to a machine learning model (e.g., Figure 1 Heatmap generated by FCNN 114 in . Figure 3F The heatmap in can be compared with Figure 1 The heatmap 116 in remains consistent.

[0034] Figure 3G A threshold heat map is shown, where the threshold heat map is a binary image that can be dilated and / or eroded to remove any screen door effect. Figure 3H It is a block heat map. Figure 3H Segment the ellipse formed in the threshold heat map into multiple blocks.

[0035] Fig. 3I A heat map of the chunks where each chunk has been detected is shown. Figure 3JA filtered tile heat map is shown, where a number of tiles have been removed from the heat map (eg, tiles that are too small, tiles whose distance from the tile's collection centroid exceeds a threshold distance, etc.).

[0036] Figure 3K Shown according to Figure 3J The fitted ellipse of the filtered tile heatmap (e.g., Figure 1 Finally, Figure 3L The corrected image is shown, where the corrected image is obtained after applying an affine transformation based on the ellipse parameters. Figure 3B An image that is cropped and / or resized to convert a skewed image depicting a circular analog meter from an ellipse to a circle depicting a circular analog meter. Figure 3L Can be used with Figure 1 The corrected image 124 and / or Figure 2 Advantageously, Figure 3L The corrected images shown improve the accuracy of image analysis algorithms, models, and / or techniques used to interpret information of the circular analog gauge.

[0037] Figure 4 4 is a flowchart of an example method 400 for implementing image correction software according to some embodiments of the present disclosure. In some embodiments, the method 400 is performed by a computer, a server (e.g., Figure 1 Server 102 in, device (e.g., Figure 1 The device 106 in the embodiment may be implemented by a processor, or other hardware and / or software configuration.

[0038] Operation 402 includes receiving an original image depicting an oblique view of a circular object. For example, operation 402 may receive Figure 1 The oblique image in 110. An oblique view of a circular object may cause the circular object to appear elliptical.

[0039] Operation 404 includes preprocessing the original image to generate a modified image (e.g., Figure 1 The image preprocessing may include one or more of the following: (i) cropping the image; (ii) adjusting the image size; (iii) converting the image to a grayscale image; (iv) applying edge detection to the image (e.g., applying a Laplacian operator, a Canny operator, a Sobel operator, a Deriche operator, a Prewitt operator, etc.); and / or (v) blurring the edge image. Figure 5 Operation 404 is discussed in more detail.

[0040] Operation 406 includes generating a heat map based on the original image (e.g., Figure 1 ), which shows an ellipse formed by an oblique view of a circular object (e.g., Figure 1 The heat map can be obtained by inputting the pre-processed raw image into a machine learning model (e.g., Figure 1 114), where the machine learning model outputs the heat map.

[0041] Operation 408 includes calculating ellipse parameters (e.g., Figure 1 Ellipse parameters 122 in the figure). The ellipse parameters may include, for example, the ellipse center, axis lengths (e.g., half lengths of the major axis and / or minor axis), angles of the major axis (e.g., the major axis), and / or other ellipse parameters (e.g., focus, eccentricity, axis intersection (e.g., vertex), angles of the minor axis (e.g., the minor axis), etc.).

[0042] Operation 410 includes performing an affine transformation on the original image to generate a corrected image (e.g., Figure 1 The affine transformation is a geometric transformation that preserves straight lines and parallelism, but does not necessarily preserve distances or angles. Affine transformations may include translation, scaling, homogenization, similarization, reflection, rotation, shear mapping, and / or any combination and / or order of the above transformations. Affine transformations may be performed using matrix algebra. Affine transformations may take the ellipse parameters calculated in operation 408 and the original image as input and output a corrected image. In the corrected image, a circular object appears circular.

[0043] Figure 5 1 is a flow chart showing an example method 500 for generating a heat map corresponding to an oblique image according to some embodiments of the present disclosure. In some embodiments, the method 500 is performed by a computer, a server (e.g., Figure 1 Server 102 in, device (e.g., Figure 1 In some embodiments, the method 500 is implemented by: Figure 4 A sub-method of operations 404 and / or 406.

[0044] Operation 502 includes cropping and / or resizing the original image. Operation 502 may crop the original image to select an identifiable region of interest (e.g., a circular object). Operation 502 may resize the image to the same size as the image used during training of the machine learning model. Operation 502 may generate a Figure 3B Image shown.

[0045] Operation 504 includes converting the cropped and / or resized image to a grayscale image. Operation 504 may convert the cropped and / or resized image to grayscale by selecting a maximum grayscale level for the red, green, and blue (RGB) channels for each pixel of the cropped and / or resized image. Advantageously, utilizing grayscale (e.g., utilizing RGB maximum values) may improve the accuracy of various aspects of the present disclosure by mitigating confounding factors introduced by various colors (e.g., multi-color analog meter scales). Additionally, the above-described particular method of converting an image to grayscale may improve the contrast of circular object-related features (e.g., dial scales in an analog meter) in the original image compared to other grayscale conversion mechanisms (e.g., based on brightness). Operation 504 may generate an image such as Figure 3C Image shown.

[0046] Operation 506 includes applying an edge detection operator (e.g., a Laplacian operator) to the grayscale image. Advantageously, the edge detection operator can emphasize important portions of a circular object in the grayscale image. For example, when the circular object is an analog meter, if the edge detection operator is a Laplacian operator, the Laplacian operator can emphasize the scale of the dial in the analog meter. Those skilled in the art will appreciate that the Laplacian operator is based on the divergence of a function gradient (e.g., the gradient of pixel features in an image). In this way, the Laplacian operator can act as a digital filter to identify the edges of the image. Operation 506 can generate an image such as Figure 3D Image shown.

[0047] Operation 508 includes blurring the edges of the edge image. Blurring the edges can improve the performance of the machine learning model in accurately generating a heat map through the machine learning model. Blurring the edges of the edge image can be achieved by adjusting the grayscale values ​​of pixels adjacent to the edge pixels (e.g., making the grayscale value of each pixel consistent with the average grayscale value of the adjacent pixels). Operation 508 can generate the following Figure 3E Image shown.

[0048] Operation 510 includes providing the blurred edge image to a machine learning model (e.g., Figure 1 The machine learning model may be trained on real and / or synthetic images of circular objects and corresponding heat maps that depict elliptical outlines corresponding to the circular objects due to oblique views of the circular objects. Operation 512 includes receiving a heat map corresponding to the original image (e.g., Figure 1 Operation 510 may generate a heat map 116 as shown in FIG. Figure 3F Image shown.

[0049] Figure 66 is a flow chart showing an example method 600 for determining ellipse parameters from a thermal map of an oblique image according to some embodiments of the present disclosure. In some embodiments, the method 600 is performed by a computer, a server (e.g., Figure 1 Server 102 in, device (e.g., Figure 1 In some embodiments, the method 600 is implemented by: Figure 4 A sub-method of operation 408 in FIG.

[0050] Operation 602 includes thresholding the heat map. The threshold may be a dynamically calculated threshold. Thus, operation 602 may generate a binary image in which each pixel represents one value or another (e.g., black or white). Operation 602 may generate a binary image such as Figure 3G Image shown.

[0051] Operation 604 includes dilating and / or eroding the threshold heat map to eliminate any residual screen door effect. Dilation can refer to thickening the edge of the image (e.g., changing the pixel color of pixels near pixels of a predetermined color (e.g., white) to the same color). Erosion can refer to shrinking the edge of the image (e.g., changing the pixel color of pixels near pixels of a predetermined color (e.g., white) to another color (e.g., black). Residual screen door effect (also known as image snow) refers to incorrect pixel coloring throughout the image, which can reduce image clarity (e.g., white pixels are surrounded by black pixels, or vice versa). Dilation and / or erosion of the threshold heat map can improve analysis of the image in future operations by reducing or eliminating any residual screen door effect.

[0052] Operation 606 includes partitioning the thresholded heat map. Operation 606 may partition the heat map into a number of small portions (e.g., blocks). Operation 606 may partition the heat map by creating radial lines starting from the center of the image and radiating outward at a predetermined angle. Operation 606 may generate a Figure 3H and / or the image shown in 3I.

[0053] Operation 608 includes filtering the blocks of the thresholded heat map. Filtering can remove abnormal (e.g., outliers) or other useless blocks. For example, operation 608 can remove blocks that are too small or are more than a distance threshold from the centroid of the block. Operation 608 can generate Figure 3J Image shown.

[0054] Operation 610 includes fitting an ellipse to the pixels associated with the remaining block of the threshold heat map. Operation 610 may involve extending or connecting the various portions of the thresholded heat map together to form a complete ellipse. Operation 610 may generate Figure 3K Image shown.

[0055] Operation 612 includes calculating ellipse parameters of the ellipse. The ellipse parameters may include, but are not limited to, the ellipse center, axis lengths (e.g., half lengths of the major axis and / or the minor axis), angles of the major axis (e.g., the major axis), and / or other ellipse parameters (e.g., focus, eccentricity, axis intersections (e.g., apex), angles of the minor axis (e.g., the minor axis), etc.).

[0056] Figure 7 A flowchart of an example method 700 for downloading, deploying, metering, and billing tilt image correction code according to some embodiments of the present disclosure is shown. In some embodiments, the method 700 is performed by a computer, a server (e.g., Figure 1 Server 102 in, device (e.g., Figure 1 In some embodiments, the method 700 is implemented by: Figure 4 One or more operations of method 400 occur simultaneously.

[0057] Operation 702 includes downloading skew image correction code (e.g., Figure 1 The tilted image correction software 104 in the embodiment of the present invention is downloaded to one or more computers (e.g., server 102). Operation 704 includes executing the tilted image correction code. Operation 704 may include executing any of the methods and / or functions described herein. Operation 706 includes metering the usage of the tilted image correction code. The usage may be metered in the following ways: for example, the amount of time the tilted image correction code is used, the number of servers and / or devices on which the tilted image correction code is deployed, the amount of resources consumed by executing the tilted image correction code, the number of tilted images processed by executing the tilted image correction code and / or the number of corrected images generated, and / or other usage measurement indicators. Operation 708 includes generating an invoice based on the usage measurement.

[0058] Various aspects of the present disclosure are described by narrative text, flow charts, block diagrams of computer systems, and / or block diagrams of machine logic contained in computer program product (CPP) embodiments. For any flow chart, depending on the technology involved, the operations may be performed in an order different from the order shown in a given flow chart. For example, also depending on the technology involved, two operations shown in consecutive flow chart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that overlaps at least partially in time.

[0059] Computer program product embodiments ("CPP embodiments" or "CPP") are terms used in this disclosure to describe one or more sets of storage media (also referred to as "media") that are collectively contained in one or more sets of storage devices that collectively contain machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" is any tangible device that can retain and store instructions for use by a computer processor. A computer-readable storage medium may be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: floppy disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanical encoding devices (such as punch cards or pits / land formations on a major surface of a disc), or any suitable combination of the foregoing. Computer-readable storage media (as that term is used in this disclosure) should not be construed as storing in the form of transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides, light pulses through fiber optic cables, electrical signals transmitted through wires, and / or other transmission media. Those skilled in the art will understand that data is typically moved at certain occasional points in time during normal operation of the storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device temporary because data is not temporary when stored.

[0060] Figure 8A block diagram of an example computing environment according to some embodiments of the present disclosure is shown. The computing environment 800 includes an example of an environment for executing at least part of the computer code involved in the method of the present invention (e.g., the tilt image correction code 846). In addition to the tilt image correction code 846, the computing environment 800 also includes, for example, a computer 801, a wide area network (WAN) 802, an end user device (EUD) 803, a remote server 804, a public cloud 805, and a private cloud 806. In this embodiment, the computer 801 includes a processor group 810 (including a processing circuit 820 and a cache 821), a communication structure 811, a volatile memory 812, a persistent storage device 813 (including an operating system 822 and the tilt image correction code 846 as described above), a peripheral device group 814 (including a user interface (UI), a device group 823, a storage device 824, and an Internet of Things (IoT) sensor group 825), and a network module 815. The remote server 804 includes a remote database 830. The public cloud 805 includes a gateway 840 , a cloud orchestration module 841 , a host physical machine group 842 , a virtual machine group 843 , and a container group 844 .

[0061] Computer 801 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smart watch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or developed in the future that is capable of running programs, accessing a network, or querying a database, such as remote database 830. As is well known in the art of computer technology, and depending on the technology, the execution of the computer-implemented method may be distributed among multiple computers and / or among multiple locations. On the other hand, in this introduction to computing environment 800, the detailed discussion focuses on a single computer, particularly computer 801, to keep the introduction as simple as possible. Computer 801 may be located in the cloud, even if it is not hosted on a server. Figure 8 On the other hand, unless explicitly stated, computer 801 need not be located in the cloud.

[0062] Processor group 810 includes one or more computer processors of any type now known or developed in the future. Processing circuit 820 can be distributed over multiple packages, such as multiple coordinated integrated circuit chips. Processing circuit 820 can implement multiple processor threads and / or multiple processor cores. Cache 821 is a memory located in the processor chip package, typically used for data or code that should be quickly accessed by threads or cores running on processor group 810. Cache memory is typically organized into multiple levels based on relative proximity to the processing circuit. Alternatively, part or all of the cache of the processor group can be located "off chip". In some computing environments, processor group 810 can be designed to process quantum bits and perform quantum computing.

[0063] Computer readable program instructions are typically loaded onto computer 801 to cause processor group 810 of computer 801 to perform a series of operating steps to implement a computer-implemented method, such that the instructions executed will instantiate the method specified in the flowchart and / or narrative description of the computer-implemented method contained in this document (collectively referred to as the "method of the present invention"). These computer readable program instructions are stored in various types of computer readable storage media, such as cache 821 and other storage media discussed below. Processor group 810 accesses program instructions and related data to control and direct the execution of the method of the present invention. In computing environment 800, at least a portion of the instructions for executing the method of the present invention can be stored in tilt image correction code 846 in persistent storage device 813.

[0064] The communication fabric 811 is the signaling paths that allow the various components of the computer 801 to communicate with each other. Typically, the fabric consists of switches and conductive paths, such as those that form a bus, a bridge, physical input / output ports, etc. Other types of signal communication paths may be used, such as fiber optic communication paths and / or wireless communication paths.

[0065] The volatile memory 812 is any type of volatile memory currently known or developed in the future. Examples include dynamic type random access memory (RAM) or static type RAM. Typically, volatile memory is characterized by random access, but this is not required unless explicitly stated. In the computer 801, the volatile memory 812 is located in a single package and is internal to the computer 801, but, as an alternative or in addition, the volatile memory can be distributed over multiple packages and / or located external to the computer 801.

[0066] The persistent storage device 813 is any form of non-volatile storage device for a computer that is currently known or developed in the future. The non-volatility of such storage means that the stored data will be maintained regardless of whether the computer 801 is powered on and / or power is directly supplied to the persistent storage device 813. The persistent storage device 813 can be a read-only memory (ROM), but usually at least a portion of the persistent storage device allows data to be written, deleted, and rewritten. Some common forms of persistent storage devices include disks and solid-state storage devices. The operating system 822 can take a variety of forms, such as various known proprietary operating systems or open source portable operating system interface type operating systems using a kernel. The code contained in the tilt image correction code 846 usually contains at least a portion of the computer code involved in executing the method of the present invention.

[0067] The peripheral device group 814 includes a peripheral device group of the computer 801. The data communication connection between the peripheral device and other components of the computer 801 can be implemented in various ways, such as a Bluetooth connection, a near field communication (NFC) connection, a connection established by a cable (such as a universal serial bus (USB) type cable), a plug-in type connection (such as a secure digital (SD) card), a connection established by a local area network, and even a connection established by a wide area network such as the Internet. In various embodiments, the UI device group 823 may include components such as display screens, speakers, microphones, wearable devices (such as goggles and smart watches), keyboards, mice, printers, touchpads, game controllers, and tactile devices. The storage device 824 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. The storage device 824 can be persistent and / or volatile. In some embodiments, the storage device 824 can take the form of a quantum computing storage device for storing data in the form of quantum bits. In embodiments where it is necessary for computer 801 to have a large amount of storage (e.g., computer 801 locally stores and manages a large database), this storage may be provided by a peripheral storage device designed to store large amounts of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. IoT sensor group 825 consists of sensors that can be used in IoT applications. For example, one sensor may be a thermometer and another sensor may be a motion detector.

[0068] The network module 815 is a collection of computer software, hardware, and firmware that allows the computer 801 to communicate with other computers via the WAN 802. The network module 815 may include hardware, such as a modem or Wi-Fi signal transceiver, software for grouping and / or unpacking data for communication network transmission, and / or Web browser software for communicating data via the Internet. In some embodiments, the network control function and network forwarding function of the network module 815 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software defined networks (SDN)), the control function and forwarding function of the network module 815 are executed on physically separate devices, so that the control function manages several different network hardware devices. Computer-readable program instructions for executing the method of the present invention can usually be downloaded to the computer 801 from an external computer or an external storage device via a network adapter card or a network interface included in the network module 815.

[0069] WAN 802 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, currently known or developed in the future. In some embodiments, the WAN may be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. A WAN and / or LAN typically includes computer hardware, such as copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0070] End-user device (EUD) 803 is any computer system used and controlled by an end-user (e.g., a customer of an enterprise operating computer 801), and may take any of the forms described above in relation to computer 801. EUD 803 typically receives helpful and useful data from the operation of computer 801. For example, assuming that computer 801 is designed to provide advice to an end-user, the advice would typically be communicated to EUD 803 from network module 815 of computer 801 via WAN 802. In this way, EUD 803 may display or otherwise present the advice to the end-user. In some embodiments, EUD 803 may be a client device, such as a thin client, a heavy client, a mainframe computer, a desktop computer, etc.

[0071] Remote server 804 is any computer system that provides at least some data and / or functionality to computer 801. Remote server 804 may be controlled and used by the same entity that operates computer 801. Remote server 804 represents a machine that collects and stores helpful and useful data for use by other computers, such as computer 801. For example, assuming that computer 801 is designed and programmed to provide recommendations based on historical data, the historical data may be provided to computer 801 from remote database 830 of remote server 804.

[0072] The public cloud 805 is any computer system available to multiple entities that provides on-demand availability of computer system resources and / or other computer capabilities, particularly data storage (cloud storage) and computing capabilities, without direct active management by users. Cloud computing typically utilizes resource sharing to achieve consistency and economies of scale. The direct and active management of the computing resources of the public cloud 805 is performed by computer hardware and / or software of the cloud orchestration module 841. The computing resources provided by the public cloud 805 are typically implemented by a virtual computing environment that runs on various computers that constitute the host physical machine group 842, which is a physical computer world in and / or available to the public cloud 805. A virtual computing environment (VCE) is typically in the form of a virtual machine from a virtual machine group 843 and / or a container from a container group 844. It will be appreciated that these VCEs can be stored as images and can be transferred between various physical machine hosts, either as images or after the VCE is instantiated. Cloud orchestration module 841 manages the transfer and storage of images, deploys new instances of VCE, and manages active instances of VCE deployments. Gateway 840 is a collection of computer software, hardware, and firmware that allows public cloud 805 to communicate over WAN 802 .

[0073] The Virtualized Computing Environment (VCE) will now be explained further. A VCE can be stored as an "image". A new active instance of a VCE can be instantiated from an image. Two common types of VCEs are virtual machines and containers. Containers are VCEs that use operating system-level virtualization. This refers to an operating system feature where the kernel allows the existence of multiple isolated user space instances, called containers. From the perspective of the programs running in them, these isolated user space instances generally behave like real computers. Computer programs running on a normal operating system can utilize all of the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, programs running within a container can only use the contents of the container and the devices assigned to the container, a feature called containerization.

[0074] Private cloud 806 is similar to public cloud 805, except that the computing resources are only available to a single enterprise. Although private cloud 806 is depicted as communicating with WAN 802, in other embodiments, the private cloud may be completely disconnected from the Internet and only accessible through a local / private network. A hybrid cloud is a combination of multiple different types of clouds (e.g., private, community, or public cloud types), typically implemented by different vendors. Each of the multiple clouds remains an independent and discrete entity, but the larger hybrid cloud architecture is tied together through standardized or proprietary technologies to enable orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, both public cloud 805 and private cloud 806 are part of a larger hybrid cloud.

[0075] The flow chart and block diagram in the figure show the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each block in the flow chart or block diagram can represent a subset of a module, segment or instruction, which includes one or more executable instructions for implementing a specified logical function.In some alternative implementations, the function indicated in the block may not occur in the order indicated in the figure.For example, two blocks displayed continuously can actually be executed substantially at the same time, or sometimes blocks can be executed in reverse order, depending on the functions involved.It should also be noted that each block in the block diagram and / or flow chart and the combination of the blocks in the block diagram and / or flow chart can be implemented by a dedicated hardware system that performs a specified function or action or performs a combination of dedicated hardware and computer instructions.

[0076] Although it is understood that the process software (e.g., any software configured to perform any part of the foregoing method and / or implement any of the foregoing functions) can be deployed by loading a storage medium such as a CD, DVD, etc. directly manually onto the client, server, and agent computers, it can also be automatically or semi-automatically deployed to the computer system by sending the process software to a central server or a group of central servers. The process software is then downloaded to the client computer that will execute the process software. Alternatively, the process software is sent directly to the client system via email. The process software can then be separated into directories or loaded into directories by executing a set of program instructions that separate the process software into directories. Another option is to send the process software directly to a directory on the client computer's hard disk. When a proxy server is present, the process will select the proxy server code, determine which computers to place the proxy server code on, transmit the proxy server code, and then install the proxy server code on the agent computer. The process software will be transmitted to the proxy server and then stored on the proxy server.

[0077] Embodiments of the present invention may also be delivered as part of a service engagement with a client company, non-profit organization, government entity, internal organizational structure, etc. These embodiments may include configuring a computer system to execute and deploy software, hardware, and network services that implement some or all of the methods described herein. These embodiments may also include analyzing the client's operations, creating recommendations based on the analysis, building a system to implement a subset of the recommendations, integrating the system into existing processes and infrastructure, metering usage of the system, allocating charges to users of the system, and billing, invoicing (e.g., generating invoices), or otherwise receiving charges for use of the system.

[0078] The terms used herein are only used to describe specific embodiments and are not intended to limit various embodiments. The singular forms "one", "an" and "the" used herein also include plural forms unless the context clearly indicates otherwise. It should also be understood that the terms "including" and / or "comprising" used in this specification specify the presence of the features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof. In the previous detailed description of the exemplary embodiments of various embodiments, reference is made to the accompanying drawings (where the same numbers represent the same elements), which constitute a part of this document and illustrate specific exemplary embodiments in which various embodiments can be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice these embodiments, but other embodiments can be used and logical, mechanical, electrical and other changes can be made without departing from the scope of the various embodiments. In the previous description, many specific details are set forth in order to fully understand the various embodiments. However, even without these specific details, the various embodiments can be practiced. In other cases, in order to avoid confusing the embodiments, well-known circuits, structures and technologies are not shown in detail.

[0079] Different uses of the term "embodiment" as used in this specification do not necessarily refer to the same embodiment, but may refer to the same embodiment. Any data and data structures shown or described herein are merely examples, and in other embodiments, different data amounts, data types, fields, number and type of fields, field names, number and type of rows, records, entries, or data organizations may be used. In addition, any data can be combined with logic, so a separate data structure may not be required. Therefore, the foregoing detailed description should not be considered restrictive.

[0080] The description of various embodiments of the present disclosure is for illustrative purposes only and is not intended to be exhaustive or limited to the disclosed embodiments. Those skilled in the art will be able to make many modifications and variations without departing from the scope and spirit of the described embodiments. The terms used herein are intended to better explain the principles of the embodiments, practical applications, or technical improvements over the prior art on the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

[0081] Although the present disclosure has been described by specific embodiments, it is foreseeable that changes and modifications thereof will be apparent to those skilled in the art. Therefore, the following claims are intended to cover all changes and modifications that fall within the true spirit and scope of the present disclosure.

[0082] Any advantages discussed in the present disclosure are exemplary advantages, and embodiments of the present disclosure may achieve all, some, or none of the advantages discussed while remaining within the spirit and scope of the present disclosure.

[0083] A series of non-limiting examples are provided below to illustrate certain aspects of the present disclosure. Example 1 is a computer-implemented method. The method includes: receiving an original image depicting an oblique view of a circular object; preprocessing the original image into an edge image; generating a heat map including an ellipse formed by the oblique view of the circular object based on the edge image through a machine learning model; calculating ellipse parameters of the ellipse describing the heat map; and performing an affine transformation on the original image using the ellipse parameters to generate a corrected image, wherein the corrected image converts the ellipse into a circle.

[0084] Example 2 includes the features described in Example 1. In this example, the circular object is an analog meter. Optionally, the ellipse is defined by a plurality of dial scales on the analog meter.

[0085] Example 3 includes the features of any one of Examples 1 to 2, including or excluding optional features. In this example, the ellipse parameters include a center of the ellipse, a semi-major axis of the ellipse, and an angle of a major axis of the ellipse.

[0086] Example 4 includes the features of any one of Examples 1 to 3, including or excluding optional features. In this example, the oblique view of the circular object makes the circular object appear elliptical in the original image, and wherein the corrected image makes the circular object appear circular.

[0087] Example 5 includes the features of any one of Examples 1 to 4, but includes or excludes optional features. In this example, the machine learning model is a fully convolutional neural network (FCNN).

[0088] Example 6 includes the features of any one of Examples 1 to 5, but includes or excludes the optional features. In this example, the preprocessing includes converting the original image to grayscale.

[0089] Example 7 includes the features of any one of Examples 1 to 6, including or excluding the optional features. In this example, the preprocessing includes: cropping the original image according to the circular object; and adjusting the cropped image to a standard size.

[0090] Example 8 includes the features of any one of Examples 1 to 7, including or excluding the optional features. In this example, the preprocessing includes blurring the edge image.

[0091] Example 9 includes the features of any one of Examples 1 to 8, including or excluding the optional features. In this example, the method further includes performing a dilation operation and an erosion operation on the heat map.

[0092] Example 10 includes the features of any one of Examples 1 to 9, including or excluding the optional features. In this example, the method further includes: dividing the heat map into a plurality of blocks; and filtering the plurality of blocks.

[0093] Example 11 includes the features of any one of Examples 1 to 10, but includes or excludes the optional features. In this example, the edge image is a Laplacian image.

[0094] Example 12 includes the features of any one of Examples 1 to 11, including or excluding the optional features. In this example, the method is performed by a server that executes tilted image correction software. Optionally, the method also includes metering usage of the tilted image correction software; and generating an invoice based on the metering of usage of the tilted image correction software.

[0095] Example 13 is a system. The system includes: one or more computer-readable storage media storing program instructions; and one or more processors, which are configured to perform the method according to any one of Examples 1 to 12, including or excluding optional features, in response to executing the program instructions.

[0096] Example 14 is a computer program product. The computer program product includes one or more computer-readable storage media, and program instructions stored together on the one or more computer-readable storage media, the program instructions including instructions configured to cause one or more processors to perform the method according to any one of Examples 1 to 12, including or excluding optional features.

Claims

1. A computer-implemented method, comprising: receiving an original image depicting an oblique view of a circular object; preprocessing the original image into an edge image; generating, based on the edge image, a heat map including an ellipse formed by the oblique view of the circular object through a machine learning model; calculating ellipse parameters of the ellipse describing the heat map; and performing an affine transformation on the original image using the ellipse parameters to generate a corrected image, wherein the corrected image converts the ellipse into a circle.

2. The method according to claim 1, wherein, the circular object is an analog meter.

3. The method according to claim 2, wherein, the ellipse is defined by a plurality of dial graduations on the analog meter.

4. The method according to claim 1, wherein, the ellipse parameters include: the center of the ellipse, the semi-major axis of the ellipse, and the angle of the major axis of the ellipse.

5. The method according to claim 1, wherein, the oblique view of the circular object causes the circular object to appear elliptical in the original image, and wherein the corrected image causes the circular object to appear circular.

6. The method according to claim 1, wherein, the machine learning model is a fully convolutional neural network (FCNN).

7. The method according to claim 1, the preprocessing comprising: converting the original image to grayscale.

8. The method according to claim 1, the preprocessing comprising: cropping the original image according to the circular object; and adjusting the cropped image to a standard size.

9. The method according to claim 1, the preprocessing comprising: blurring the edge image.

10. The method according to claim 1, further comprising: performing a dilation operation and an erosion operation on the heat map.

11. The method according to claim 1, further comprising: dividing the heat map into a plurality of blocks; and filtering the plurality of blocks.

12. The method according to claim 1, wherein, the edge image is a Laplacian image.

13. The method according to claim 1, wherein, the method is executed by a server implementing tilt image correction software, and wherein the method further comprises: metering the usage of the tilt image correction software; and generating an invoice based on the metering of the usage of the tilt image correction software.

14. A system, comprising: one or more computer-readable storage media storing program instructions; and one or more processors configured to execute a method in response to executing the program instructions, the method comprising: receiving an original image depicting an oblique view of a circular object; preprocessing the original image into an edge image; generating, based on the edge image, a heat map including an ellipse formed by the oblique view of the circular object through a machine learning model; calculating ellipse parameters of the ellipse describing the heat map; and performing an affine transformation on the original image using the ellipse parameters to generate a corrected image, wherein the corrected image converts the ellipse into a circle.

15. The system according to claim 14, wherein, The circular object is an analog meter, wherein the ellipse is defined by a plurality of scale graduations on the analog meter, and wherein the ellipse parameters include: the center of the ellipse, the semi-major axis of the ellipse, and the angle of the major axis of the ellipse.

16. The system according to claim 14, wherein, the perspective view of the circular object causes the circular object to appear elliptical in the original image, and wherein the corrected image causes the circular object to appear circular.

17. The system according to claim 14, wherein, the machine learning model is a fully convolutional neural network (FCNN).

18. The system according to claim 14, the preprocessing comprises: cropping the original image according to the circular object; adjusting the cropped image to a standard size; converting the original image to grayscale; and blurring the edge image.

19. The system according to claim 14, the method further comprises: performing dilation operations and erosion operations on the heat map; dividing the heat map into a plurality of blocks; and filtering the plurality of blocks.

20. A computer program product, comprising one or more computer-readable storage media, and program instructions jointly stored on the one or more computer-readable storage media, the program instructions comprising instructions configured to cause one or more processors to execute a method, the method comprises: receiving an original image depicting a perspective view of a circular object; preprocessing the original image into an edge image; generating, based on the edge image, a heat map including an ellipse formed by the perspective view of the circular object through a machine learning model; calculating ellipse parameters of the ellipse describing the heat map; and performing an affine transformation on the original image using the ellipse parameters to generate a corrected image, wherein the corrected image converts the ellipse into a circle.

Images