Image correction processing method and device, and electronic equipment

Abstract

The present disclosure provides a method and device for correcting an image, and an electronic device, and relates to the field of image processing technology. The method for correcting an image includes: obtaining the contour straight line segments of an object in an image to obtain a first straight line segment set; determining the target contour straight line segments in the first straight line segment set according to the shape characteristics of the target object; obtaining key points related to the target contour straight line segments, and determining a mapping relationship according to the current position information of the key points and the target position information after correction processing; performing a perspective transformation on the image according to the mapping relationship, so that the target object in the corrected image is basically perpendicular to the user's perspective. This technical solution can correct the quality of images generated by oblique shooting in batches, so that users can quickly and easily obtain information about target objects in the image.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular, to an image correction processing method, an image correction processing device, and an electronic device for implementing the image correction processing method. Background Art

[0002] During the image acquisition process, due to objective reasons (such as geographical location restrictions, the target being too high, etc.) or subjective reasons (such as the negligence of the acquisition personnel), the photographer may shoot the target at an angle, so the captured image is not perpendicular to the user's perspective, resulting in low image quality and even invalid images.

[0003] In the related art, by strengthening the operation, the photographers are educated to shoot with the lens facing the target, thereby reducing the generation of oblique images at the source. For the oblique images that have been generated, the images can be corrected one by one through image processing software (such as Photoshop, etc.).

[0004] However, the solutions provided by the related art have the problem of low processing efficiency.

[0005] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0006] The purpose of the present disclosure is to provide an image correction processing method, an image correction processing device, and an electronic device, thereby overcoming the problem of low efficiency of image correction processing in related technologies to a certain extent.

[0007] According to a first aspect of the present disclosure, a method for correcting an image is provided, the method comprising: obtaining contour straight line segments of an object in an image to obtain a first straight line segment set; determining target contour straight line segments in the first straight line segment set according to shape features of the target object; obtaining key points related to the target contour straight line segments, and determining a mapping relationship according to current position information of the key points and target position information after correction processing; and, according to the mapping relationship, performing a perspective transformation on the image so that the target object in the corrected image is substantially perpendicular to the user's viewing angle.

[0008] In an exemplary embodiment of the present disclosure, the method of obtaining the contour straight line segments of an object in an image based on the aforementioned embodiment includes: preprocessing the image to obtain a preprocessed image; and obtaining the contour straight line segments of the object in the preprocessed image through straight line detection.

[0009] In an exemplary embodiment of the present disclosure, based on the aforementioned embodiment, according to the shape features of the target object, the target contour straight line segments are determined in the above-mentioned first straight line segment set, including: processing the contour straight line segments in the above-mentioned first straight line segment set by a sliding window method; and, determining the target window according to the shape features of the target object, and obtaining at least two target contour straight line segments in the second contour set corresponding to the above-mentioned target window.

[0010] In an exemplary embodiment of the present disclosure, based on the above embodiment, before determining the target contour straight line segments, the method further includes: screening the first straight line segment set according to the position features of the target object in the image.

[0011] In an exemplary embodiment of the present disclosure, the key points related to the above-mentioned target contour straight line segment are obtained based on the aforementioned embodiment, including: determining two side lines parallel to the edge of the above-mentioned image; and determining the intersection of the straight line where the above-mentioned target contour straight line segment is located and the above-mentioned two side lines as the above-mentioned key point.

[0012] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the two side lines are the left side line of the image and the right side line of the image.

[0013] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, determining the mapping relationship according to the current position information of the key point and the corrected target position information includes:

[0014] In the same coordinate system, obtaining the current position coordinates of the key point in the image, and determining the target position coordinates of the key point in the image;

[0015] The mapping relationship is determined based on the current position coordinates and the target position coordinates.

[0016] In an exemplary embodiment of the present disclosure, based on the above embodiment and according to the above mapping relationship, perspective transformation is performed on the above image, including:

[0017] The pixel position information in the above image is used as the original coordinates, and the above original coordinates are converted into target coordinates according to the above mapping relationship.

[0018] According to a second aspect of the present disclosure, there is provided an image correction processing device, the device comprising: a contour line acquisition module, configured to acquire contour straight line segments of an object in an image to obtain a first straight line segment set; a target contour line determination module, configured to determine the target contour straight line segments in the first straight line segment set according to shape features of the target object; a mapping relationship determination module, configured to acquire key points related to the target contour straight line segments, and determine a mapping relationship according to current position information of the key points and target position information after correction processing; and a transformation module, configured to perform a perspective transformation on the image according to the mapping relationship, so that the target object in the corrected image is substantially perpendicular to the user's viewing angle.

[0019] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned contour straight line segment acquisition module includes: a preprocessing unit and a straight line detection unit. Among them:

[0020] The preprocessing unit is configured to: preprocess the image to obtain a preprocessed image; the straight line detection unit is configured to: obtain the contour straight line segments of the object in the preprocessed image through straight line detection.

[0021] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned target contour straight line segment determination module includes: a sliding window processing unit and a target window determination unit. Among them:

[0022] The above-mentioned sliding window processing unit is configured to: process the contour straight line segments in the above-mentioned first straight line segment set by means of a sliding window; the above-mentioned target window determination unit is configured to: determine the target window according to the shape characteristics of the target object, and obtain at least two target contour straight line segments in the second contour set corresponding to the above-mentioned target window.

[0023] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the image correction processing device further includes: a straight line segment screening module. Among them:

[0024] The straight line segment screening module is configured to screen the first straight line segment set according to the position features of the target object in the image before the sliding window processing unit determines the target contour straight line segment.

[0025] In an exemplary embodiment of the present disclosure, based on the aforementioned embodiment, the mapping relationship determination module is specifically configured to: determine two side lines parallel to the edge of the image; and determine the intersection of the straight line where the target contour straight line segment is located and the two side lines as the key point.

[0026] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the two side lines are the left side line of the image and the right side line of the image.

[0027] In an exemplary embodiment of the present disclosure, based on the aforementioned embodiment, the above-mentioned mapping relationship determination module is also specifically configured to: in the same coordinate system, obtain the current position coordinates of the above-mentioned key point in the above-mentioned image, and determine the target position coordinates of the above-mentioned key point in the above-mentioned image; determine the above-mentioned mapping relationship based on the above-mentioned current position coordinates and the above-mentioned target position coordinates.

[0028] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned transformation module is specifically configured to: use the pixel position information in the above-mentioned image as the original coordinates, and convert the above-mentioned original coordinates into target coordinates according to the above-mentioned mapping relationship.

[0029] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the image correction processing method described in any embodiment of the first aspect is implemented.

[0030] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the image correction processing method of any embodiment in the first aspect by executing the executable instructions.

[0031] The exemplary embodiments of the present disclosure may have some or all of the following beneficial effects:

[0032] In the image correction processing method provided in an example embodiment of the present disclosure, firstly, the contour straight line segments of the object in the image are obtained, and then, according to the shape features of the target object (such as the part of interest in the image), the target contour straight line segments are determined in the above-mentioned first straight line segment set. Furthermore, key points related to the above-mentioned target contour straight line segments are obtained, and a mapping relationship is determined according to the current position information of the key points and the target position information after the correction processing; finally, according to the mapping relationship, the image is perspective transformed so that the above-mentioned target object in the corrected image is basically perpendicular to the user's perspective. This technical solution can correct the quality of images generated by oblique shooting in batches, so that users can quickly and easily obtain information about target objects in the image.

[0033] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0035] Figure 1 A schematic diagram showing an exemplary system architecture of an image correction processing method and device to which the embodiments of the present disclosure can be applied;

[0036] Figure 2 A flowchart of an image correction processing method according to an embodiment of the present disclosure is schematically shown;

[0037] Figure 3 An image to be corrected according to an embodiment of the present disclosure is schematically shown;

[0038] Figure 4 A flowchart of a straight line segment acquisition method according to an embodiment of the present disclosure is schematically shown;

[0039] Figure 5 A schematic diagram of a contour straight line segment in an embodiment of the present disclosure is schematically shown;

[0040] Figure 6 A schematic diagram of a method for determining an attribute value of a contour straight line segment in another embodiment of the present disclosure is schematically shown;

[0041] Figure 7 A flowchart of a method for acquiring a straight line segment of a target contour according to an embodiment of the present disclosure is schematically shown;

[0042] Figure 8 A schematic diagram schematically illustrates a method for determining key points according to an embodiment of the present disclosure;

[0043] Fig. 9 A flowchart schematically shows a method for determining key points according to an embodiment of the present disclosure;

[0044] Fig.10 Schematically shows an image after correction processing according to an embodiment of the present disclosure;

[0045] Fig.11 The structure of an image correction processing device according to an embodiment of the present disclosure is schematically shown;

[0046] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0047] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0048] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0049] Figure 1 A schematic diagram of a system architecture of an exemplary application environment in which an image correction processing method and device according to an embodiment of the present disclosure can be applied is shown.

[0050] like Figure 1 As shown, the system architecture 100 may include one or more of terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc. The terminal devices 101, 102, 103 may be various electronic devices with display screens, including but not limited to desktop computers, portable computers, smart phones, tablet computers, etc. It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. According to the implementation requirements, there may be any number of terminal devices, networks and servers. For example, the server 105 may be a server cluster composed of multiple servers.

[0051] The image correction processing method provided in the embodiment of the present disclosure is generally executed by the server 105, and accordingly, the image correction processing device is generally set in the server 105. However, it is easy for those skilled in the art to understand that the image correction processing method provided in the embodiment of the present disclosure can also be executed by the terminal devices 101, 102, and 103, and accordingly, the image correction processing device can also be set in the terminal devices 101, 102, and 103, which is not particularly limited in this exemplary embodiment.

[0052] For example, in an exemplary embodiment, the terminal devices 101, 102, and 103 may obtain the image to be processed and send it to the server 105, so that the server 105 obtains the contour straight line segments of the object contained in the image in the above image to obtain a first straight line segment set; then, the server 105 determines the target contour straight line segment in the above first straight line segment set according to the shape characteristics of the target object. Further, the server 105 obtains the key points related to the above target contour straight line segment, and determines the mapping relationship according to the current position information of the key points and the target position information after correction processing; the server 105 then performs perspective transformation on the entire image according to the above mapping relationship. Exemplarily, in the corrected image, the target object is substantially perpendicular to the user's perspective, and the server 105 can also send the corrected image to the terminal devices 101, 102, and 103, so that the user can conveniently view the corrected image through the terminal devices 101, 102, and 103.

[0053] For example, one use scenario may be to enrich the geographic information system by collecting street view images. Specifically, street view images may be collected by crowdsourcing, and point of interest (POI) information may be obtained through the street view images. Then, the geographic information system may be updated or enriched based on the POI information, so that the geographic information system is closer to reality and more convenient for users to use.

[0054] The crowdsourcing method is a practice in which a company or organization outsources work tasks that were previously performed by employees to non-specific (and usually large) volunteers on a voluntary basis. The POI can be a house, a shop, a mailbox, a bus stop, etc.

[0055] For example, let's take a POI as a shop sign. For an image generated by oblique shooting, the POI shop sign will not be perpendicular to the observer (for example, tilted or rotated). As a result, the POI shop sign in the image is greatly deformed, and it is difficult for POI reviewers to determine the text on the shop sign, which greatly reduces the quality of the collected photos and may even produce invalid images.

[0056] This technical solution aims to solve the problem of images generated by oblique shooting, and correct the images generated by oblique shooting so that the POI in the image is projected onto the user's visual plane. Thereby, the image quality is improved and the proportion of effective images is increased. Furthermore, post-processing (such as image recognition, etc.) can be performed based on the corrected images, which is conducive to improving the efficiency of post-processing.

[0057] The technical solution of the embodiment of the present disclosure is described in detail below:

[0058] Figure 2 The flowchart of the image correction processing method according to an embodiment of the present disclosure is schematically shown. Figure 2 , the embodiment shown in the figure includes:

[0059] Step S210, obtaining the contour straight line segments of the object in the image to obtain a first straight line segment set;

[0060] Step S220, determining a target contour straight line segment in the first straight line segment set according to the shape feature of the target object;

[0061] Step S230, acquiring key points related to the target contour straight line segment, and determining a mapping relationship according to current position information of the key points and corrected target position information; and

[0062] Step S240: performing perspective transformation on the image according to the mapping relationship, so that the target object in the corrected image is substantially perpendicular to the user's viewing angle.

[0063] It should be noted that the angle between the target object in the corrected image and the user's viewing angle is in the range of [90-l, 90+j] degrees, where l and j are both small positive values, such as 5 or 10.

[0064] exist Figure 2 In the technical solution provided by the illustrated embodiment, the contour straight line segments of the object in the image are first obtained, and then, according to the shape features of the target object (e.g., the part of interest in the image), the target contour straight line segments are determined in the above-mentioned first straight line segment set. Furthermore, key points related to the target contour straight line segments are obtained, and a mapping relationship is determined according to the current position information of the key points and the target position information after correction processing; finally, according to the mapping relationship, the image is perspective transformed so that the target object in the corrected image is basically perpendicular to the user's perspective. This technical solution can batch correct the quality of images generated by oblique shooting, so that users can quickly and easily obtain information about target objects in the image.

[0065] The following Figure 2 The specific implementation methods of each step in the embodiment shown are described in detail:

[0066] In an exemplary embodiment, the image in step S210 may be an image containing obliquely photographed POIs. In order to facilitate the counting of POIs in the image, the image is corrected by the present technical solution. Figure 3 , the POI in the image is a shop sign as an example. For example, the image contains multiple objects, such as walls, steps, floor tiles, etc. Each object is composed of its own contour line. In this embodiment, the image correction processing is displayed based on the contour line of the object in the image. Specifically:

[0067] In step S210, the contour straight line segments of the object in the image are obtained to obtain a first straight line segment set. Figure 4 The flowchart of the method for obtaining a straight line segment according to an embodiment of the present disclosure is schematically shown. Figure 4 , the embodiment shown in the figure includes steps S410 to S430.

[0068] In step S410, a grayscale image of the original image is obtained, and a Gaussian filter is performed on the grayscale image to obtain a blurred image.

[0069] In an exemplary embodiment, after obtaining the rectified image, Gaussian filtering is first performed to extract features (such as edge texture features, etc.) for identifying each object in the image. Exemplarily, high-frequency signals such as edge and texture details in the image can be obtained by high-pass filtering.

[0070] In step S420, edge detection and dilation processing are performed on the blurred image to obtain a pre-processed image.

[0071] In an exemplary embodiment, edge detection and dilation processing are performed on the image after Gaussian filtering, thereby obtaining a pre-processed image.

[0072] The corrected image is preprocessed through steps S410 and S420 to improve the accuracy of obtaining the contour straight line segments.

[0073] In step S430, the contour straight line segments of the object in the preprocessed image are acquired by straight line detection.

[0074] In an exemplary embodiment, reference Figure 5 , the contour straight line segments of the object in the preprocessed image can be detected by Hough transform to obtain a first straight line segment set. Exemplarily, in the first straight line segment set, each contour straight line segment includes the following attribute values: the angle value of the straight line segment in the image, and the position information of the straight line segment in the image. Specifically, refer to Figure 6, a coordinate system XOY is determined based on the upper left corner of the image 600 and the two sides with the upper left corner as the intersection point.

[0075] According to step S430, the contour straight line segment 61 and the contour straight line segment 62 are detected in the image 600. The attribute value of the contour straight line segment 61 includes: the angle value N presented in the image 600, and the position information in the image 600 (e.g., the midpoint coordinate A1 (x1, y1) of the contour straight line segment 61). Similarly, the attribute value of the contour straight line segment 62 includes: the angle value M presented in the image 500, and the position information in the image 600 (e.g., the midpoint coordinate A2 (x2, y2) of the contour straight line segment 62).

[0076] Continue to refer Figure 2 After determining the above-mentioned first set of straight line segments, in step S220, according to the shape characteristics of the target object, the target contour straight line segment is determined in the first set of straight line segments. The above-mentioned target object can be a POI. In this embodiment, the POI is a shop sign in the image. Among them, the shape characteristics of the shop sign include that the contour lines of a set of opposite sides are generally parallel, then in the image, especially in the oblique image, the contour straight line segments of the shop sign appear to be basically parallel. Exemplarily, the angle value of a contour straight line segment is α, then the angle value that is basically parallel to it can be in the interval [α-Δ, α+Δ], where Δ is a smaller degree value, such as 5 degrees.

[0077] In an exemplary embodiment, Figure 7 The flowchart of the method for obtaining the straight line segment of the target contour according to an embodiment of the present disclosure is schematically shown. Figure 7 , the embodiment shown in the figure includes steps S710 to S730.

[0078] In step S710, the first set of straight line segments is screened according to the position features of the target object in the image.

[0079] In an exemplary embodiment, the first straight line segment set includes contour straight line segments of multiple objects in the image. In order to improve the efficiency of determining the target contour segment, the first straight line segment can be firstly screened according to the positional features of the target object in the image. Exemplarily, in this embodiment, when the POI is a shop sign, its positional features in the image are generally located at a relatively upper position in the image. Then, a height threshold can be determined, and the height threshold is compared with the positional information of the contour straight line segments in the first straight line segment set in the image, and then the contour line segments of the objects located at the lower part of the image are screened out, while the contour straight line segments of the objects located at the upper part of the image are retained for the subsequent determination of the target contour straight line segments.

[0080] In step S720, the contour straight line segments in the filtered first straight line segment set are processed by a sliding window method.

[0081] In an exemplary embodiment, first, the attribute values ​​of the sliding window can be determined according to the shape features of the target object, including: window width W and sliding step step. For example, in the oblique image, the contour straight line segments of the shop sign appear to be basically parallel, such as the error is within the range of Δ. Then the value of the above window width W belongs to the following interval: [(tanβ1, tan(β1+Δ)), (tan(β2-Δ)), tanβ2]. For example, the above β1 can be taken as 150 degrees, and the above β2 can be taken as 30 degrees. In addition, the size of the sliding step can be determined according to actual needs. In the exemplary embodiment of this embodiment, the sliding step step is 3 degrees.

[0082] In an exemplary embodiment, the first set of straight line segments after the above screening is detected with the above window width value W and the above sliding step step. Exemplarily, with the angle value A as the detection starting point, the angle range in the first window is [A, A+W], and the straight line segments in the first window are detected to obtain the number of substantially parallel contour straight line segments; further, the second window is determined according to the sliding step step, and the angle range in the first window is [A+step, A+step+W], and the straight line segments in the second window are detected to obtain the number of substantially parallel contour straight line segments. By analogy, the detection of contour straight line segments in the first set of straight line segments after the screening is realized through the above sliding window.

[0083] The straight line segments in the filtered first straight line segment set are represented in the form of polar coordinates.

[0084] In step S730, a target window is determined according to the shape feature of the target object, and at least two target contour straight line segments are obtained from a second contour set corresponding to the target window.

[0085] In the above embodiment, the shape feature of the target object (i.e., the POI shop sign) is indeed a set of opposite sides that are basically parallel. Therefore, during the processing of step S720, a target window with the largest number of parallel contour straight line segments is obtained, and two target contour straight line segments (i.e., a set of opposite sides of the above-mentioned POI shop sign) are obtained in the second contour set corresponding to the target window.

[0086] Exemplary, reference Figure 8 , combined with the shape features of the target object and the position features of the target object in the image, it is determined that the target contour straight line segment is a set of opposite edges (S1 and S2) of the POI store sign.

[0087] In an exemplary embodiment, after the target contour straight line segment is determined, in step S230 , key points related to the target contour straight line segment are acquired.

[0088] For example, Fig. 9 The flowchart of the method for determining the key points according to an embodiment of the present disclosure is schematically shown. Fig. 9 , the embodiment shown in the figure includes step S910 and step S920.

[0089] In step S910, two side lines parallel to the edge of the image are determined.

[0090] In an exemplary embodiment, in an oblique image (regardless of whether the camera lens is on the left, right, top or bottom of the target object), the straight line corresponding to the straight line segment of the target contour of the POI intersects with a set of straight lines on opposite sides of the image. Of course, it can also be said that it intersects with a set of parallel lines parallel to the edge of the image.

[0091] In this embodiment, the two edge lines are specifically the left edge line of the image and the right edge line of the image. Figure 8 , the straight line where OL is located in the figure is the above-mentioned left line, and the straight line where JK is located is the above-mentioned right line.

[0092] In step S920, the intersection point between the straight line where the target contour straight line segment is located and the two edge lines is determined as the key point.

[0093] Exemplary, reference Figure 8 , the target contour straight line segments (a set of opposite sides of the POI store sign) are S1 and S2, and the straight lines are T1 and T2 respectively. In this embodiment, the intersection points of straight lines T1 and T2 with the left line and the right line are (a, b, c, d) as the key points.

[0094] In an exemplary embodiment, after determining the key point, in step S230, a mapping relationship is further determined according to the current position information of the key point and the target position information after correction. Specifically, in the same coordinate system, the current position coordinates of the key point in the image are obtained, and the target position coordinates of the key point in the image are determined; and the mapping relationship is determined according to the current position coordinates and the target position coordinates.

[0095] In an exemplary embodiment, the current position information is the position information of the key point in the oblique image (the image before correction processing). In this embodiment, the position information is represented by the coordinate information. A specific implementation method for determining the current coordinates of the key point may be:

[0096] Transform the straight line T1 and the straight line T2 into the form of y=k*x+b, and obtain two straight line equations: y=k1*x+b1 and y=k2*x+b2. Then, according to the two straight line equations, the coordinates of the above key points are determined as: a(0, b1), c(w, k1*w+b1), b(0, b2), d(w, k2*w+b2).

[0097] For example, Fig.10 The image after correction processing according to an embodiment of the present disclosure is schematically shown. In order to achieve the viewing effect that the target object in the image after correction processing is substantially perpendicular to the user's viewing angle, the above two contour lines of the POI should be presented horizontally in the image after correction processing. Therefore, in this embodiment, the vertical coordinates of the above key points a' and c' should be the same, and the vertical coordinates of the above key points b' and d' should be the same. Thus, it can be determined that the target position information of the above key points can be the first group: a'(0, max(b1, k1*w+b1)), c'(w, max(b1, k1*w+b1)), b(0, max(b2, k2*w+b2)), d(w, max(b2, k2*w+b2)), or the second group: a'(0, min(b1, k1*w+b1)), c'(w, min(b1, k1*w+b1)), b(0, min(b2, k2*w+b2)), d(w, min(b2, k2*w+b2)).

[0098] Exemplarily, the first or second group of target coordinate information is selected as the key point based on the following factors: the position feature of the target object in the image (including: relatively upper, relatively lower) and the method of determining the coordinate system (taking the upper left corner of the image as the origin, taking the lower left corner of the image as the origin). In this embodiment, the position feature of the target object POI store sign in the image is: relatively upper, and the coordinate system is determined with the upper left corner of the image as the origin, so the first group is selected as the target coordinate information of the key point.

[0099] In an exemplary embodiment, the getPerspectiveTransform function in opencv is used to calculate the mapping matrix according to the current coordinates and target coordinates of the key points, thereby determining the mapping relationship in step S230.

[0100] Continue to refer Figure 2 After determining the above mapping relationship, in step S240, the image is perspective transformed according to the mapping relationship so that the target object in the corrected image is substantially perpendicular to the user's viewing angle.

[0101] In an exemplary embodiment, the entire image is transformed into a perspective using the mapping matrix. Specifically, the warpPerspective function in opencv is used to pass the image to be corrected and the mapping matrix to achieve perspective transformation. Thus, the position information of each pixel in the image is used as the original coordinates, and the original coordinates are converted into target coordinates according to the mapping relationship to obtain Fig.10 The image after rectification is shown.

[0102] This technical solution can solve the problem of inconvenience in viewing POIs in oblique images. By correcting oblique images through this technical solution, the image quality can be effectively improved. Furthermore, post-processing (such as image recognition, etc.) based on the corrected images is conducive to improving the efficiency of post-processing. For example, when applying optical character recognition (OCR) and other technologies to recognize text in an image, the recognition accuracy of the image processed by this technical solution is significantly improved.

[0103] Those skilled in the art will appreciate that all or part of the steps to implement the above embodiments are implemented as a computer program executed by a processor (including a CPU and a GPU). When the computer program is executed by the processor, the above functions defined by the above method provided by the present disclosure are performed. The program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk or an optical disk, etc.

[0104] In addition, it should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiment of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0105] Furthermore, in this exemplary embodiment, a device for image correction is also provided. Fig.11 As shown, the image correction processing device 1100 includes: a contour line acquisition module 1101, a target contour line determination module 1102, a mapping relationship determination module 1103 and a transformation module 1104. Among them:

[0106] The above-mentioned contour line acquisition module is configured to acquire the contour straight line segments of the object in the image to obtain a first straight line segment set;

[0107] The target contour line determination module 1102 is configured to determine the target contour straight line segment in the first straight line segment set according to the shape feature of the target object;

[0108] The mapping relationship determination module 1103 is configured to obtain key points related to the target contour straight line segment, and determine the mapping relationship according to the current position information of the key points and the corrected target position information;

[0109] The transformation module 1104 is configured to perform perspective transformation on the image according to the mapping relationship, so that the target object in the corrected image is substantially perpendicular to the user's viewing angle.

[0110] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned contour straight line segment acquisition module 1101 includes: a preprocessing unit and a straight line detection unit. Among them:

[0111] The preprocessing unit is configured to: preprocess the image to obtain a preprocessed image; the straight line detection unit is configured to: obtain the contour straight line segments of the object in the preprocessed image through straight line detection.

[0112] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned target contour straight line segment determination module 1102 includes: a sliding window processing unit and a target window determination unit. Among them:

[0113] The above-mentioned sliding window processing unit is configured to: process the contour straight line segments in the above-mentioned first straight line segment set by means of a sliding window; the above-mentioned target window determination unit is configured to: determine the target window according to the shape characteristics of the target object, and obtain at least two target contour straight line segments in the second contour set corresponding to the above-mentioned target window.

[0114] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the image correction processing device 1100 further includes: a straight line segment screening module. Among them:

[0115] The straight line segment screening module is configured to screen the first straight line segment set according to the position features of the target object in the image before the sliding window processing unit determines the target contour straight line segment.

[0116] In an exemplary embodiment of the present disclosure, based on the aforementioned embodiment, the mapping relationship determination module 1103 is specifically configured to: determine two side lines parallel to the edge of the image; and determine the intersection of the straight line where the target contour straight line segment is located and the two side lines as the key point.

[0117] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the two side lines are the left side line of the image and the right side line of the image.

[0118] In an exemplary embodiment of the present disclosure, based on the aforementioned embodiment, the above-mentioned mapping relationship determination module 1103 is also specifically configured to: in the same coordinate system, obtain the current position coordinates of the above-mentioned key point in the above-mentioned image, and determine the target position coordinates of the above-mentioned key point in the above-mentioned image; determine the above-mentioned mapping relationship according to the above-mentioned current position coordinates and the above-mentioned target position coordinates.

[0119] In an exemplary embodiment of the present disclosure, based on the above-mentioned embodiment, the above-mentioned transformation module 1104 is specifically configured to: use the pixel position information in the above-mentioned image as the original coordinates, and convert the above-mentioned original coordinates into target coordinates according to the above-mentioned mapping relationship.

[0120] The specific details of each module or unit in the above-mentioned image correction processing device have been described in detail in the corresponding image correction processing method, so they will not be repeated here.

[0121] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present invention is shown.

[0122] It should be noted that Fig.12 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0123] like Fig.12 As shown, the computer system 1200 includes a processor 1201, wherein the processor 1201 may include: a graphics processing unit (GPU), a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1202 or a program loaded from a storage part 1208 to a random access memory (RAM) 1203. In RAM 1203, various programs and data required for system operation are also stored. The processor (GPU / CPU) 1201, ROM 1202, and RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0124] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0125] In particular, according to an embodiment of the present disclosure, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1209, and / or installed from the removable medium 1211. When the computer program is executed by the processor (GPU / CPU) 1201, various functions defined in the system of the present application are executed. In some embodiments, the computer system 1200 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0126] It should be noted that the computer-readable medium shown in the embodiment of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by an instruction execution system, device or device or used in combination with it. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0127] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0128] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0129] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0130] For example, the electronic device can implement Figure 2 As shown in: step S210, obtaining the contour straight line segments of the object in the image to obtain a first straight line segment set; step S220, determining the target contour straight line segments in the first straight line segment set according to the shape features of the target object; step S230, obtaining key points related to the target contour straight line segments, and determining a mapping relationship according to the current position information of the key points and the target position information after correction processing; and step S240, performing a perspective transformation on the image according to the mapping relationship, so that the target object in the corrected image is basically perpendicular to the user's perspective.

[0131] For example, the electronic device can achieve Figures 3 to 8 The steps shown in .

[0132] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0133] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0134] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0135] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for image correction, characterized in that: The method comprises: Acquire contour straight line segments of objects in an image to obtain a first straight line segment set, wherein the image is a street view image obtained by oblique shooting, and the objects are points of interest; According to the shape features of the target object, determining the target contour straight line segment in the first straight line segment set; Determine two sidelines parallel to the edge of the image, the two sidelines being the left sideline of the image and the right sideline of the image, and determine the intersection of the straight line where the target contour straight line segment is located and the two sidelines as a key point; Determine a mapping relationship according to the current position information of the key point and the target position information after correction; According to the mapping relationship, the image is perspective transformed so that the target object in the corrected image is substantially perpendicular to the user's viewing angle, and the corrected image is used to update or enrich the geographic information system.

2. The image correction processing method according to claim 1, characterized in that: The step of obtaining the contour straight line segments of the object in the image comprises: Preprocessing the image to obtain a preprocessed image; The contour straight line segments of the object in the preprocessed image are obtained by straight line detection.

3. The image correction processing method according to claim 2, characterized in that: The preprocessing of the image to obtain a preprocessed image includes: Obtaining a grayscale image of the image, and performing Gaussian filtering on the grayscale image to obtain a blurred image; Edge detection and dilation processing are performed on the blurred image to obtain a preprocessed image.

4. The image correction method according to claim 1, characterized in that: The step of determining the target contour straight line segment in the first straight line segment set according to the shape feature of the target object comprises: Processing the contour straight line segments in the first straight line segment set by a sliding window method; A target window is determined according to the shape feature of the target object, and at least two target contour straight line segments are obtained from a second contour set corresponding to the target window.

5. The image correction processing method according to claim 4, characterized in that: The step of processing the contour straight line segments in the first straight line segment set by means of a sliding window comprises: Screening the first set of straight line segments according to position features of the target object in the image; The contour straight line segments in the filtered first straight line segment set are processed by a sliding window method.

6. The image correction processing method according to claim 4, characterized in that: Before determining the target contour straight line segment, the method further includes: The first straight line segment set is screened according to the position feature of the target object in the image.

7. The image correction processing method according to any one of claims 1 to 6, characterized in that: The determining of the mapping relationship according to the current position information of the key point and the target position information after correction includes: In the same coordinate system, obtaining the current position coordinates of the key point in the image, and determining the target position coordinates of the key point in the image; The mapping relationship is determined according to the current position coordinates and the target position coordinates.

8. The image correction method according to any one of claims 1 to 6, characterized in that: The performing perspective transformation on the image according to the mapping relationship comprises: The pixel position information in the image is used as the original coordinates, and the original coordinates are converted into target coordinates according to the mapping relationship.

9. An image correction processing device, characterized in that: The device comprises: A contour line acquisition module is configured to acquire contour straight line segments of an object in an image to obtain a first straight line segment set, wherein the image is a street view image obtained by oblique shooting, and the object is a point of interest; A target contour line determination module is configured to determine a target contour straight line segment in the first straight line segment set according to a shape feature of the target object; a mapping relationship determination module, configured to determine two sidelines parallel to the edge of the image, the two sidelines being the left sideline of the image and the right sideline of the image, and to determine the intersection of the straight line where the target contour straight line segment is located and the two sidelines as a key point; and to determine the mapping relationship according to the current position information of the key point and the target position information after correction processing; The transformation module is configured to perform perspective transformation on the image according to the mapping relationship so that the target object in the corrected image is substantially perpendicular to the user's viewing angle, and the corrected image is used to update or enrich the geographic information system.

10. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the image correction method according to any one of claims 1 to 8 by executing the executable instructions.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the image correction processing method according to any one of claims 1 to 8 is implemented.

Images