Photographing terminal and image correction method

Abstract

The present application discloses a photographing terminal and an image correction method, which belong to the field of electronic technology. The photographing terminal includes a wide-angle lens and a controller, and the controller is used to: obtain a reference image captured by the wide-angle lens; perform distortion correction on the reference image based on a reference field of view to obtain a corrected image, and when the corrected image meets the target conditions, reduce the reference field of view, and the initial value of the reference field of view is the upper limit value of the field of view; the target conditions include the presence of auxiliary pixel points in the corrected image, and the imaging points mapped by the auxiliary pixel points are located outside the reference image; when the corrected image does not meet the target conditions, the current reference field of view is determined as the target field of view angle based on which the distortion correction of the image captured by the wide-angle lens is performed. The present application solves the problem that the image acquired by the photographing terminal has a poor presentation effect on the scene. The present application is used for image correction.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and in particular to a camera terminal and an image correction method. Background Art

[0002] With the development of electronic technology, wide-angle lenses are widely used in various photographing terminals (such as cameras or mobile phones) so that the photographing terminals can capture images of scenes in a larger range.

[0003] Because images captured by wide-angle lenses are distorted, distortion correction is required to obtain images that match the actual scene. Distortion correction is performed based on the target field of view set for the wide-angle lens. In related art, the same target field of view is set for the same batch of wide-angle lenses based on empirical values. However, the degree of distortion in images captured by different wide-angle lenses varies, and the target field of view determined based on empirical values ​​is difficult to adapt to every wide-angle lens in the batch.

[0004] Therefore, the distortion correction effect of the image captured by the wide-angle lens is poor, and the image obtained by the camera terminal has a poor rendering effect of the scene. Summary of the Invention

[0005] This application provides a camera terminal and an image correction method, which can solve the problem that images captured by the camera terminal have poor scene presentation. The technical solution is as follows:

[0006] In one aspect, a photographing terminal is provided, comprising: a wide-angle lens and a controller, wherein the controller is configured to:

[0007] Acquire a reference image captured by the wide-angle lens;

[0008] Performing distortion correction on the reference image based on a reference field of view to obtain a corrected image;

[0009] When the corrected image meets a target condition, reducing the reference field of view angle; the initial value of the reference field of view angle is an upper limit value of the field of view angle; the target condition includes the presence of auxiliary pixels in the corrected image, and the imaging point mapped by the auxiliary pixels is located outside the reference image;

[0010] When the corrected image does not meet the target condition, the current reference field of view angle is determined as the target field of view angle based on which the distortion correction of the image captured by the wide-angle lens is performed.

[0011] In another aspect, an image correction method is provided for use in a controller of a photographing terminal, wherein the photographing terminal further includes a wide-angle lens. The method includes:

[0012] Acquire a reference image captured by the wide-angle lens;

[0013] Performing distortion correction on the reference image based on a reference field of view to obtain a corrected image;

[0014] When the corrected image meets a target condition, reducing the reference field of view angle; the initial value of the reference field of view angle is an upper limit value of the field of view angle; the target condition includes the presence of auxiliary pixels in the corrected image, and the imaging point mapped by the auxiliary pixels is located outside the reference image;

[0015] When the corrected image does not meet the target condition, the current reference field of view angle is determined as the target field of view angle based on which the distortion correction of the image captured by the wide-angle lens is performed.

[0016] The beneficial effects of the technical solution provided by this application include at least:

[0017] The camera terminal provided in this application can adjust the field of view used for distortion correction based on the positions of the imaging points mapped to the reference image by the pixels in the correction image, to obtain a suitable target field of view. This allows the camera terminal to use its own target field of view to perform distortion correction on subsequent images captured by the wide-angle lens, thereby improving the distortion correction effect of the image. The distortion-corrected image can be more closely aligned with the captured scene, thereby enhancing the scene presentation quality of the image captured by the camera terminal. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0019] Figure 1 This is a structural diagram of a photographing terminal provided in an embodiment of the present application;

[0020] Figure 2 This is a flowchart of an image correction method improved by an embodiment of the present application;

[0021] Figure 3 is a flowchart of another image correction method provided by an embodiment of the present application;

[0022] Figure 4 is a schematic diagram of a display interface of a photographing terminal provided in an embodiment of the present application;

[0023] Figure 5 1 is a schematic diagram of a scene in which a wide-angle lens captures a reference image according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of a corrected image provided in an embodiment of the present application;

[0025] Figure 7 is a schematic diagram of another corrected image provided in an embodiment of the present application;

[0026] Figure 8 This is a structural block diagram of a photographing terminal provided in an embodiment of the present application;

[0027] Figure 9 This is a software structure diagram of a photographing terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0029] Currently, camera terminals are required to capture images within a wide field of view (FOV). These images must closely match the actual scene, minimizing distortion. Camera terminals can be equipped with wide-angle lenses to capture images of a wider range of scenes. Raw images captured by wide-angle lenses often exhibit distortion. For example, images captured by wide-angle lenses can exhibit barrel distortion, where objects at the edges of the image appear to bend outward. This distorted raw image poorly represents the captured scene, so it is necessary to correct the distortion before displaying it to the user.

[0030] Distortion correction of the original image is performed based on a set field of view (FOV). When this FOV matches the wide-angle lens, distortion correction is most effective. The corrected image accurately represents the captured scene and contains all the information in the original image. The FOV that matches the wide-angle lens is also the wide-angle lens's maximum effective FOV. If the set FOV is too large, the distortion-corrected image obtained by applying distortion to the original image captured by the wide-angle lens may contain pixels that have no corresponding image points in the original image. These pixels may display colors that are significantly different from the original image (such as high-brightness green or red), resulting in an abnormal image display. The resulting distortion-corrected image may also contain distortion caused by overcorrection, resulting in a poor match between the image and the actual scene being captured. For example, objects in the image may appear compressed toward the center of the image. If the set field of view angle is too small, the image obtained after distortion correction of the original image cannot contain all the information in the original image, resulting in the image actually obtained by the camera terminal reflecting a smaller scene range, which is equivalent to losing the field of view angle of the wide-angle lens.

[0031] In the related art, a batch of wide-angle lenses is produced with the same field of view angle, based on empirical values. This allows the camera terminal to perform distortion correction on images captured by the wide-angle lens based on this field of view after the wide-angle lens is installed in the camera terminal. To ensure that the images corrected based on the field of view can be displayed properly by each camera terminal, the field of view angle is set relatively small in the related art. This results in the wide-angle lens not being able to effectively perform its function. Furthermore, there is no guarantee that the set field of view angle will match every wide-angle lens in the batch. It is possible that the field of view angle is too large for a particular wide-angle lens, resulting in distortion in the images captured by the camera terminal where the wide-angle lens is located.

[0032] The embodiments of the present application provide a photographing terminal and an image correction method, which can enable each photographing terminal equipped with a wide-angle lens to use its own matching field of view to correct image distortion, thereby improving the distortion correction effect of the photographing terminal on the image, improving the presentation effect of the image obtained by the photographing terminal on the scene, and effectively exerting the role of the wide-angle lens.

[0033] Figure 1 This is a schematic diagram of the structure of a photographing terminal provided by an embodiment of the present application. Figure 1 As shown, the photographing terminal 10 may include a wide-angle lens J and a controller ( Figure 1 (not shown), the controller may include an image processor. The camera terminal may be any device with an image acquisition function, Figure 1 Taking the photographing terminal as a smart phone as an example, optionally, the photographing terminal may also be a tablet computer, a camera, a video camera or a monitoring device, etc.

[0034] Figure 2 This is a flowchart of an image correction method improved by the embodiment of the present application, which can be used in a controller in a camera terminal. Figure 2 As shown, the method may include:

[0035] Step 201: Acquire a reference image captured by a wide-angle lens.

[0036] The reference image captured by a wide-angle lens is equivalent to the image of the scene being photographed through the wide-angle lens. The pixels in the reference image can reflect the imaging points of each point in the photographed scene. Images captured by a wide-angle lens are distorted, and the reference image can be a distorted image. After production, wide-angle lenses can have a corresponding maximum field of view. Wide-angle lenses can capture images based on this maximum field of view, allowing them to capture images of the largest possible range of scenes.

[0037] Step 202: Perform distortion correction on the reference image based on the reference field of view to obtain a corrected image.

[0038] The controller can perform distortion correction on the reference image based on a set distortion correction algorithm to obtain a corrected image. The parameters of the set distortion correction algorithm include a field of view angle, such as a reference field of view angle. The process of performing distortion correction on the reference image is equivalent to adjusting the position of the imaging point in the reference image.

[0039] For example, during the distortion correction process, the controller can construct a corrected image using imaging points in a reference image based on a reference field of view angle. For example, the imaging points in the reference image are mapped to the corrected image, and the relative positions between the imaging points in the reference image are the same as the relative positions between the pixels mapped to the imaging points in the corrected image. For example, if imaging point a in the reference image is above imaging point b, then the pixel mapped to imaging point a in the corrected image is also above the pixel mapped to imaging point b. The controller can establish a coordinate system in the reference image and determine the coordinates of each imaging point in the reference image based on the coordinate system.

[0040] Step 203: When the corrected image meets the target condition, reduce the reference field of view angle; the initial value of the reference field of view angle is the upper limit value of the field of view angle; the target condition includes the presence of auxiliary pixels in the corrected image, and the imaging point mapped by the auxiliary pixels is located outside the reference image.

[0041] After performing distortion correction on the reference image based on the reference field of view angle to obtain a corrected image, the controller can determine whether auxiliary pixels exist in the corrected image to check whether the corrected image meets the target conditions. For example, after performing distortion correction, the controller can determine the mapping relationship between pixels in the corrected image and imaging points in the reference image. Based on this mapping relationship, the controller can determine whether there are auxiliary pixels in the corrected image whose imaging points are mapped outside the reference image, i.e., the auxiliary pixels are not mapped to imaging points in the reference image. For example, after performing distortion correction, the controller can determine the coordinates of the imaging points to which the pixels in the corrected image are mapped. If the imaging point to which a pixel in the corrected image is mapped cannot be determined, the coordinates of the pixel to which the pixel is mapped can be determined based on the positional relationship between the pixel and its surrounding pixels and the coordinates of the imaging points mapped by the surrounding pixels. In this way, the coordinates of the imaging points to which all pixels in the corrected image are mapped can be determined, thereby determining whether there are auxiliary pixels in the corrected image whose imaging points are mapped outside the reference image.

[0042] If the target conditions are met, such as the presence of auxiliary pixels in the corrected image, this indicates that the field of view angle used to correct the distortion of the reference image is too large, i.e., the reference field of view angle is too large. The controller can then reduce the reference field of view angle and re-execute steps 202 and 203. This allows the distortion of the reference image to be corrected again using a smaller reference field of view angle. Based on the mapping relationship between each pixel and the imaging point in the corrected image after distortion correction, the controller can then determine whether the current reference field of view angle is appropriate.

[0043] When the corrected image meets the target conditions, the degree of conformity between the image presented in the corrected image and the actual scene being photographed is lower than the threshold, and the corrected image cannot accurately present the photographed scene. When the corrected image does not meet the target conditions, the degree of conformity between the image presented in the corrected image and the actual scene being photographed is higher than the threshold, and the corrected image accurately presents the photographed scene. In an embodiment of the present application, a staff member can set an upper limit value for the field of view angle based on empirical values, and use the upper limit value for the field of view angle as the initial value of the reference field of view angle to perform distortion correction on the reference image. Thereafter, the controller gradually reduces the reference field of view angle from the upper limit value of the field of view angle until a suitable reference field of view angle is determined. In this way, as long as it is determined that the corrected image does not meet the target conditions, the reference field of view angle at this time is the maximum value that makes the degree of conformity higher than the threshold. Furthermore, on the basis of ensuring that the corrected image accurately presents the photographed scene, it can be ensured that the corrected image contains information within the maximum range of the photographed scene, fully demonstrating the role of the wide-angle lens.

[0044] Step 204 : When the corrected image does not meet the target condition, the current reference field angle is determined as the target field angle based on which the distortion correction is performed on the image captured by the wide-angle lens.

[0045] If the corrected image obtained after distortion correction of the reference image does not meet the target conditions, the controller can determine that the current reference field of view angle enables the corrected image to accurately represent the captured scene, and can then determine the current reference field of view angle as the target field of view angle. The controller can then perform distortion correction on subsequent images captured by the wide-angle lens based on the target field of view angle. Optionally, the camera terminal also includes a display screen, and the controller can also control the display screen to display the image after distortion correction.

[0046] Optionally, the reference image can be the first frame of image captured by the wide-angle lens after the camera terminal leaves the factory. For example, after receiving the camera terminal, the user can start the camera program of the camera terminal, and then the wide-angle lens of the camera terminal can begin to capture images at a certain frame rate. Each frame of image captured by the wide-angle lens can be presented to the user, such as through the display screen of the camera terminal, so that the user can press the capture button after seeing a satisfactory image, triggering the camera terminal to store the image captured by the camera terminal when the capture button was pressed. Because the original image captured by the wide-angle lens is distorted, it is necessary to correct the distortion of the original image and then present the distortion-corrected image to the user. After the user presses the capture button, the camera terminal stores the image captured by the wide-angle lens when the capture button was pressed, after the distortion is corrected.

[0047] For the reference image captured by the wide-angle lens, after the controller determines the target field of view angle, the display screen displays the corrected image after the distortion of the reference image is corrected based on the target field of view angle. Optionally, the wide-angle lens can stop capturing images during the period between the wide-angle lens capturing the reference image and the display screen displaying the corrected image; after the display screen displays the corrected image corresponding to the reference image, the wide-angle lens continues to capture images. Optionally, during the period between the wide-angle lens capturing the reference image and the display screen displaying the corrected image, the wide-angle lens still captures images at its frame rate, but the controller does not perform distortion correction on the images during this period, and the display screen does not display the images during this period. After the display screen displays the corrected image corresponding to the reference image, the controller performs distortion correction on the images subsequently captured by the wide-angle lens based on the target field of view angle, and the display screen displays the subsequently determined corrected image.

[0048] In the embodiment of the present application, the target field of view angle is determined based on the first frame image captured by the wide-angle lens after the photographing terminal leaves the factory. The images displayed by the photographing terminal are all images with good distortion correction effect obtained based on the target field of view angle, so the image display effect of the photographing terminal is better.

[0049] In summary, in the image correction method provided by this application, the camera terminal can adjust the field of view used for distortion correction based on the position of the imaging point mapped by the pixel points in the correction image in the reference image to obtain a suitable target field of view. In this way, the camera terminal can use the target field of view that is adapted to itself to perform distortion correction on images subsequently captured by the wide-angle lens, which can improve the distortion correction effect of the image. The distortion-corrected image can be more consistent with the captured scene, thereby improving the rendering effect of the image captured by the camera terminal on the scene.

[0050] Figure 3 This is a flow chart of another image correction method provided by an embodiment of the present application. This method can be used in a controller in a camera terminal that also includes a wide-angle lens. The method may include:

[0051] Step 301: Obtain the upper limit value of the field of view angle.

[0052] Staff can set an upper limit for the field of view angle based on empirical data and input this upper limit into the camera terminal. For example, staff can test the maximum effective field of view angles of multiple wide-angle lenses already in use and then determine the maximum of these multiple maximum effective field of view angles as the upper limit for the field of view angle. For another example, staff can directly set a larger field of view angle as the upper limit for the field of view angle. If this upper limit for the field of view angle is the maximum field of view angle of the wide-angle lens, the wide-angle lens can capture images of the largest range of scenes at this field of view angle.

[0053] Step 302: Acquire a reference image captured by a wide-angle lens.

[0054] The reference image can be any frame of image captured by a wide-angle lens, or it can be an image of a specified object captured by a wide-angle lens. For example, the camera terminal can display a prompt message instructing the operator to operate the camera terminal to capture the specified object so that the wide-angle lens of the camera terminal captures the reference image.

[0055] For example, Figure 4 This is a schematic diagram of a display interface of a photographing terminal provided in an embodiment of the present application. Figure 4 As shown, the display screen of the photographing terminal may display "Please take an image with a black rectangle in the middle and white edges to optimize the distortion correction effect." In this case, the designated object may be a picture card with a designated pattern. Figure 5 Schematic diagram of a scene in which a wide-angle lens captures a reference image provided by an embodiment of the present application. Figure 5 As shown, the camera terminal 10 can be fixed to the bracket 20 and photographed by a chart 30 having a specified pattern. For example, the center of the chart is a black rectangle, while the remaining areas are pure white. When photographing, the center of the wide-angle lens is aligned within the black rectangle, and the surrounding edges are aligned within the pure white area. It should be noted that the size of the chart and the black rectangle can be adjusted according to actual needs and is not limited in this embodiment of the application.

[0056] Optionally, a camera program is installed in the photographing terminal. After the camera program is activated, the wide-angle lens can begin capturing images. The photographing terminal can then display the image without distortion correction, or display an image obtained by performing distortion correction on the captured image based on the initial field of view. Upon receiving a capture command (e.g., detecting a press of a capture button), the image captured by the wide-angle lens at this time can be used as a reference image, and the field of view used for the distortion correction process can be adjusted based on this reference image.

[0057] Step 303: Perform distortion correction on the reference image based on the reference field of view angle to obtain a corrected image. The initial value of the reference field of view angle is the upper limit value of the field of view angle.

[0058] After acquiring a reference image captured by the wide-angle lens, the controller can perform distortion correction on the reference image based on a distortion correction algorithm. The distortion correction algorithm initially stored in the camera terminal uses the upper limit of the field of view as the reference field of view. For example, based on the wide-angle lens's maximum field of view and the set reference field of view, the controller can map imaging points in the reference image to corresponding positions in the correction image, thereby generating the correction image. Figure 6 This is a schematic diagram of a corrected image provided in an embodiment of the present application. The corrected image may be an image obtained after the controller performs distortion correction on a reference image based on an upper limit value of the field of view angle. Figure 7 This is a schematic diagram of another corrected image provided in an embodiment of the present application. This corrected image can be an image obtained by the controller performing distortion correction on the reference image based on the final reference field of view angle. In the embodiment of the present application, the controller of the camera terminal can perform distortion correction in an uncalibrated manner.

[0059] Optionally, if the reference field of view angle is too large, there may be a position in the corrected image that is not mapped to the imaging point in the reference image, and this position is located in the edge area of ​​the corrected image. The controller can configure the pixel point at this position to ensure that a corrected image that can be displayed completely is obtained. The pixel point at this position is also the pixel point where the mapped imaging point is located outside the reference image. In the embodiment of the present application, the pixel point at this position is referred to as an auxiliary pixel point. Optionally, the controller can configure a reference pixel value for the pixel point in the corrected image so that the pixel point displays a reference color.

[0060] In one optional manner, the reference pixel value can be a fixed value pre-set by a staff member. The reference pixel value can represent a reference color, such as black. It should be noted that the pixel value of each pixel point can include three grayscale values ​​corresponding to the three color components of red, green, and blue. When the reference color is black, the three grayscale values ​​corresponding to the three color components are all 0. Optionally, different grayscale values ​​can be set for each color component so that the auxiliary pixel point displays red, blue, green, or other reference colors. For example, the grayscale value corresponding to the red component in the reference pixel value can be 255, and the grayscale values ​​corresponding to the blue component and the green component can be both 0, so that the auxiliary pixel point can display red. Optionally, when auxiliary pixels exist in the corrected image, there are usually multiple auxiliary pixels. Reference pixel values ​​can be configured for these multiple auxiliary pixels based on a set pixel value configuration rule. For example, the pixel value configuration rule is to alternately configure the auxiliary pixels using two different pixel values, or to configure the auxiliary pixels using these multiple pixel values ​​in a fixed order.

[0061] In another optional embodiment, the reference pixel value may be the pixel value of a pixel at a specified location in the corrected image, where the pixel at the specified location has a mapped imaging point in the reference image. For example, the specified location may be located in the center region of the corrected image. If the center region of the corrected image is black, the controller may use the pixel value of the center region of the corrected image to configure auxiliary pixels (pixels not mapped to imaging points in the reference image) in the corrected image.

[0062] Optionally, in an embodiment of the present application, the reference image and the reference pixel value can be matched, such as the pixel value of the pixel point in the edge area of ​​the reference image can be different from the reference pixel value. For example, the specific processing method of the image during the distortion correction process can be set before obtaining the reference image, and based on the reference pixel value set in the distortion correction algorithm, the staff can be instructed to operate the camera terminal to collect a reference image that can match the reference pixel value. If the reference pixel value in the distortion correction algorithm is a fixed value, and the grayscale value corresponding to each color component in the reference pixel value is 0, then the reference image can be an image with a pure white edge area. If the reference pixel value in the distortion correction algorithm is determined based on the pixel value of the pixel point at the center position of the corrected image, then the reference image can be an image with different colors in the center area and the edge area, such as Figure 5 Image of the chart shown.

[0063] Step 304: Determine whether the corrected image meets the target condition, which includes the presence of auxiliary pixels in the corrected image. If the corrected image meets the target condition, execute step 305; if the corrected image does not meet the target condition, execute step 306.

[0064] In an embodiment of the present application, an auxiliary pixel point refers to a pixel point in the corrected image where the image point mapped is located outside the reference image. The controller can determine whether there are auxiliary pixels in the corrected image based on the way the auxiliary pixels are configured during the distortion correction process. For example, during the distortion correction process, the pixel values ​​of the pixels in the middle area of ​​the corrected image are used as reference pixel values, and the middle area of ​​the reference image is black. The controller can determine whether the pixels in the edge area of ​​the corrected image have reference pixel values ​​to determine whether there are auxiliary pixels in the corrected image. The controller can determine the pixels in the edge area of ​​the corrected image that have reference pixel values ​​as auxiliary pixels. When there are auxiliary pixels in the corrected image, the controller can determine that the corrected image meets the target condition and then directly execute step 305; when there are no auxiliary pixels in the corrected image, the controller can determine that the corrected image does not meet the target condition and then directly execute step 306. For example, the edge area of ​​the image is the area where at least one row of pixels close to each side of the image is located. If the corrected image is rectangular and the at least one row of pixels includes only one row of pixels, the edge region of the corrected image may include the first row of pixels, the last row of pixels, the first column of pixels, and the last column of pixels in the corrected image. The at least one row of pixels may also include two, three, or even more rows of pixels.

[0065] Optionally, the controller can convert the corrected image into a grayscale image and then determine whether the pixel is an auxiliary pixel based on the grayscale of the pixel in the edge region of the grayscale image. For example, the controller can determine that a pixel is an auxiliary pixel when the grayscale of a pixel in the edge region of the grayscale image is equal to a reference grayscale. The reference grayscale is the grayscale corresponding to the reference pixel value, which can be 0. Optionally, the controller can determine that the pixel is an auxiliary pixel when the grayscale of a pixel in the edge region of the grayscale image is less than a grayscale threshold. Optionally, the controller can determine that the pixel has the reference pixel value and thus determine that the pixel is an auxiliary pixel when the grayscale of a pixel in the edge region of the grayscale image is less than a grayscale threshold. It should be noted that it is difficult to ensure that the grayscale of all pixels in the central region of the reference image captured by a wide-angle lens is 0. Some pixels may not actually appear to be pure black, but rather a lower-brightness gray. In the embodiments of the present application, a grayscale threshold is set to determine that pixels with grayscales less than the grayscale threshold are auxiliary pixels. This can enhance the applicability of the auxiliary pixel determination method and improve the accuracy of auxiliary pixel determination.

[0066] In the embodiment of the present application, the corrected image may be a polygon, such as a rectangle. Alternatively, the corrected image may be a triangle, pentagon, hexagon, or other polygon, which is not limited in the embodiment of the present application. The target condition may include other conditions in addition to the presence of auxiliary pixels in the corrected image.

[0067] In an optional embodiment, the target condition further comprises: in at least one row of pixels adjacent to any edge of the corrected image, the total number of auxiliary pixels is greater than a first threshold; and the pixels in each of the at least one row of pixels are arranged sequentially along the direction of the edge. In this embodiment, the controller detects whether the total number of black pixels in the edge region of the corrected image is greater than the first threshold.

[0068] For example, if the at least one row of pixels only includes one row of pixels, the controller may first record the total number of auxiliary pixels as 0. Then, the controller starts from the first pixel in the first row of pixels of the corrected image and detects the pixel value of each pixel in sequence along the row direction. When it is determined that a pixel has a reference pixel value, the total number of auxiliary pixels is increased by 1. After each update of the total number of auxiliary pixels, the updated total number of auxiliary pixels is compared with the first threshold value until the total number of auxiliary pixels is greater than the first threshold value, or the pixel value of each pixel in the row of pixels is detected. Optionally, the controller may also compare the obtained total number of auxiliary pixels with the first threshold value after detecting the pixel value of each pixel in the row of pixels.

[0069] If it is determined that the total number of auxiliary pixels is greater than the first threshold value during the detection of the first row of pixels of the corrected image, it is directly determined that the corrected image meets the target condition. If it is determined that the total number of auxiliary pixels is greater than the first threshold value after the detection of the first row of pixels of the corrected image is completed, the detection of the total number of auxiliary pixels is continued for other sides. Until it is determined that the total number of auxiliary pixels is greater than the first threshold value for a certain side, it is determined that the corrected image meets the target condition; or until it is determined that the total number of auxiliary pixels is not greater than the first threshold value for all sides of the corrected image, it is determined that the corrected image does not meet the target condition. Optionally, the controller can also perform the above-mentioned detection of the total number of auxiliary pixels for each side of the corrected image, and then determine whether there is a situation where the total number of auxiliary pixels is greater than the first threshold value for any of the multiple sides. It should be noted that the embodiment of the present application only takes the at least one row of pixels as an example. The at least one row of pixels can also include two rows of pixels, three rows of pixels or even more rows of pixels, and the embodiment of the present application is not limited thereto.

[0070] In another optional embodiment, the target condition further includes: in at least one row of pixels adjacent to any edge of the corrected image, the number of auxiliary pixels arranged continuously in the direction of the edge is greater than a second threshold; and each row of pixels in the at least one row of pixels is arranged sequentially along the direction of the edge. In this embodiment, the controller detects whether the total number of pixels displayed in a continuous black pattern in the edge region of the corrected image is greater than the second threshold. Optionally, the second threshold can be less than the first threshold.

[0071] For example, the at least one row of pixels only includes one row of pixels, and the controller can first record the number of auxiliary pixels arranged in series as 0. For ease of description, the number of auxiliary pixels arranged in series is referred to as the target number below. The controller can start from the first pixel in the first row of pixels of the corrected image and detect the pixel value of each pixel in sequence along the row direction. When it is determined that a pixel has a reference pixel value, the target number is updated to 1, and then the pixel value of the next pixel is detected. When the next pixel also has a reference pixel value, the target number is added by 1, and the pixel value of the next pixel is detected. If the next pixel of a certain pixel does not have a reference pixel value, the target number is reset to 0, and the pixel value of the next pixel is detected. After each update of the target number, the controller compares the updated target number with the second threshold until the target number is greater than the second threshold, or the pixel value of each pixel in the row of pixels is detected.

[0072] If the number of targets is determined to be greater than the second threshold during detection of the first row of pixels in the corrected image, the corrected image is directly determined to meet the target condition. If the number of targets is determined to be greater than the second threshold after detection of the first row of pixels in the corrected image is completed, target number detection is continued for other edges. The corrected image is determined to meet the target condition until the number of targets is determined to be greater than the second threshold for a particular edge; or the corrected image is determined not to meet the target condition until the number of targets is determined to be less than the second threshold for all edges of the corrected image. Optionally, the controller may also perform the above target number detection for each edge of the corrected image, and then determine whether the number of targets is greater than the second threshold for any of the multiple edges. It should be noted that the embodiment of the present application only uses the example of the at least one row of pixels comprising one row of pixels. The at least one row of pixels may also comprise two rows of pixels, three rows of pixels, or even more rows of pixels, and this embodiment of the present application is not limited thereto. Optionally, when the at least one row of pixels comprises multiple rows of pixels, the target condition is determined to be met only if the number of continuously arranged auxiliary pixels in each row of pixels is greater than the second threshold.

[0073] Step 305: Reduce the reference viewing angle to obtain an updated reference viewing angle. Execute step 303.

[0074] When the controller determines that the corrected image meets the target conditions in step 304, the controller can determine that the reference field of view angle on which the corrected image is based is too large. Furthermore, the controller can reduce the reference field of view angle and re-execute step 303 and subsequent steps to determine whether the reduced reference field of view angle is appropriate. For example, the controller can obtain a rated angle, and in step 305, the controller can reduce the reference field of view angle by the rated angle to obtain an updated reference field of view angle. It should be noted that the above process from step 303 to step 305 can be executed in a loop until it is determined in step 304 that the corrected image does not meet the target conditions.

[0075] Optionally, the controller can also obtain the lower limit value of the field of view angle. If the wide-angle lens is assembled properly, the maximum effective field of view angle corresponding to the wide-angle lens should be between the field of view angle limit value and the field of view angle upper limit value. After step 305, the controller can also determine whether the updated reference field of view angle is less than the lower limit value of the field of view angle. When the updated reference field of view angle is greater than or equal to the lower limit value of the field of view angle, step 303 is executed again. When the updated reference field of view angle is less than the lower limit value of the field of view angle, the controller stops executing the process from step 303 to step 305. Optionally, the controller can also control the camera terminal to prompt that the wide-angle lens is assembled incorrectly, such as controlling the display screen to display information that the wide-angle lens is assembled incorrectly, or controlling the speaker to emit a sound indicating that the wide-angle lens is assembled incorrectly.

[0076] Step 306: Determine the current reference field angle as the target field angle based on which distortion correction is performed on the image captured by the wide-angle lens.

[0077] After the controller determines in step 304 that the corrected image does not meet the target conditions, it can determine that the distortion correction applied to the reference image is effective and that the reference field of view angle used to obtain the corrected image after distortion correction of the reference image is appropriate. The controller can then record the current reference field of view angle and determine it as the target field of view angle for subsequent distortion correction of the image. If the field of view angle used to correct the image distortion can be a parameter in the distortion correction algorithm, this process is equivalent to updating the parameters of the distortion correction algorithm.

[0078] After determining the target field of view, if the camera terminal subsequently activates the camera program to capture images with the wide-angle lens, the controller performs distortion correction on each frame of the image captured by the wide-angle lens based on the target field of view. The camera terminal's display then displays the corrected image.

[0079] Optionally, Figure 3The method shown can be executed after the photographing terminal leaves the factory. For example, after the photographing terminal leaves the factory, if the camera program of the photographing terminal is started, the photographing terminal can instruct the user to control the photographing terminal to shoot a specified object to obtain a reference image, and then the photographing terminal performs the above steps based on the reference image to obtain the target field of view. Afterwards, the photographing terminal can perform distortion correction on each frame of the image captured by the wide-angle lens based on the target field of view to ensure a good distortion correction effect of the image. Optionally, Figure 3 The method shown can also be performed before the camera terminal leaves the factory. For example, before the camera terminal leaves the factory, a spot check is performed on the camera terminal. During the spot check, the staff can start the camera program in the camera terminal and control the camera terminal to take a picture of a specified object to obtain a reference image. Then, the controller of the camera terminal takes a picture of a specified object based on the reference image. Figure 3 The method is used to determine whether a suitable target field of view angle can be determined. After the suitable target field of view angle can be determined, the distortion correction algorithm in the camera terminal is determined to be qualified, and the camera terminal is qualified for production. Then, the camera terminal can be mass-produced or sold.

[0080] In the embodiments of the present application, the camera terminal can, based on a captured reference image, cyclically perform distortion correction on the reference image and determine whether the resulting target image meets the target conditions. This allows the camera terminal to self-adjust the field of view used for distortion correction and determine an appropriate target field of view. This allows each camera terminal to perform distortion correction on the image using the target field of view that matches its wide-angle lens, improving the effectiveness of the distortion correction performed by the camera terminal.

[0081] In summary, in the image correction method provided by this application, the camera terminal can adjust the field of view used for distortion correction based on the position of the imaging point mapped by the pixel points in the correction image in the reference image to obtain a suitable target field of view. In this way, the camera terminal can use the target field of view that is adapted to itself to perform distortion correction on images subsequently captured by the wide-angle lens, which can improve the distortion correction effect of the image. The distortion-corrected image can be more consistent with the captured scene, thereby improving the rendering effect of the image captured by the camera terminal on the scene.

[0082] Figure 8 This is a structural block diagram of a photographing terminal provided by an embodiment of the present application. Figure 8 As shown, the photo-taking terminal 10 may include: a radio frequency (RF) circuit 150, an audio circuit 160, a wireless fidelity (Wi-Fi) module 170, a Bluetooth module 180, a power supply 190, a camera 1032, a processor 1101 and a crystal oscillator unit 120 and other components, and the camera 1032 includes at least a wide-angle lens J.

[0083] The camera 1032 can be used to capture still images or videos. The object is projected through the lens into a photosensitive element, which can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the processor 1101 for conversion into a digital image signal.

[0084] The processor 1101 is the control center of the camera terminal 10. It connects the various components of the entire terminal using various interfaces and lines. It executes the various functions of the camera terminal 10 and processes data by running or executing software programs stored in the memory 140 and accessing data stored in the memory 140. In some embodiments, the processor 1101 may include one or more processing units. The processor 1101 may also integrate an application processor (AP) and a baseband processor (BP). The application processor primarily handles the operating system, user interface, and application programs, while the baseband processor primarily handles wireless communications. It is understood that the baseband processor may not be integrated into the processor 1101. In this application, the processor 1101 can run the operating system and application programs, control the user interface display, and implement the camera terminal control method provided in the embodiments of this application. In addition, the processor 1101 is coupled to the input unit and the touch screen display 130.

[0085] The touchscreen display 130 can be used to receive input digital or character information and generate signal input related to user settings and function control of the camera terminal 10. Optionally, the touchscreen display 130 can also be used to display information input by the user or provided to the user, as well as a graphical user interface (GUI) for various menus of the camera terminal 10. The touchscreen display 130 may include a display screen disposed on the front of the camera terminal 10. The display screen may be configured in the form of a liquid crystal display, light-emitting diode, etc. The touchscreen display can be used to display the various graphical user interfaces described in this application.

[0086] The touchscreen display 130 includes a display screen and a touch screen located on the front of the camera terminal 10. The display screen can be used to display preview images. The touchscreen can detect user touch operations on or near it, such as clicking buttons and dragging scroll boxes. The touchscreen can be overlaid on the display screen or integrated with the display screen to implement the input and output functions of the camera terminal 10. This integrated system can be referred to as a touchscreen display.

[0087] The memory 140 can be used to store software programs and data. The processor 1101 executes the various functions and data processing of the camera terminal 10 by running the software programs or data stored in the memory 140. The memory 140 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 140 stores the operating system that enables the camera terminal 10 to operate. In the present application, the memory 140 can store the operating system and various application programs, and can also store code for executing the methods provided in the embodiments of the present application.

[0088] RF circuit 150 can be used to receive and transmit signals during information transmission or calls. It can receive downlink data from the base station and pass it to processor 1101 for processing; it can also send uplink data to the base station. Typically, RF circuits include but are not limited to antennas, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, and other components.

[0089] The audio circuit 160, speaker 161, and microphone 162 can provide an audio interface between the user and the camera terminal 10. The audio circuit 160 can convert the received audio data into an electrical signal and transmit it to the speaker 161, which converts it into a sound signal for output. The camera terminal 10 can also be equipped with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 162 converts the collected sound signal into an electrical signal, which is received by the audio circuit 160 and converted into audio data. The audio data is then output to the RF circuit 150 for transmission to, for example, another terminal, or the audio data is output to the memory 140 for further processing. In this application, the microphone 162 can capture the user's voice.

[0090] Wi-Fi is a short-range wireless transmission technology. The camera terminal 10 can help users send and receive emails, browse web pages, and access streaming media through the Wi-Fi module 170, which provides users with wireless broadband Internet access.

[0091] The Bluetooth module 180 is used to exchange information with other Bluetooth devices having a Bluetooth module through the Bluetooth protocol. For example, the camera terminal 10 can establish a Bluetooth connection with a wearable camera terminal (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 180 to exchange data.

[0092] The camera terminal 10 also includes a power supply 190 (e.g., a battery) that supplies power to various components. The power supply can be logically connected to the processor 1101 via a power management system, enabling the power management system to manage charging, discharging, and power consumption. The camera terminal 10 can also be equipped with a power button for turning the terminal on and off, as well as locking the screen.

[0093] The camera terminal 10 may include at least one sensor 1110, such as a motion sensor 11101, a distance sensor 11102, a fingerprint sensor 11103, and a temperature sensor 11104. The camera terminal 10 may also be equipped with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor.

[0094] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the photographing terminal and each device described above can refer to the corresponding processes in the aforementioned method embodiment and will not be repeated here.

[0095] Figure 9 This is a software block diagram of a camera terminal provided in an embodiment of the present application. A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other via software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0096] The application layer can include a series of application packages. Figure 9 As shown, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message. The application framework layer provides the application programming interface (API) and programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0097] like Figure 9 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0098] The window manager is used to manage window programs. The window manager can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0099] Content providers are used to store and retrieve data and make it accessible to applications. The data may include videos, pictures, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0100] The view system includes visual controls, such as those for displaying text and images. The view system is used to build applications. A display interface can consist of one or more views. For example, a display interface containing a text notification icon might include a view for displaying text and a view for displaying images.

[0101] The phone manager is used to provide communication functions of the camera terminal 10, such as management of call status (including answering, hanging up, etc.).

[0102] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0103] The Notification Manager allows applications to display notifications in the status bar. These messages can be displayed briefly and then disappear automatically, without requiring user interaction. For example, the Notification Manager can be used to announce the completion of downloads, message reminders, and so on. The Notification Manager can also display notifications in the form of icons or scrolling text in the top status bar, such as notifications from background applications, or in the form of dialog windows on the screen. Examples include displaying text messages in the status bar, emitting alert sounds, vibrating the communication terminal, and flashing indicator lights.

[0104] The Android runtime includes the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0105] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0106] The application layer and application framework layer run in a virtual machine. The virtual machine executes Java files in the application layer and application framework layer as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0107] The system library can include multiple functional modules, such as surface manager, media libraries, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0108] The surface manager is used to manage the display subsystem and provide fusion of 2D and 3D layers for multiple applications.

[0109] The media library supports playback and recording of a variety of common audio and video formats, as well as static image files. The media library can support a variety of audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0110] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0111] A 2D graphics engine is a drawing engine for 2D drawings.

[0112] The kernel layer is the layer between hardware and software. The kernel layer includes at least display driver, camera driver, audio driver, and sensor driver.

[0113] The present application also provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method provided in the above embodiment, for example Figure 2 or Figure 3 The method shown.

[0114] The present application also provides a computer program product including instructions. When the computer program product is run on a computer, the computer executes the method provided in the above method embodiment, for example: Figure 2 or Figure 3 The method shown.

[0115] It should be noted that the method embodiments provided in the embodiments of the present application can be referenced with the corresponding device embodiments, and the embodiments of the present application are not limited thereto. The order of the steps of the method embodiments provided in the embodiments of the present application can be appropriately adjusted, and the steps can also be increased or decreased accordingly according to the circumstances. Any technician familiar with the technical field can easily think of the changed methods within the technical scope disclosed in this application, and they should be included in the scope of protection of this application, so they will not be repeated here. In this application, the term "at least one" refers to one or more, and "a plurality of" refers to "two or more".

[0116] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A photographing terminal, characterized in that: The photographing terminal includes a wide-angle lens and a controller, wherein the controller is configured to: Acquire a reference image captured by the wide-angle lens; Performing distortion correction on the reference image based on a reference field of view to obtain a corrected image; When the corrected image meets the target condition, reducing the reference field angle; The initial value of the reference field of view angle is the upper limit value of the field of view angle; the target condition includes the presence of auxiliary pixels in the corrected image, and the imaging point mapped by the auxiliary pixels is located outside the reference image; When the corrected image does not meet the target condition, determining the current reference field of view angle as the target field of view angle based on which distortion correction is performed on the image captured by the wide-angle lens; In the process of the distortion correction, reference pixel values ​​are configured for the pixel points of the imaging points mapped in the corrected image that are located outside the reference image; and the controller is used to: determine the pixel points with the reference pixel values ​​in the edge area of ​​the corrected image as the auxiliary pixel points.

2. The photographing terminal according to claim 1, characterized in that: The controller is used to: Converting the corrected image into a grayscale image; When the grayscale of a pixel point in the edge area of ​​the grayscale image is less than a grayscale threshold, the pixel point is determined as the auxiliary pixel point.

3. The photographing terminal according to claim 1, characterized in that: The corrected image is polygonal, and the target condition also includes: in at least one row of pixel points close to any edge of the corrected image, the total number of auxiliary pixel points is greater than a first threshold; and each row of pixel points in the at least one row of pixel points is arranged sequentially along the extension direction of any edge.

4. The photographing terminal according to claim 1, wherein: The target condition also includes: in at least one row of pixel points close to any edge of the corrected image, the number of auxiliary pixel points continuously arranged in the extension direction of any edge is greater than a second threshold; and each row of pixel points in the at least one row of pixel points is arranged sequentially along the extension direction of any edge.

5. The photographing terminal according to any one of claims 1 to 4, characterized in that: The pixel values ​​of the pixels in the edge area of ​​the reference image are different from the reference pixel values.

6. The photographing terminal according to any one of claims 1 to 4, characterized in that: The reference image is the first frame of image captured by the wide-angle lens after the photographing terminal leaves the factory.

7. The photographing terminal according to any one of claims 1 to 4, characterized in that: The controller is also used for: When the reduced reference viewing angle is less than the viewing angle lower limit, it is prompted that the wide-angle lens is incorrectly assembled.

8. An image correction method, characterized in that: A controller for a photographing terminal, wherein the photographing terminal further includes a wide-angle lens, and the method includes: Acquire a reference image captured by the wide-angle lens; Performing distortion correction on the reference image based on a reference field of view to obtain a corrected image; When the corrected image meets a target condition, reducing the reference field of view angle; the initial value of the reference field of view angle is an upper limit value of the field of view angle; the target condition includes the presence of auxiliary pixels in the corrected image, and the imaging point mapped by the auxiliary pixels is located outside the reference image; When the corrected image does not meet the target condition, determining the current reference field of view angle as the target field of view angle based on which distortion correction is performed on the image captured by the wide-angle lens; The method further comprises: During the distortion correction process, reference pixel values ​​are configured for pixel points whose imaging points mapped in the corrected image are located outside the reference image; and pixel points with the reference pixel values ​​in the edge area of ​​the corrected image are determined as the auxiliary pixel points.

Images