Image operation method, image operation device and electronic equipment

Abstract

The invention relates to an image operation method, an image operation device and electronic equipment. The method comprises the following steps: responding to image operation; according to the distortion removing effect of the image, determining a target image of which the distortion degree meets the requirement, and taking the target image as a to-be-operated image; and performing operation processing on the to-be-operated image. According to the method, under the condition that the distortion degree of the image meets the requirement, further operation processing (such as distortion removal processing) is carried out on the image, so that the quality of the image can be improved, the high-quality requirement of a user on the image is met, and the use experience of the user is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image operation method, an image operation device and an electronic device. Background Art

[0002] Usually, a large number of de-distortion algorithms are applied in camera applications of terminal devices such as mobile phones, but the de-distortion effect of the de-distortion algorithm is usually uncertain. At present, there are more and more applications for images, such as preview, various portrait modes, photo effects, beautification editing and other scenarios, so the requirements for image quality are getting higher and higher. Summary of the invention

[0003] The present disclosure provides an image operation method, an image operation device and an electronic device. The present disclosure performs further operation processing (such as de-distortion processing) on ​​the image when the degree of distortion of the image meets the requirements, which can improve the quality of the image, meet the user's high-quality requirements for the image, and improve the user's usage experience.

[0004] According to a first aspect of an embodiment of the present disclosure, an image operation method is provided, comprising:

[0005] Responding to image manipulation;

[0006] According to the dedistortion effect of the image, a target image whose distortion degree meets the requirements is determined, and the target image is used as the image to be operated;

[0007] Performing operation processing on the image to be operated.

[0008] According to a second aspect of an embodiment of the present disclosure, an image operation device is provided, comprising:

[0009] A response module, for responding to image operations;

[0010] A determination module, used to determine a target image whose distortion degree meets the requirements according to the dedistortion effect of the image, and use the target image as the image to be operated;

[0011] A processing module is used to perform operation processing on the image to be operated.

[0012] According to a third aspect of an embodiment of the present disclosure, an electronic device is proposed, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image operation method as described above.

[0013] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is proposed. When instructions in the storage medium are executed by a processor, the processor is enabled to perform the above-mentioned image operation method.

[0014] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is proposed, including a computer program, and when the computer program is executed by a processor, the above-mentioned image operation method is implemented.

[0015] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0016] The image operation method, image operation device and electronic device disclosed in the present invention respond to the image operation and determine the target image whose distortion degree meets the requirements according to the de-distortion effect of the image, take the target image as the image to be operated, and then perform operation processing on the image to be operated. Therefore, the present invention performs further operation processing (such as de-distortion processing) on ​​the image when the distortion degree of the image meets the requirements, which can improve the quality of the image, meet the user's high-quality requirements for the image, and improve the user's use experience.

[0017] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0019] Figure 1 It is about the undistorted and distorted graphs of the chessboard;

[0020] Figure 2 is a flowchart of an image operation method according to an embodiment of the present disclosure;

[0021] Figure 3 is a flow chart for determining the degree of image distortion according to an embodiment of the present disclosure;

[0022] Figure 4 are two ideal checkerboard images of different pixel sizes according to one embodiment of the present disclosure;

[0023] Figure 5 is a schematic diagram of calculating the time-division field of view of a corner point error according to an embodiment of the present disclosure;

[0024] Figure 6 is a block diagram of an image operating device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0025] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0027] In the camera shooting of terminal devices such as mobile phones, due to factors such as imperfect manufacturing and design of camera lenses and assembly errors of camera lens modules, image distortion will be introduced. Distortion will cause straight lines in the image to bend at the edges of the image, such as Figure 1 As shown, it greatly affects the visual perception of the image, and will also affect some back-end applications such as binocular depth calculation and image quality perception.

[0028] Usually, a large number of de-distortion algorithms are applied in camera applications of terminal devices such as mobile phones, but the de-distortion effect of the de-distortion algorithm is usually uncertain. At present, there are more and more applications for images, such as preview, various portrait modes, photo effects, beautification editing and other scenarios, so the requirements for image quality are getting higher and higher.

[0029] To this end, the present disclosure proposes an image operation method, an image operation device and an electronic device, which perform further operation processing (such as de-distortion processing) on ​​the image when the degree of distortion of the image meets the requirements, thereby improving the quality of the image, meeting the user's high-quality requirements for the image, and improving the user's usage experience.

[0030] The image operation method, image operation device, and electronic device according to the embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0031] Figure 2 is a flowchart of an image operation method according to an embodiment of the present disclosure.

[0032] It should be noted that the execution subject of the image operation method of the embodiment of the present disclosure is an image operation device, and the image operation device can be configured in an electronic device so that the electronic device can execute the image operation method.

[0033] like Figure 2 As shown, the image operation method of the embodiment of the present disclosure includes the following steps:

[0034] S201, responding to image operation.

[0035] The image operation includes at least one of the following operations: an image capturing operation, an image display operation, and an image recognition operation.

[0036] For example, when a user uses a camera of an electronic device to perform an image capture operation, the electronic device will use a de-distortion algorithm such as a distortion model algorithm and a 21-point coefficient method to de-distort the image.

[0037] For another example, when a user uses an electronic device such as a camera, mobile phone, computer, or wearable device to view or display an image, the electronic device first uses a dedistortion algorithm to dedistort the original image.

[0038] For another example, when a user needs to use software or a program in an electronic device to identify, analyze and process an original image and extract information or features from the original image, the electronic device also needs to first use a distortion algorithm to dedistort the image.

[0039] S202, according to the dedistortion effect of the image, determining a target image whose distortion degree meets the requirements, and using the target image as the image to be operated.

[0040] In this step, after the image is subjected to image distortion processing, the degree of distortion used to indicate the image dedistortion effect is determined according to the corner point spacing between the image and the chessboard image, and the image whose distortion degree satisfies the corner point spacing threshold range of the chessboard image (the corner point spacing threshold range can be set according to actual needs) is used as the target image, and the target image is used as the image to be operated that needs to be dedistorted again using the dedistortion algorithm.

[0041] S203, performing operation processing on the image to be operated.

[0042] This step continues to use a dedistortion algorithm such as a portrait algorithm and an LDC (Lens Distortion Correction) algorithm to dedistort the image to be operated until the degree of distortion of the image no longer meets the requirements, and the dedistortion processing of the image is completed.

[0043] Therefore, the image operation method disclosed in the present invention can improve the quality of the image, meet the user's high-quality requirements for the image, and improve the user's usage experience by performing further operation processing (such as de-distortion processing) on ​​the image when the degree of image distortion meets the requirements.

[0044] Figure 3 The present invention is a flowchart of determining the degree of image distortion according to an embodiment of the present disclosure.

[0045] It should be noted that a checkerboard is displayed in the image of the embodiment of the present disclosure.

[0046] like Figure 3 As shown, the process of determining the degree of image distortion in the embodiment of the present disclosure includes the following steps:

[0047] S301, performing corner point detection on the image to obtain target corner point coordinates.

[0048] In step S301, a corner point extraction algorithm can be used to extract the corner points of the checkerboard in the image as target corner points, and generate the target corner point coordinates. The corner point extraction algorithm generally uses an adaptive threshold algorithm in the related art, an algorithm based on sub-pixel corner point extraction, and the corner point coordinates require sub-pixel accuracy.

[0049] In order to ensure that the corner points at the edge of the image can also be completely extracted, because if the corner points are missing, it will affect the subsequent determination of the degree of image distortion by the edge field of view. To this end, before executing step S302, the number of all corner points in the checkerboard calibration plate is also obtained as the target number, and the number of all target corner point coordinates is obtained as the first number to be detected. If the first number to be detected reaches the target number, it means that all the corner points in the target image have been extracted. At this time, the target corner point coordinates are stored, where the data format of the corner point coordinates can be (r, c, x, y), which means that the current corner point is located in the rth row and the cth column, and the specific pixel coordinates are (x, y).

[0050] For example, the size of the chessboard in the image is counted as M rows and N columns, which can be expressed as (M, N), where the number of columns is the number of corner points N in the horizontal direction of the chessboard, and the number of rows is the number of corner points M in the vertical direction of the chessboard.

[0051] It should be noted that, in order to facilitate testing, a checkerboard calibration plate is usually photographed. Before leaving the factory, the image acquisition process during the de-distortion effect detection process can be as follows:

[0052] Use a holder to fix the phone, connect the phone to the computer via a data cable, and use the computer to control the phone camera to take pictures;

[0053] Place a checkerboard calibration plate in front of the mobile phone camera so that the checkerboard calibration plate completely covers the camera's field of view. Specifically, adjust the camera's position and angle so that the checkerboard calibration plate is clearly visible in the camera preview interface. Then check whether the checkerboard calibration plate in the camera preview interface covers the entire image field of view. If the checkerboard calibration plate does not completely cover the entire image field of view, continue to adjust the camera's position and angle until the checkerboard calibration plate completely covers the entire image field of view.

[0054] When taking images, the checkerboard calibration plate is placed at different distances from the front of the mobile phone camera, such as 0.5m, 1.0m, and 2.0m, and multiple (3-4) images can be obtained each time.

[0055] After the mobile phone camera captures the image, the image is dedistorted using the dedistortion algorithm used by the backend project, and the target image obtained after dedistortion is stored for subsequent determination of the degree of distortion of the target image.

[0056] It should also be noted that the multi-distance shooting is to simulate various situations in real applications in order to fully evaluate the de-distortion effect.

[0057] S302, determining a chessboard image according to the target corner point coordinates.

[0058] It should be noted that the ideal chessboard refers to a chessboard pattern defined in theory or mathematical models. It has a perfect geometric shape, uniform square arrangement and precise corner point positions. In practice, the ideal chessboard can be used to evaluate and correct the distortion and distortion in real chessboard images.

[0059] Assume that the size of a chessboard is known to be (M, N), and a pixel size b can be used to control the ideal chessboard model. The pixel width of each chessboard is b×b, and the actual corner point coordinates are (x×b, y×b), such as Figure 3 As shown, two ideal chessboard images with different b are shown.

[0060] The checkerboard calibration plate we photographed belongs to an ideal calibration plate Pb. In fact, the image I of the checkerboard calibration plate is obtained by adding distortion to the ideal calibration plate Pb through a single-value change matrix H (central projection process). Therefore, the above process can be summarized as the following formula (1):

[0061] I=distortion(H×P b ) (1)

[0062] Among them, distortion is the distortion coefficient.

[0063] Through the above formula (1), we can conclude that there is a one-to-one correspondence between the corner points of the ideal calibration plate Pb and the corner points of the image. Therefore, we can obtain an ideal chessboard through the correspondence between the corner points and by constraining the reprojection error.

[0064] After estimating the single-sex change matrix H and the size parameter b of the ideal chessboard, the chessboard image can be obtained.

[0065] Based on the above content, the process of obtaining a chessboard image includes: first obtaining the total number of rows and columns of chessboards in the image, and setting a calibration image (i.e., an image corresponding to an ideal calibration plate Pb without any transformation) according to the total number of rows and columns and a pixel size b to be determined, and performing corner point detection on the calibration image to obtain calibration corner point coordinates, wherein the calibration corner point coordinates correspond to the target corner point coordinates in terms of the number of rows and columns, and then establishing a constraint relationship between the target corner point coordinates and the calibration corner point coordinates through a homography transformation matrix to be determined and a pixel size to be determined, and finally using the LM algorithm to determine the homography transformation matrix and the pixel size by minimizing the distance between the target corner point coordinates and the calibration corner point coordinates, and then generating a chessboard image according to the homography transformation matrix and the pixel size.

[0066] For example, according to the chessboard size (M, N) and the pixel size b to be determined, an ideal chessboard is set, and the coordinates (r, c, x1, y1) of each corner point in the ideal chessboard are obtained, where the coordinates represent: the corner point is located at the rth row, the cth column, and the specific pixel coordinates are (x, y); then the coordinates (r, c, x2, y2) of the corner point to be detected obtained in the above step S202 are obtained, and the corresponding corner point coordinates at the corresponding position in the image are matched one by one; finally, constraint equations are established, and two sets of constraint equations can be established for each calibrated corner point and the corner point to be detected, and the calibrated corner points in the calibrated image are mapped to the image through the homography transformation matrix H and the pixel size b, and the LM algorithm is used to solve the optimal homography transformation matrix H and pixel size b through the minimum reprojection error.

[0067] Among them, the corner point and homography matrix formula are as follows (2):

[0068]

[0069] Among them, the homography transformation matrix

[0070] The relationship between the calibration corner point coordinates and the pixel size b is as follows (3):

[0071]

[0072] Thus, the homography transformation matrix H and the pixel size b can be obtained. After obtaining the homography transformation matrix H and the pixel size b, the influence of the distortion algorithm can be separated separately, so that the chessboard image can be obtained. The process of obtaining the chessboard image includes: generating a calibration image according to the total number of rows, the total number of columns and the pixel size b, and then transforming the calibration image through the homography transformation matrix H to obtain the chessboard image.

[0073] S303, performing corner point detection on the chessboard image to obtain reference corner point coordinates.

[0074] The method for obtaining the reference corner point coordinates in step S303 may adopt the content described in step S301 above, and the details will not be repeated here.

[0075] S304: Determine the degree of distortion according to the distance between the target corner point coordinates and the reference corner point coordinates.

[0076] In step S304, the distance between the quantized target corner point coordinates and the reference corner point coordinates is used to obtain the degree of distortion.

[0077] It can be understood that in the above step S302, the ideal chessboard is obtained, and the corresponding homography transformation matrix is ​​obtained at the same time. Therefore, we transform the ideal chessboard through the H matrix and map it to the real picture plane to obtain the coordinates of the chessboard corner points (x1, y1) at this time. At the same time, the ideal chessboard is directly mapped to the real picture plane, and the corner point coordinates are (x2, y2); if there is no distortion in the image, then (x1, y1) and (x2, y2) should coincide; however, due to the distortion of the real shooting, there is a distance difference between the two, that is, the corner point deviation. Therefore, the de-distortion effect of the de-distortion algorithm can be judged by the corner point deviation.

[0078] As an implementable method of step S304, it includes: obtaining the number of all target corner point coordinates as the first number to be detected; calculating the pixel distance between the corner point pairs of the first number to be detected; wherein the corner point pair includes a pair of target corner point coordinates and reference corner point coordinates whose rows and columns are the same; summing the pixel distances of the first number to be detected to obtain a first pixel distance sum value; calculating the ratio between the first pixel distance sum value and the first number to be detected as the overall corner point deviation; and determining the overall distortion degree of the image based on the overall corner point deviation.

[0079] In order to more conveniently express the degree of overall image distortion, after obtaining the overall corner deviation, the resolution of the target image is obtained, and the overall angle deviation or the corner deviation of each field of view is normalized according to the resolution.

[0080] The above process can be described by the following formula to calculate the first pixel distance and value d all The formula is as follows (3):

[0081]

[0082] Wherein, num is the first number to be detected.

[0083] Calculate the overall corner deviation d error The formula is as follows (4):

[0084]

[0085] Among them, resolution is the resolution of the target image.

[0086] Another implementable method of step S304 includes: dividing the image into multiple fields of view; obtaining the number of target corner point coordinates in each field of view as the second number to be detected; calculating the pixel distance between the corner point pairs of the second number to be detected in each field of view; wherein the corner point pair includes a pair of target corner point coordinates and reference corner point coordinates in which the rows and columns of the corner points are the same; summing the pixel distances of the second number to be detected in each field of view to obtain a second pixel distance sum value; calculating the ratio between the second pixel distance sum value in each field of view and the second number to be detected as the corner point deviation of the corresponding field of view; and determining the degree of distortion of each area of ​​the image according to the corner point deviation of each field of view.

[0087] In order to more conveniently express the degree of distortion of each area of ​​the image, after obtaining the corner point deviation of each field of view, the resolution of the image is obtained, and the corner point deviation of each field of view is normalized according to the resolution.

[0088] For example, Figure 5 As shown, the image can be divided into 10 fields of view, and the deviation of the corner points in each field of view is counted separately to measure the distortion effect in the field of view, which is more conducive to analyzing the degree of distortion in each area of ​​the image.

[0089] It should be noted that before factory testing, the present invention can process multiple images taken at the same distance separately to obtain the corner point deviation of each image, and then obtain the degree of image distortion by taking the average value.

[0090] The degree of distortion determined in the present invention is achieved by quantifying the effect of the de-distortion algorithm. The back-end application can use specific standards to evaluate whether the degree of distortion meets the expectations of the project, ensuring that the quality of the input image meets the requirements. It also helps to solve the problems caused by camera image de-distortion and improve image quality. At the same time, it can be extended to the image processing process when shooting with other types of cameras, as well as image processing software.

[0091] For example, the first application scenario of determining the degree of distortion of the present disclosure is: the degree of distortion of images taken by ultra-wide-angle cameras. Ultra-wide-angle cameras usually have a very large field of view. Due to the wide-angle design, they may introduce severe radial distortion in the image, especially at the edge of the image. De-distortion processing is crucial for ultra-wide-angle cameras to ensure that objects and geometric shapes in the image present real-world shapes. The present disclosure can be used to measure the degree of distortion of the image after de-distortion, and evaluate the quality of the image as a whole or in a region based on the degree of distortion, thereby evaluating the image quality effect of the ultra-wide-angle camera.

[0092] The second application scenario of determining the degree of distortion disclosed in the present invention is to evaluate the degree of distortion of images in the preprocessing of binocular depth calculation. In binocular vision, distortion correction is a key task because it can ensure the accuracy of image matching between the left and right cameras. Only with good distortion-free left and right camera images can the quality of binocular image row alignment be guaranteed to be excellent, thereby improving the accuracy of depth calculation.

[0093] Of course, the determination of the de-distortion effect disclosed in the present invention also includes other scenarios in which the de-distortion algorithm is applied, and specific examples will not be given here one by one.

[0094] In summary, the image operation method of the embodiment of the present disclosure responds to the image operation and determines the target image whose distortion degree meets the requirements according to the dedistortion effect of the image, takes the target image as the image to be operated, and then performs operation processing on the image to be operated. Therefore, the present disclosure performs further operation processing (such as dedistortion processing) on ​​the image when the distortion degree of the image meets the requirements, which can improve the quality of the image, meet the user's high-quality requirements for the image, and improve the user's usage experience.

[0095] Figure 6 is a block diagram of an image operating device according to an embodiment of the present disclosure.

[0096] like Figure 6 As shown, the image operation device 600 of the embodiment of the present disclosure includes:

[0097] A response module 610, for responding to image operations;

[0098] A determination module 620 is used to determine a target image whose distortion degree meets the requirements according to the dedistortion effect of the image, and use the target image as the image to be operated;

[0099] The processing module 640 is used to perform operation processing on the image to be operated.

[0100] In one embodiment of the present disclosure, the image is an image obtained by dedistortion processing.

[0101] In one embodiment of the present disclosure, the image operation includes at least one of the following operations:

[0102] Image capture operation;

[0103] Image display operation;

[0104] Image recognition operations.

[0105] In one embodiment of the present disclosure, the determination module 620 is used to determine, based on the dedistortion effect of the image, whether the distortion degree of the image meets the requirements of the target image, including:

[0106] Determining the degree of distortion of the image according to the corner point spacing between the image and the chessboard image; wherein the degree of distortion is used to indicate the de-distortion effect;

[0107] An image whose distortion degree satisfies a corner point spacing threshold range of the chessboard image is taken as a target image.

[0108] In one embodiment of the present disclosure, the determination module 620 is used to determine the degree of distortion of the image according to the corner point distance between the image and the chessboard image, including:

[0109] Performing corner point detection on the image to obtain target corner point coordinates;

[0110] Determining the chessboard image according to the target corner point coordinates;

[0111] Performing corner point detection on the chessboard image to obtain reference corner point coordinates;

[0112] The degree of distortion is determined according to the distance between the target corner point coordinates and the reference corner point coordinates.

[0113] In one embodiment of the present disclosure, a checkerboard is displayed in the target image, and the determination module 620 is used to determine the checkerboard image according to the target corner point coordinates, including:

[0114] Get the total number of rows and columns of the chessboard in the image;

[0115] According to the total number of rows and columns and the pixel size to be determined, a calibration image is set;

[0116] Perform corner point detection on the calibration image to obtain calibration corner point coordinates; wherein the calibration corner point coordinates correspond to the target corner point coordinates in terms of row and column numbers;

[0117] By means of the homography transformation matrix to be determined and the pixel size to be determined, a constraint relationship between the target corner point coordinates and the calibration corner point coordinates is established;

[0118] The LM algorithm is used to determine the homography transformation matrix and pixel size by minimizing the difference between the target corner point coordinates and the calibration corner point coordinates;

[0119] Determine the chessboard image based on the homography transformation matrix and pixel size.

[0120] In one embodiment of the present disclosure, when the determination module is used to determine the chessboard image according to the homography transformation matrix and the pixel size, it includes:

[0121] Determine the calibration image according to the total number of rows, the total number of columns and the pixel size;

[0122] The calibration image is transformed through the homography transformation matrix to obtain the chessboard image.

[0123] In one embodiment of the present disclosure, the determination module 620 is used to determine the degree of distortion according to the distance between the target corner point coordinates and the reference corner point coordinates, including:

[0124] Obtain the number of all target corner point coordinates as the first number to be detected;

[0125] Calculate the pixel distances between the first number of corner point pairs to be detected; wherein the corner point pairs include a pair of target corner point coordinates and reference corner point coordinates in which the corner points are located in the same row and column;

[0126] Summing the first number of pixel distances to be detected to obtain a first pixel distance sum value;

[0127] Calculate the ratio between the first pixel distance and the first number to be detected as the overall corner point deviation;

[0128] According to the overall corner point deviation, the overall distortion degree of the image is determined.

[0129] In another embodiment of the present disclosure, the determination module 620 is used to determine the degree of distortion according to the distance between the target corner point coordinates and the reference corner point coordinates, including:

[0130] Divide the image into multiple fields of view;

[0131] Obtaining the number of target corner point coordinates in each field of view as the second number to be detected;

[0132] Calculate the pixel distances between the second number of corner point pairs to be detected in each field of view; wherein the corner point pairs include a pair of target corner point coordinates and reference corner point coordinates in which the corner points are located in the same row and column;

[0133] Summing the pixel distances of the second number to be detected in each field of view to obtain a second pixel distance sum value;

[0134] Calculate the ratio between the second pixel distance and the second number to be detected in each field of view as the corner point deviation of the corresponding field of view;

[0135] According to the corner point deviation of each field of view, the degree of distortion of each area of ​​the image is determined.

[0136] In one embodiment of the present disclosure, after obtaining the overall angle deviation or the corner point deviation of each field of view, the determination module 620 is further used to:

[0137] Get the resolution of the image;

[0138] Depending on the resolution, the overall angular deviation or the corner point deviation of each field of view is normalized.

[0139] In one embodiment of the present disclosure, a checkerboard is displayed on the calibration board, and before the determination module 620 is used to determine the checkerboard image, it is also used to:

[0140] Get the number of all corner points in the calibration plate as the target number;

[0141] Obtain the number of all target corner point coordinates as the first number to be detected;

[0142] In response to the first to-be-detected number reaching the target number, the target corner point coordinates are stored.

[0143] It should be noted that for details not disclosed in the image operation device of the embodiment of the present disclosure, please refer to the details disclosed in the image operation method of the embodiment of the present disclosure, and the details will not be repeated here.

[0144] According to the image operation device of the embodiment of the present disclosure, the response module responds to the image operation, the determination module determines the target image whose distortion degree meets the requirements according to the de-distortion effect of the image, takes the target image as the image to be operated, and performs operation processing on the image to be operated through the processing module. Therefore, when the distortion degree of the image meets the requirements, the device performs further operation processing (such as de-distortion processing) on ​​the image, which can improve the quality of the image, meet the user's high-quality requirements for the image, and improve the user's use experience.

[0145] In order to implement the above embodiments, the present disclosure also provides an electronic device.

[0146] The electronic device of an embodiment of the present disclosure includes: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the above method.

[0147] The electronic device of the disclosed embodiment, by adopting the above-mentioned image operation method, performs further operation processing (such as de-distortion processing) on ​​the image when the degree of image distortion meets the requirements, thereby improving the quality of the image, meeting the user's high-quality requirements for the image, and improving the user's usage experience.

[0148] In order to implement the above embodiments, the present disclosure also proposes a computer-readable storage medium.

[0149] When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method.

[0150] The computer-readable storage medium of the disclosed embodiment, by adopting the above-mentioned image operation method, performs further operation processing (such as de-distortion processing) on ​​the image when the degree of distortion of the image meets the requirements, thereby improving the quality of the image, meeting the user's high-quality requirements for the image, and improving the user's usage experience.

[0151] When the computer program product is executed by a processor of an electronic device, the electronic device is enabled to execute the above method.

[0152] The computer-aided product of the disclosed embodiment, by adopting the above-mentioned image operation method, performs further operation processing (such as de-distortion processing) on ​​the image when the degree of distortion of the image meets the requirements, thereby improving the quality of the image, meeting the user's high-quality requirements for the image, and improving the user's usage experience.

[0153] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0154] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0155] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.

[0156] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0157] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0158] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0159] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0160] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.

Claims

1. An image operation method, characterized in that: include: Responding to image manipulation; According to the dedistortion effect of the image, a target image whose distortion degree meets the requirements is determined, and the target image is used as the image to be operated; Performing operation processing on the image to be operated.

2. The method according to claim 1, characterized in that The image is an image obtained through de-distortion processing.

3. The method according to claim 1, characterized in that The image operation includes at least one of the following operations: Image capture operation; Image display operation; Image recognition operations.

4. The method according to claim 1, characterized in that: The step of determining, based on the dedistortion effect of the image, a target image whose distortion degree meets the requirements includes: Determining the degree of distortion of the image according to the corner point spacing between the image and the chessboard image; wherein the degree of distortion is used to indicate the dedistortion effect; In response to the distortion degree satisfying an image within a threshold range of a corner point spacing of the chessboard image, an image is used as the target image.

5. The method according to claim 4, characterized in that Determining the degree of distortion of the image according to the corner point spacing between the image and the chessboard image includes: Performing corner point detection on the image to obtain target corner point coordinates; Determining the chessboard image according to the target corner point coordinates; Performing corner point detection on the chessboard image to obtain reference corner point coordinates; The degree of distortion is determined according to the distance between the target corner point coordinates and the reference corner point coordinates.

6. The method according to claim 5, characterized in that The image shows a chessboard, and determining the chessboard image according to the target corner point coordinates includes: Obtaining the total number of rows and columns of the chessboard in the image; Setting a calibration image according to the total number of rows and the total number of columns, and a pixel size to be determined; Performing corner point detection on the calibration image to obtain calibration corner point coordinates; wherein the calibration corner point coordinates correspond to the target corner point coordinates in terms of the number of rows and columns; Establishing a constraint relationship between the target corner point coordinates and the calibration corner point coordinates through the homography transformation matrix to be determined and the pixel size to be determined; Using the LM algorithm, the distance between the target corner point coordinates and the calibration corner point coordinates is minimized to determine the homography transformation matrix and the pixel size; The chessboard image is determined according to the homography transformation matrix and the pixel size.

7. The method according to claim 6, characterized in that Determining the chessboard image according to the homography transformation matrix and the pixel size includes: Determining the calibration image according to the total number of rows, the total number of columns and the pixel size; The calibration image is transformed by using the homography transformation matrix to obtain the chessboard image.

8. The method according to claim 5, characterized in that The step of determining the degree of distortion according to the distance between the target corner point coordinates and the reference corner point coordinates includes: Obtaining the number of all target corner point coordinates as the first number to be detected; Calculating the pixel distances between the first number of corner point pairs to be detected; wherein the corner point pairs include a pair of the target corner point coordinates and the reference corner point coordinates in which the corner points are located in the same row and column; summing the pixel distances of the first number to be detected to obtain a first pixel distance sum value; Calculate the ratio between the first pixel distance and value and the first number to be detected as the overall corner point deviation; The overall distortion degree of the image is determined according to the overall corner point deviation.

9. The method according to claim 5, characterized in that The step of determining the degree of distortion according to the distance between the target corner point coordinates and the reference corner point coordinates includes: dividing the target image into a plurality of fields of view; Acquire the number of the target corner point coordinates in each of the fields of view as the second number to be detected; Calculating the pixel distances between the second number of corner point pairs to be detected in each of the fields of view; wherein the corner point pairs include a pair of the target corner point coordinates and the reference corner point coordinates in the same row and column; summing the pixel distances of the second number to be detected in each of the fields of view to obtain a second pixel distance sum value; Calculating a ratio between the second pixel distance and value in each of the fields of view and the second number to be detected as a corner point deviation of the corresponding field of view; The degree of distortion of each area of ​​the image is determined according to the corner point deviations of each of the fields of view.

10. The method according to claim 8 or 9, characterized in that: After obtaining the overall angle deviation or the corner point deviation of each field of view, it also includes: Acquire the resolution of the image; According to the resolution, the overall angle deviation or the corner point deviation of each field of view is normalized.

11. The method according to claim 6, characterized in that The chessboard is displayed on the calibration board, and before determining the chessboard image according to the target corner point coordinates, the method further includes: Obtaining the number of all corner points in the calibration plate as the target number; Obtaining the number of all target corner point coordinates as the first number to be detected; In response to the first to-be-detected number reaching the target number, the target corner point coordinates are stored.

12. An image operation device, characterized in that: include: A response module, for responding to image operations; A determination module, used to determine a target image whose distortion degree meets the requirements according to the dedistortion effect of the image, and use the target image as the image to be operated; A processing module is used to perform operation processing on the image to be operated.

13. The device according to claim 12, characterized in that The image is an image obtained through de-distortion processing.

14. The device according to claim 12, characterized in that The image operation includes at least one of the following operations: Image capture operation; Image display operation; Image recognition operations.

15. The device according to claim 12, characterized in that The determining module is used to determine, based on the dedistortion effect of the image, whether the distortion degree of the image meets the requirements of the target image, including: Determining the degree of distortion of the image according to the corner point spacing between the image and the chessboard image; wherein the degree of distortion is used to indicate the dedistortion effect; In response to the distortion degree satisfying an image within a threshold range of a corner point spacing of the chessboard image, an image is used as the target image.

16. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image operation method according to any one of claims 1 to 11.

Images