Lens distortion parameter determination method, device and computer equipment of photographing apparatus

Abstract

This application relates to a method, apparatus, and computer device for determining lens distortion parameters of a shooting device. The method includes: performing distortion correction processing on an image of a first original image showing a first lens field of view overlapping with another image of a second original image showing a second lens field of view overlapping with yet another image using at least two lens distortion parameters, resulting in at least two image pairs; and determining the lens distortion parameter corresponding to the image pair with the smallest difference as the target lens distortion parameter of the shooting device. This method enables the rapid and accurate determination of the optimal target lens distortion parameter for the shooting device, whether it is equipped with a protective housing or not, based on the target lens distortion parameter in a single identification process. It also determines whether the shooting device is equipped with a housing and the distortion correction parameters for images captured by the shooting device with a housing, eliminating the need for manual selection.

Description

Technical Field

[0001] This application relates to the technical field of lens distortion parameter determination for shooting equipment, and in particular to a method, apparatus and computer device for determining lens distortion parameters for shooting equipment. Background Technology

[0002] In scenarios where the lens of a shooting device is easily contaminated or damaged by the external environment, it is generally necessary to cover the shooting device or its lens with a transparent protective shell to protect it from contamination or damage. For example, when shooting at amusement parks or underwater, most shooting devices require a transparent protective shell to capture images. Because the protective shell alters the light propagation path, it causes distortion in the captured images. Before post-processing the images (stitching or distortion correction), it is generally necessary to select the appropriate distortion correction parameters based on the type of protective shell worn by the shooting device during image capture. Currently, existing software typically determines distortion correction parameters by manually selecting the type of protective shell worn by the shooting device or by manually inputting the distortion parameters.

[0003] Therefore, it can be seen that the current method of manually determining the distortion correction parameters of captured images is very unintelligent and cumbersome, which raises the usage threshold of software for determining lens distortion parameters of shooting equipment and results in a poor user experience. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, and computer device for determining the lens distortion parameters of a shooting device that can automatically identify the type of protective housing worn by the shooting device to automatically determine the distortion correction parameters of the video, in order to address the above-mentioned technical problems.

[0005] In a first aspect, this application provides a method for determining lens distortion parameters of an imaging device, the method comprising:

[0006] Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens;

[0007] Determine the overlapping region of the field of view in the first original image and the second original image to obtain the overlapping region of the field of view of the first lens corresponding to the first original image, and the overlapping region of the field of view of the second lens corresponding to the second original image.

[0008] At least two lens distortion parameters are used to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view, resulting in at least two sets of image pairs.

[0009] Identify at least two image pairs where the difference between the first-shot distortion-free image and the second-shot distortion-free image is the smallest.

[0010] The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0011] In one embodiment, at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing.

[0012] Determining the lens distortion parameters corresponding to the image pair with the smallest difference as the target lens distortion parameters for the imaging device also includes:

[0013] If the image with the smallest difference is obtained based on the lens distortion parameters with a protective housing, then the shooting device is determined to be in a state with a protective housing.

[0014] The image pair with the smallest difference corresponds to the lens distortion parameter of the lens with a protective housing, which is then determined as the target lens distortion parameter of the shooting device.

[0015] In one embodiment, at least two lens distortion parameters include at least two lens distortion parameters corresponding to different protective housing types;

[0016] The method further includes: determining the type of housing worn by the shooting device based on the housing type corresponding to the lens distortion parameters of the lens with a protective housing.

[0017] In one embodiment, at least two lens distortion parameters are used to perform distortion correction processing on the images of the overlapping regions of the first and second lens field of view, resulting in at least two image pairs, including:

[0018] The first lens field of view overlap region image and the second lens field of view overlap region image are reprojected respectively to obtain the first lens field of view overlap region image and the second lens field of view overlap region image after reprojection.

[0019] At least two lens distortion parameters are used to perform distortion correction on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection, resulting in at least two sets of image pairs.

[0020] In one embodiment, determining the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image among at least two image pairs includes:

[0021] Calculate the image quality evaluation index for the first shot distortion-free image and the second shot distortion-free image in at least two image pairs respectively;

[0022] Image quality evaluation metrics are used to characterize the image pair with the highest similarity between the first-shot distortion-free image and the second-shot distortion-free image as the image pair with the smallest difference.

[0023] In one embodiment, determining the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image among at least two image pairs includes:

[0024] Calculate the disparity score between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs;

[0025] Obtain the image pair with the smallest difference between the first-lens distortion-free image and the second-lens distortion-free image, as represented by the disparity score.

[0026] In one embodiment, the disparity score between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs is calculated, including:

[0027] Based on a predefined scoring function, determine the disparity score between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs; the scoring function is a functional expression that takes the disparity of image blocks or pixels in the image pair as a variable and characterizes the magnitude of the difference between the two images in the image pair.

[0028] In one embodiment, determining the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image among at least two image pairs includes:

[0029] Calculate the image quality evaluation index for the first shot distortion-free image and the second shot distortion-free image in at least two image pairs respectively;

[0030] The image quality evaluation index characterizes the image pair with the highest similarity between the first-lens distortion-free image and the second-lens distortion-free image as the first recognition result;

[0031] Calculate the disparity score between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs;

[0032] The image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image, represented by the disparity score, is taken as the second recognition result;

[0033] Based on the first and second recognition results, determine the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image from at least two image pairs.

[0034] Secondly, this application also provides a method for determining lens distortion parameters of an imaging device, the method comprising:

[0035] Acquire multiple frames of first raw images captured by the first lens of the shooting device and multiple frames of second raw images captured by the second lens;

[0036] Determine the overlapping area of ​​the field of view in the first original image and the second original image at multiple frames at the same time, obtain the overlapping area of ​​the field of view in the first original image at multiple frames at the same time, obtain the overlapping area of ​​the field of view of the first lens, and the overlapping area of ​​the field of view of the second lens corresponding to the second original image at multiple frames at the same time.

[0037] At least two lens distortion parameters are used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view at multiple frames simultaneously, to obtain at least two sets of image pairs.

[0038] Identify the image pair with the smallest difference between the first-shot distortion-free image and the second-shot distortion-free image from at least two image pairs at the same time.

[0039] The lens distortion parameters corresponding to the images with the smallest differences at the same moment are determined as the alternative lens distortion parameters for the shooting device;

[0040] Multiple candidate lens distortion parameters are classified, and the candidate lens distortion parameter with the most classification results is determined as the target lens distortion parameter of the shooting device.

[0041] In one embodiment, at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing.

[0042] The lens distortion parameters corresponding to the images with the smallest differences at the same moment are also used to determine the alternative lens distortion parameters for the shooting device.

[0043] If the image with the smallest difference is obtained based on the lens distortion parameters with a protective housing, then the shooting device is determined to be in a state with a protective housing.

[0044] The lens distortion parameters corresponding to the image pairs with the smallest differences, which are protected by the lens housing, are determined as the alternative lens distortion parameters for the shooting equipment.

[0045] In one embodiment, at least two lens distortion parameters include at least two lens distortion parameters corresponding to different protective housing types;

[0046] The method further includes: determining the type of housing worn by the shooting device based on the housing type corresponding to the lens distortion parameters of the lens with a protective housing.

[0047] In one embodiment, at least two lens distortion parameters are used to perform distortion correction processing on the overlapping regions of the first and second lens field of view images at multiple frames simultaneously, resulting in at least two image pairs, including:

[0048] The overlapping area images of the first and second lens fields of view at multiple frames simultaneously are reprojected to obtain the reprojected overlapping area images of the first and second lens fields of view.

[0049] At least two lens distortion parameters are used to perform distortion correction on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection, resulting in at least two sets of image pairs.

[0050] Thirdly, this application also provides a lens distortion parameter determination device for an imaging device, the device comprising:

[0051] The acquisition module is used to acquire a first raw image captured by the first lens and a second raw image captured by the second lens of the shooting device;

[0052] The image extraction module is used to determine the overlapping area image of the field of view in the first original image and the second original image, and to obtain the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image.

[0053] The distortion correction module is used to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view using at least two lens distortion parameters, to obtain at least two sets of image pairs.

[0054] The recognition module is used to identify the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs;

[0055] The determination module is used to determine the lens distortion parameters corresponding to the image pairs with the smallest differences as the target lens distortion parameters of the shooting device.

[0056] In one embodiment, the acquisition module is further configured to acquire multiple frames of first original images captured by the first lens of the shooting device and multiple frames of second original images captured by the second lens;

[0057] The image extraction module is also used to determine the overlapping area image of the field of view in the first original image and the second original image at multiple frames at the same time, to obtain the overlapping area image of the field of view of the first original image at multiple frames at the same time, and the overlapping area image of the field of view of the second original image at multiple frames at the same time.

[0058] The distortion correction module is also used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view at multiple frames simultaneously using at least two lens distortion parameters, so as to obtain at least two sets of image pairs.

[0059] The recognition module is also used to determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image among at least two image pairs at the same time;

[0060] The determination module is also used to determine the lens distortion parameters corresponding to the image pairs with the smallest differences at the same moment as the candidate lens distortion parameters of the shooting device; classify the candidate lens distortion parameters, and determine the candidate lens distortion parameter with the most classification results as the target lens distortion parameter of the shooting device.

[0061] Fourthly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0062] Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens;

[0063] Determine the overlapping region of the field of view in the first original image and the second original image to obtain the overlapping region of the field of view of the first lens corresponding to the first original image, and the overlapping region of the field of view of the second lens corresponding to the second original image.

[0064] At least two lens distortion parameters are used to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view, resulting in at least two sets of image pairs.

[0065] Identify at least two image pairs where the difference between the first-shot distortion-free image and the second-shot distortion-free image is the smallest.

[0066] The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0067] Fifthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0068] Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens;

[0069] Determine the overlapping region of the field of view in the first original image and the second original image to obtain the overlapping region of the field of view of the first lens corresponding to the first original image, and the overlapping region of the field of view of the second lens corresponding to the second original image.

[0070] At least two lens distortion parameters are used to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view, resulting in at least two sets of image pairs.

[0071] Identify at least two image pairs where the difference between the first-shot distortion-free image and the second-shot distortion-free image is the smallest.

[0072] The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0073] Sixthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0074] Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens;

[0075] Determine the overlapping region of the field of view in the first original image and the second original image to obtain the overlapping region of the field of view of the first lens corresponding to the first original image, and the overlapping region of the field of view of the second lens corresponding to the second original image.

[0076] At least two lens distortion parameters are used to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view, resulting in at least two sets of image pairs.

[0077] Identify at least two image pairs where the difference between the first-shot distortion-free image and the second-shot distortion-free image is the smallest.

[0078] The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0079] The aforementioned method, apparatus, and computer equipment for determining lens distortion parameters of the shooting device determine the overlapping area image of the field of view in the first original image and the second original image, obtaining the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image; perform distortion correction processing on the overlapping area images of the first and second lens field of view using at least two lens distortion parameters, obtaining at least two image pairs; determine the lens distortion parameter corresponding to the image pair with the smallest difference as the target lens distortion parameter of the shooting device; based on the target lens distortion parameter, the optimal target lens distortion parameter can be quickly and accurately determined whether the shooting device is wearing a protective shell or not through a single identification; based on the target lens distortion parameter, it is also possible to determine whether the shooting device is wearing a protective shell and the type of protective shell, without the need for manual selection of whether the shooting device is wearing a protective shell or the target lens distortion parameter of the shooting device. Attached Figure Description

[0080] Figure 1This is an application environment diagram of a method for determining lens distortion parameters of a shooting device in one embodiment.

[0081] Figure 2 This is a flowchart illustrating a method for determining lens distortion parameters of an imaging device in one embodiment;

[0082] Figure 3 This is a schematic diagram of a first original image and a second original image in one embodiment;

[0083] Figure 4 This is a schematic diagram of the process for obtaining at least two sets of image pairs in another embodiment;

[0084] Figure 5 This is a flowchart illustrating the process of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among at least two image pairs in one embodiment.

[0085] Figure 6 This is a flowchart illustrating the process of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs, as described in another embodiment.

[0086] Figure 7 This is a flowchart illustrating the process of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs, as described in another embodiment.

[0087] Figure 8 This is a schematic diagram illustrating, in one embodiment, determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among at least two image pairs based on the first identification result and the second identification result;

[0088] Figure 9 This is a flowchart illustrating a method for determining lens distortion parameters of an imaging device in another embodiment;

[0089] Figure 10 This is a schematic diagram illustrating the determination of alternative lens distortion parameters in one embodiment;

[0090] Figure 11 A structural block diagram of a lens distortion parameter determination device for a shooting device in one embodiment;

[0091] Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0092] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0093] The method for determining lens distortion parameters of an imaging device provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the shooting device 102 transmits a first original image captured by a first lens and a second original image captured by a second lens to a computer device 104 via a network, or stores the images in a storage device and transmits them to the computer device 104; the computer device 104 acquires the first original image captured by the first lens and the second original image captured by the second lens; the computer device 104 determines the overlapping area image of the field of view in the first and second original images, obtaining the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image; the computer device 104 performs distortion correction processing on the overlapping area image of the first and second lens field of view using at least two lens distortion parameters, obtaining at least two image pairs; the computer device 104 determines the image pair with the smallest difference between the first and second lens distortion correction images in the at least two image pairs; the computer device 104 determines the lens distortion parameter corresponding to the image pair with the smallest difference as the target lens distortion parameter of the shooting device. The data storage system can store the first raw image, the second raw image, the first lens field-of-view overlapping area image, the second lens field-of-view overlapping area image, at least two sets of image pairs, and distortion correction parameters that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed in the cloud or on other network servers. The shooting device 102 can be, but is not limited to, various fisheye shooting devices, ordinary wide-angle shooting devices, and ultra-wide-angle shooting devices. The computer device 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and servers. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster composed of multiple servers.

[0094] In another embodiment, the lens distortion parameter determination method for the imaging device provided in this application can be applied to, for example... Figure 1In the application environment shown, the shooting device 102 acquires a first original image captured by a first lens and a second original image captured by a second lens; the shooting device 102 determines the overlapping area image of the field of view in the first and second original images, obtaining the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image; the shooting device 102 performs distortion correction processing on the overlapping area image of the first and second lens field of view using at least two lens distortion parameters, obtaining at least two sets of image pairs; the shooting device 102 determines the image pair with the smallest difference between the first and second lens distortion-corrected images in the at least two sets of image pairs; the shooting device 102 determines the lens distortion parameter corresponding to the image pair with the smallest difference as the target lens distortion parameter of the shooting device. The shooting device 102 transmits the distortion correction parameters to the computer device 104 via a network, or stores the distortion parameters in the captured image or video files. The data storage system can store the distortion correction parameters that the server 104 needs to process. The data storage system can be integrated on server 104, or it can be located in the cloud or on other network servers. The imaging device 102 can be, but is not limited to, various fisheye imaging devices, ordinary wide-angle imaging devices, and ultra-wide-angle imaging devices. The computer device 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, portable wearable devices, and servers. Portable wearable devices can be smartwatches, smart bracelets, head-mounted devices, etc. The server can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0095] In one embodiment, such as Figure 2 As shown, a method for determining lens distortion parameters of a shooting device is provided, which can be applied to... Figure 1 Taking the shooting device 102 or computer device 104 as an example, the following steps are included:

[0096] Step 202: Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens.

[0097] The shooting device can be a device with at least two lenses, and the two lenses have overlapping field of view areas. For example, the shooting device can be a wide-angle camera or a fisheye camera. A wide-angle camera is a camera with a lens that has a very wide angle of view; generally, cameras with a focal length of less than 28mm are called wide-angle cameras. A fisheye camera is a special type of wide-angle camera.

[0098] Determining the application scenarios and timing of lens distortion parameters for shooting equipment is possible whenever images captured by the equipment can be obtained. For example, identification can be performed within the shooting equipment when it is powered on or before shooting, on a computer or mobile phone with processing software installed, or before splicing / distortion correction processing of the video captured by the shooting equipment.

[0099] Optionally, the shooting device or computer device acquires the video captured by the shooting device, and extracts a first original image and a second original image captured simultaneously by the first shot and the second shot from the video, such as... Figure 3 As shown, the first original image is denoted as Figure A, and the second original image is denoted as Figure B.

[0100] Step 204: Determine the overlapping region image of the field of view in the first original image and the second original image to obtain the overlapping region image of the first lens field of view corresponding to the first original image and the overlapping region image of the second lens field of view corresponding to the second original image.

[0101] The specific method for identifying the overlapping area of ​​the field of view in the first original image and the second original image is as follows: obtain camera model information from the captured video file, and determine the overlapping area of ​​the field of view in the first original image and the second original image based on the camera model.

[0102] Optionally, such as Figure 3 As shown, the image of the overlapping area of ​​the first lens field of view of the first original image is denoted as image a, and the image of the overlapping area of ​​the second lens field of view of the second original image is denoted as image b. Image a is extracted from the first original image, and image b is extracted from the second original image.

[0103] Step 206: Use at least two lens distortion parameters to perform distortion correction processing on the image of the overlapping area of ​​the first lens's field of view and the image of the overlapping area of ​​the second lens's field of view, to obtain at least two sets of image pairs.

[0104] Among them, at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing, and the at least one lens distortion parameter with a protective housing is different from each other.

[0105] Optionally, such as Figure 3As shown, this example uses the distortion parameters of one lens without a housing and two lenses with housings, where the distortion parameters of the two lenses with housings are different. First, the distortion parameters x of the lens without a housing are used to perform distortion correction on the overlapping area image a of the first lens's field of view and the overlapping area image b of the second lens's field of view, resulting in a first-lens-corrected distortion image a1 and a second-lens-corrected distortion image b1. These two images are denoted as the first image pair. Second, the distortion parameters y of the lens with a housing are used to perform distortion correction on the overlapping area image a of the first lens's field of view and the overlapping area image b of the second lens's field of view, resulting in a first-lens-corrected distortion image a2 and a second-lens-corrected distortion image b2. These two images are denoted as the second image pair. Then, the distortion parameters z of the second lens housing are used to perform distortion correction processing on the overlapping area image a of the first lens field of view and the overlapping area image b of the second lens field of view, respectively, to obtain the first lens distortion correction image a3 and the second lens distortion correction image b3. The first lens distortion correction image a3 and the second lens distortion correction image b3 are recorded as the third image pair.

[0106] Step 208: Determine the image pair with the smallest difference between the first-shot distortion-free image and the second-shot distortion-free image from at least two image pairs.

[0107] The smaller the difference between the first and second lens-corrected images of an image pair, the closer the distortion parameters used in the correction of the first and second lens-corrected images are to the actual values. Therefore, the more likely the wearing state corresponding to the distortion parameters of the image pair is correct. Thus, in this embodiment, the lens distortion parameters corresponding to the image pair with the smallest difference between the first and second lens-corrected images are selected as the target lens distortion parameters for the imaging device.

[0108] The difference between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs can be quantified, for example, by calculating image quality evaluation metrics (PSNR, MSE, etc.) or other algorithms. This embodiment does not limit the method for quantifying the difference between the first-shot distortion-free image and the second-shot distortion-free image.

[0109] Optionally, the imaging device or computer device employs a method to quantify the difference between the first lens-distorted image and the second lens-distorted image, quantifying the difference between the first lens-distorted image and the second lens-distorted image in at least two image pairs, and obtaining the image pair with the smallest difference between the first lens-distorted image and the second lens-distorted image in at least two image pairs.

[0110] Step 210: The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0111] When the first and second lenses of the shooting device are not equipped with protective housings, they exhibit radial and tangential distortion. When the first and second lenses are equipped with protective housings, the housings alter the light propagation path, causing distortion in both the first and second original images. Therefore, regardless of whether the shooting device is equipped with a protective housing, distortion correction is required for both the first and second original images. However, current software typically determines distortion correction parameters manually by selecting the type of protective housing or manually inputting the distortion parameters. This manual method of determining distortion correction parameters is unintelligent and cumbersome, raising the barrier to entry for image processing software and reducing user experience. To automatically identify the target lens distortion parameters of the shooting device, this embodiment uses at least two lens distortion parameters to perform distortion correction on the overlapping areas of the first and second lens field of view, resulting in at least two image pairs. The lens distortion parameter corresponding to the image pair with the smallest difference is determined as the target lens distortion parameter of the shooting device.

[0112] The method for determining the lens distortion parameters of the aforementioned shooting device involves identifying overlapping regions of the field of view in the first and second original images to obtain overlapping regions of the first and second original images, respectively. At least two lens distortion parameters are used to perform distortion correction on both the overlapping regions of the first and second original images, resulting in at least two image pairs. The lens distortion parameter corresponding to the image pair with the smallest difference is determined as the target lens distortion parameter of the shooting device. Based on the target lens distortion parameter, the optimal target lens distortion parameter can be quickly and accurately determined whether the shooting device is equipped with or without a protective housing after a single identification. Furthermore, the target lens distortion parameter can also determine whether the shooting device is equipped with a protective housing and the type of protective housing, eliminating the need for manual selection of whether the shooting device is equipped with a protective housing or the target lens distortion parameter.

[0113] In one embodiment, since different types of protective housings have different distortion parameters, it is necessary to determine whether the shooting device is wearing a protective housing before performing distortion parameter correction, and to determine the corresponding distortion parameters based on the type of protective housing. Existing methods for determining whether the shooting device is wearing a protective housing generally involve manual on-site inspection followed by manual input of the protective housing wearing status. Therefore, the current method of manually determining the protective housing wearing status of the shooting device is very unintelligent and cumbersome. To solve the above problem, this embodiment uses both the distortion parameters of the lens without a protective housing and at least one lens distortion parameter with a protective housing to perform distortion correction on the overlapping area images of the first and second lens fields of view, obtaining at least two image pairs; the image pair with the smallest difference between the first and second lens distortion correction images is determined; if the image pair with the smallest difference is obtained based on the lens distortion parameters with a protective housing, then the shooting device is determined to be wearing a protective housing; the lens distortion parameters with a protective housing corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device.

[0114] In some embodiments, the lens distortion parameters with a protective housing include lens distortion parameters corresponding to at least two different protective housing types. When the shooting device is in a protective housing state, the shooting device or computer device determines the housing type worn by the shooting device based on the housing type corresponding to the lens distortion parameters with a protective housing.

[0115] For example, Figure 3 In the first image pair, the second image pair, and the third image pair, if the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image is the second image pair, and the lens distortion parameter corresponding to the second image pair is the lens distortion parameter z of the lens with the housing, then the lens distortion parameter z corresponding to the second image pair can be determined as the target lens distortion parameter of the shooting device. At the same time, based on the lens distortion parameter corresponding to the second image pair, it is determined that the shooting device is wearing a protective housing, and based on the housing type corresponding to the lens distortion parameter z, the type of protective housing worn by the shooting device is determined.

[0116] Optionally, the shooting device or computer device performs distortion correction processing on the first lens field-of-view overlap area image and the second lens field-of-view overlap area image using both the lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing, resulting in at least two image pairs. The image pair with the smallest difference between the first lens distortion correction image and the second lens distortion correction image is determined from the at least two image pairs. If the image pair with the smallest difference is obtained based on the lens distortion parameter with a protective housing, then the shooting device is determined to be in a protective housing state, and the lens distortion parameter with a protective housing corresponding to the image pair with the smallest difference is determined as the target lens distortion parameter of the shooting device. The shooting device or computer device can pre-store the correspondence between the protective housing type and the distortion parameter in a server or in the shooting device itself. Based on the correspondence between the protective housing type and the distortion parameter, and the known target lens distortion parameter, the type of protective housing worn by the shooting device is determined, and displayed through the display device of the shooting device or computer device, or the protective housing wearing status, target lens distortion parameter, and protective housing type are announced via voice.

[0117] In this embodiment, distortion correction is performed on the images of the overlapping areas of the first and second lens field of view using both the lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing, resulting in at least two image pairs. If the image pair with the smallest difference is obtained based on the lens distortion parameter with a protective housing, then the shooting device is determined to be in a protective housing state. The lens distortion parameter with a protective housing corresponding to the image pair with the smallest difference is determined as the target lens distortion parameter of the shooting device. Based on the target lens distortion parameter, the type of protective housing worn by the shooting device is determined. In the above method, if the image pair with the smallest difference is obtained based on the lens distortion parameter with a protective housing, then the presence or absence of a protective housing, the type of protective housing, and the target lens distortion parameter can be quickly and accurately determined in a single identification based on the target lens distortion parameter, without the need for manual selection of whether the shooting device is wearing a protective housing and the type of protective housing.

[0118] In one embodiment, if the distortion of the first original image and the second original image is small, directly removing the distortion from the overlapping region images of the first and second lens field of view facilitates the calculation of image differences (e.g., the overlapping region images of the first and second lens field of view satisfy the condition that, at least in a specific direction in the two-dimensional plane, the overlapping region images of the first and second lens field of view are row-aligned). Therefore, distortion removal processing can be directly performed on the overlapping region images of the first and second lens field of view. However, in practice, there are cases where the distortion of the first and second original images is large, making the directly removed overlapping region images of the first and second lens field of view inconvenient for subsequent image difference calculations. Therefore, to solve this problem, such as... Figure 4 As shown, distortion correction is performed on the images of the overlapping regions of the first and second lens field of view using at least two lens distortion parameters to obtain at least two image pairs, including the following steps:

[0119] Step 402: Reproject the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view to obtain the reprojected overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view.

[0120] The reprojection operation is performed to facilitate subsequent image difference calculations in the overlapping region images after distortion correction. In this embodiment, a projection model can be used to reproject the overlapping region images of the first and second lens field of view. The selected projection model should ensure that the reprojected overlapping region images of the first and second lens field of view facilitate subsequent image difference calculations as much as possible. For example, the reprojected overlapping region images of the first and second lens field of view should be row-aligned at least in a specific direction on the two-dimensional plane. That is, if feature points are extracted from the reprojected overlapping region images of the first and second lens field of view, the lines connecting all correctly matched feature points in the reprojected overlapping region images of the first and second lens field of view should be kept as parallel as possible. In practice, cylindrical projection is preferred.

[0121] Optionally, after selecting a suitable projection model, the shooting device or computer device can reproject and correct the distortion of the overlapping area images of the first and second lens fields of view using the selected projection model to obtain the overlapping area images of the first and second lens fields of view with clearer features and easier subsequent image difference calculation.

[0122] Step 404: Use at least two lens distortion parameters to perform distortion correction processing on the reprojected images of the overlapping areas of the first lens field of view and the reprojected images of the overlapping areas of the second lens field of view, to obtain at least two sets of image pairs.

[0123] The distortion correction process is the same as step 206 above, and will not be repeated here.

[0124] Optionally, the imaging device or computer equipment uses the lens distortion parameters without a protective housing to perform distortion correction processing on the reprojected images of the overlapping regions of the first and second field of view angles, resulting in an image pair. Alternatively, the imaging device or computer equipment uses at least one lens distortion parameter with a protective housing to perform distortion correction processing on the reprojected images of the overlapping regions of the first and second field of view angles, respectively, resulting in at least one image pair.

[0125] In this embodiment, when the distortion of the overlapping area images of the first and second lens fields of view is significant, a reprojection operation is performed on both images. Then, at least two lens distortion parameters are used to perform distortion correction processing on the reprojected overlapping area images of the first and second lens fields of view, resulting in a clearer first and second lens distortion-corrected images. This solves the problem that the distortion-corrected overlapping area images of the first and second lens fields of view are inconvenient for subsequent image difference calculations.

[0126] In one embodiment, such as Figure 5 As shown, determining the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image from at least two image pairs includes the following steps:

[0127] Step 502: Calculate the image quality evaluation index of the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0128] Image quality evaluation metrics can characterize the similarity between the first-shot distortion-free image and the second-shot distortion-free image in an image pair. Image quality evaluation metrics can include peak signal-to-noise ratio (PSNR) or mean square error (MSE), and one or more metrics can be used to comprehensively measure the difference between the first-shot distortion-free image and the second-shot distortion-free image. Image quality evaluation metrics can be used as quantitative difference values, or specific quantitative difference values ​​can be obtained by calculating the image quality evaluation metrics.

[0129] Step 504: The image pair with the highest similarity between the first-shot distortion-free image and the second-shot distortion-free image, as represented by the image quality evaluation index, is taken as the image pair with the smallest difference.

[0130] The image pair with the highest similarity was selected as the image pair with the smallest difference because the more similar the image pairs are, the closer the distortion parameters corresponding to the image pair are to the actual values ​​during distortion correction, and the more likely the wearing state corresponding to the distortion parameters of the image pair is to be correct.

[0131] In this embodiment, the image quality evaluation index represents that the higher the similarity between the first lens distortion-free image and the second lens distortion-free image, the smaller the difference between the first lens distortion-free image and the second lens distortion-free image. The image quality evaluation index is used to quantify the difference between the first lens distortion-free image and the second lens distortion-free image of the image pair, thereby improving the image recognition accuracy.

[0132] In one embodiment, such as Figure 6 As shown, determining the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image from at least two image pairs includes the following steps:

[0133] Step 602: Calculate the disparity score between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0134] Among them, the disparity score is a score function formulated based on the disparity law to evaluate the disparity between the first-shot distortion-free image and the second-shot distortion-free image, and is used to characterize the similarity between the first-shot distortion-free image and the second-shot distortion-free image in the image pair.

[0135] Parallax refers to the two-dimensional displacement of corresponding feature points in the first and second image pairs after distortion correction. This two-dimensional displacement includes horizontal and vertical displacement. There are many methods for calculating parallax. For example, the first and second image pairs can be divided into several image blocks, and the parallax of each block can be calculated. The set of parallaxes for each block can then be used as the parallax between the first and second image pairs. Alternatively, the parallax of each pixel in both images can be calculated separately, and the set of parallaxes for each pixel can be used as the parallax between the first and second image pairs.

[0136] In some embodiments, a parallax score is determined between a first-lens distortion-free image and a second-lens distortion-free image in at least two image pairs, based on a predefined scoring function.

[0137] There are many ways to quantify image disparity in the scoring function. For example, disparity can be quantified by averaging the absolute values ​​of the horizontal disparity of all image patches or pixels in an image pair; the smaller the disparity score, the smaller the difference between the image pairs. Another example is to quantify disparity by using the number of image patches or pixels in an image pair whose absolute values ​​of the horizontal disparity are greater than a preset threshold; the smaller the disparity score, the smaller the difference between the image pairs.

[0138] The scoring function can be formulated based on the known differences in parallax patterns between images captured without a protective housing and images captured with at least one type of protective housing. The goal is to maximize the difference in parallax scores between images captured without a protective housing and images captured with a specific protective housing, so as to distinguish them from each other.

[0139] In some embodiments, the scoring function is set based on a first disparity rule and a second disparity rule. The first disparity rule is the disparity relationship under the basic shooting scenario. For example, when most of the captured images are distant scenes at a certain distance, the smaller the average absolute value of the horizontal disparity of all image blocks or pixels in the image pair, the smaller the difference between the first-lens distortion-free image and the second-lens distortion-free image in the image pair. The second disparity rule is a different disparity rule that follows when the captured images are at certain specific distances. Penalty terms and / or constraint terms are introduced according to the second disparity rule to ensure that the disparity score determined by the scoring function truly reflects the difference between the image pairs. For example, under certain calibration methods, the disparity between the two lenses of a multi-view camera must conform to the first disparity rule, and the scoring function is formulated based on the first disparity rule. However, the first parallax rule may only apply to specific object distances. Once these distances exceed a certain range, the parallax rule may change. For example, when the captured image is within a certain distance range, the parallax rule becomes that the larger the average absolute value of the horizontal parallax of all image blocks or pixels in the image pair, the closer the distortion parameters used to obtain the image pair are to the actual distortion parameters. Therefore, it is necessary to consider the complexity of the parallax rule. Based on the first parallax rule, additional penalty or constraint terms should be added for special cases that violate the first parallax rule, so that in special cases, the calculation result of the scoring function can also reflect the differences between the actual image pairs. As for the specific patterns of the first and second parallax rules between the two lenses of a multi-view camera, this is related to the calibration method used by the manufacturer; different calibration methods will result in different first parallax rules.

[0140] In some embodiments, the disparity score is corrected according to a third disparity rule. This third disparity rule characterizes a pattern that cannot occur in either the first lens-distorted image or the second lens-distorted image of the image pair. For example, the third disparity rule states that if at least one lens with a protective housing performs distortion correction on both the first and second field-of-view overlapping regions, the resulting image pair will not contain a disparity greater than or less than a certain specific value. If, after determining that the shooting device is wearing a protective housing based on the scoring function, the image pair contains a disparity greater than or less than a certain specific value during verification, the conclusion that the shooting device is wearing a protective housing can be overturned, and the conclusion can be changed to that the shooting device is not wearing a protective housing.

[0141] Specifically, the shooting device or computer device sets a scoring function based on the first parallax rule. If the shooting device exceeds a specific distance range, a penalty or constraint term is added to the scoring function according to the second parallax rule, so that the calculation result of the scoring function can reflect the difference between the real image pairs in special cases. The parallax score between the first lens distortion-free image and the second lens distortion-free image in at least two sets of image pairs is determined according to the corrected scoring function. After it has been determined that the shooting device is wearing a protective shell according to the scoring function, if it is found during the verification that the obtained image pairs contain parallax greater than or less than a certain specific value, the parallax score is corrected according to the third parallax rule, and the conclusion that the shooting device is wearing a protective shell is changed to the conclusion that the shooting device is not wearing a protective shell.

[0142] Step 604: Obtain the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image represented by the parallax score.

[0143] The disparity score and the difference between the image pairs are not necessarily the same. A smaller disparity score may represent a greater difference, or vice versa, depending on the design of the scoring function. Therefore, this embodiment obtains the image pair with the smallest difference between the first-lens distortion-free image and the second-lens distortion-free image represented by the disparity score.

[0144] In this embodiment, a scoring function is set according to the first disparity rule and the second disparity rule. The disparity score between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs is determined according to the scoring function. The disparity score is corrected according to the third disparity rule. The difference between the first lens distortion-corrected image and the second lens distortion-corrected image of the image pair is quantified using the disparity score, thereby improving the image recognition accuracy.

[0145] In one embodiment, such as Figure 7 As shown, the image pair with the smallest difference between the first-shot distortion-corrected image and the second-shot distortion-corrected image among at least two image pairs includes:

[0146] Step 702: Calculate the image quality evaluation index of the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0147] This step is the same as step 502 above, and will not be repeated here.

[0148] Step 704: The image pair with the highest similarity between the first lens distortion-free image and the second lens distortion-free image, as represented by the image quality evaluation index, is taken as the first recognition result.

[0149] The first identification result is the image pair with the smallest difference determined based on image quality evaluation indicators. This step is the same as step 504 above, and will not be repeated here.

[0150] Step 706: Calculate the disparity score between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0151] This step is the same as step 602 above, and will not be repeated here.

[0152] Step 708: The image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image represented by the disparity score is taken as the second recognition result.

[0153] The second recognition result is the image pair with the smallest difference determined based on the disparity score. This step is the same as step 604 above, and will not be repeated here.

[0154] Step 710: Based on the first recognition result and the second recognition result, determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image among at least two image pairs.

[0155] If the first recognition result and the second recognition result of the image pair are the same, then the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image in the image pair is the first recognition result or the second recognition result.

[0156] If the first recognition result is different from the second recognition result, the image pair whose distortion parameters are corrected by not wearing protective glasses is selected as the recognition result (meaning the image recognition fails, and the default recognition result is output, i.e., not wearing protective glasses).

[0157] Optionally, such as Figure 8As shown, the imaging device or computer device calculates the image quality evaluation index for the first image pair, the second image pair, and the third image pair, respectively, and calculates the disparity score for each of the three image pairs. If the image quality evaluation index represents the smallest difference in the first image pair, and the disparity score represents the smallest difference in the second image pair, then the image pair whose distortion parameters are corrected using the non-protective lens is selected as the recognition result.

[0158] In this embodiment, the image pair with the smallest difference is determined based on the image quality evaluation index and the parallax score. If the image pairs with the smallest difference are not the same, the final image pair with the smallest difference is re-determined based on the nature of the difference. The above method uses a dual quantitative method to determine the image pair with the smallest difference, which improves the accuracy of difference quantification and thus improves the accuracy of determining the lens distortion parameters of the shooting device.

[0159] In another embodiment, such as Figure 9 As shown, a method for determining lens distortion parameters of a shooting device is provided, including the following steps:

[0160] Step 902: Acquire multiple frames of first original images captured by the first lens of the shooting device and multiple frames of second original images captured by the second lens.

[0161] Among them, multiple frames of the first original image captured by the first lens and the second original image captured by the second lens are extracted simultaneously from the video stream captured by the shooting device.

[0162] For example, such as Figure 10 As shown, Figure A and Figure B are the first original image captured by the first lens and the second original image captured by the second lens at time t1, respectively. Figure C and Figure D are the first original image captured by the first lens and the second original image captured by the second lens at time t2, respectively.

[0163] Step 904: Determine the overlapping area image of the field of view in the first original image and the second original image at multiple frames simultaneously, obtain the overlapping area image of the field of view of the first original image at multiple frames simultaneously, and obtain the overlapping area image of the field of view of the first lens, and the overlapping area image of the field of view of the second lens corresponding to the second original image at multiple frames simultaneously.

[0164] The difference between this step and step 204 above is that step 202 processes a single frame image, while this step processes multiple frames. Compared to step 204, this step requires repeating the process of step 202 multiple times. The specific recognition and extraction process is the same as that of step 204, and will not be described again here.

[0165] Figure 10In the above, the image of the overlapping area of ​​the first lens field of view of the first original image at time t1 is denoted as image a, the image of the overlapping area of ​​the second lens field of view of the second original image at time t1 is denoted as image b, and the image of the overlapping area of ​​the first lens field of view of the first original image at time t2 is denoted as image c, and the image of the overlapping area of ​​the second lens field of view of the second original image at time t2 is denoted as image d.

[0166] Step 906: At least two lens distortion parameters are used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view at multiple frames simultaneously, to obtain at least two sets of image pairs.

[0167] The difference between this step and step 204 above is that step 202 processes a single frame image, while this step processes multiple frames. Compared to step 204, this step requires repeating the process of step 202 multiple times. The specific distortion removal process is the same as that in step 204, and will not be described again here.

[0168] For example, such as Figure 10 As shown, this example uses the distortion parameters of one lens without a housing and two lenses with housings, where the distortion parameters of the two lenses with housings are different. First, the distortion parameters of the lens without a housing, x, are used to perform distortion correction on the overlapping area images a, b, c, and d of the first lens's field of view, respectively, to obtain the first lens distortion correction image a1, the second lens distortion correction image b1, the first lens distortion correction image c1, and the second lens distortion correction image d1. The first lens distortion correction image a1 and the second lens distortion correction image b1 are denoted as the first image pair, and the first lens distortion correction image c1 and the second lens distortion correction image d1 are denoted as the fourth image pair. Secondly, using the lens distortion parameter y of the first housing and the same principle, distortion correction processing is performed on the overlapping area images a, b, c, and d of the first lens field of view, respectively, to obtain the second image pair and the fifth image pair. Finally, using the lens distortion parameter z of the second housing and the same principle, distortion correction processing is performed on the overlapping area images a, b, c, and d of the first lens field of view, respectively, to obtain the third image pair and the sixth image pair.

[0169] Step 908: Determine the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among at least two image pairs at the same time.

[0170] This step is the same as step 208, and will not be repeated here.

[0171] Step 910: The lens distortion parameters corresponding to the image pairs with the smallest differences at the same moment are determined as the alternative lens distortion parameters for the shooting device.

[0172] This step is the same as step 210, and will not be repeated here.

[0173] Step 912: Classify the candidate lens distortion parameters and determine the candidate lens distortion parameter with the most classification results as the target lens distortion parameter of the shooting device.

[0174] In particular, by following steps 902-910, candidate lens distortion parameters corresponding to the first and second original images at multiple frames simultaneously can be obtained. The candidate lens distortion parameters are classified, and the candidate lens distortion parameter with the highest number of classification results is taken as the target lens distortion parameter of the shooting device.

[0175] For example, Figure 10 In the process, the shooting device or computer device obtains multiple frames of the first original image and the second original image at the same moment, and classifies the multiple candidate lens distortion parameters. If the candidate lens distortion parameter y is obtained the most times, then the candidate lens distortion parameter y is taken as the target lens distortion parameter of the shooting device.

[0176] In this embodiment, candidate lens distortion parameters corresponding to the first and second original images captured simultaneously across multiple frames are used. The candidate lens distortion parameter with the most classification results is determined as the target lens distortion parameter of the shooting device. By using a classification and statistical method to determine the lens distortion parameters determined from the images captured simultaneously across multiple frames by the same shooting device, the final target lens distortion parameter of the shooting device can be determined, which can improve the accuracy of the target lens distortion parameter and the determination precision of the lens distortion parameter of the shooting device.

[0177] In one embodiment, a method for determining lens distortion parameters of a shooting device is provided, specifically including the following steps:

[0178] Step 1: Acquire multiple frames of first raw images captured by the first lens of the shooting device and multiple frames of second raw images captured by the second lens.

[0179] Step 2: Determine the overlapping regions of the field of view in the first and second original images at multiple frames simultaneously. Extract the overlapping regions of the field of view from the first original image at multiple frames simultaneously to obtain the overlapping regions of the field of view of the first lens. Extract the overlapping regions of the field of view of the second lens from the second original image at multiple frames simultaneously.

[0180] Step 3: Reproject the overlapping area images of the first lens field of view and the second lens field of view respectively to obtain the reprojected overlapping area images of the first lens field of view and the second lens field of view.

[0181] Step 4: Using at least two lens distortion parameters, perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection at multiple frames simultaneously, to obtain at least two sets of image pairs.

[0182] Step 5: Calculate the image quality evaluation index for the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0183] Step 6: The image pair with the highest similarity between the first-lens distortion-free image and the second-lens distortion-free image, as represented by the image quality evaluation index, is taken as the first recognition result.

[0184] Step 7: Calculate the disparity score between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs.

[0185] Step 8: The image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image, as represented by the disparity score, is taken as the second recognition result.

[0186] Step 9: Based on the first recognition result and the second recognition result, determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image from at least two image pairs.

[0187] Step 10: Determine the lens distortion parameters corresponding to the image pairs with the smallest differences at the same time as the alternative lens distortion parameters for the shooting device.

[0188] Step 11: Classify the candidate lens distortion parameters and determine the candidate lens distortion parameter with the most classification results as the target lens distortion parameter of the shooting device.

[0189] Step 12: If the target lens distortion parameters are the same as those of a lens with a protective housing, then the shooting device is determined to be in a state of wearing a protective housing. Based on the housing type corresponding to the lens distortion parameters with a protective housing, the housing type worn by the shooting device is determined.

[0190] In this embodiment, the target lens distortion parameters of the shooting device are determined by classifying and statistically analyzing the lens distortion parameters of multiple frames captured simultaneously by the same shooting device. Based on the target lens distortion parameters, it is possible to quickly and accurately determine whether the shooting device is equipped with a protective shell, and the distortion correction parameters of the images captured by the shooting device with the protective shell. This eliminates the need for manual selection of whether the shooting device is equipped with a protective shell and the need for manual selection of the distortion parameters of the images captured by the shooting device with the protective shell.

[0191] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0192] Based on the same inventive concept, this application also provides a lens distortion parameter determination device for implementing the lens distortion parameter determination method of the shooting device described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the lens distortion parameter determination device provided below can be found in the limitations of the lens distortion parameter determination method of the shooting device described above, and will not be repeated here.

[0193] In one embodiment, such as Figure 11 As shown, a lens distortion parameter determination device for a shooting apparatus is provided, comprising: an acquisition module 100, an image extraction module 200, a distortion correction module 300, a recognition module 400, and a determination module 500, wherein:

[0194] The acquisition module 100 is used to acquire a first original image captured by the first lens of the shooting device and a second original image captured by the second lens.

[0195] The image extraction module 200 is used to determine the overlapping area image of the field of view in the first original image and the second original image, and to obtain the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image.

[0196] The distortion correction module 300 is used to perform distortion correction processing on the image of the overlapping area of ​​the first lens field of view and the image of the overlapping area of ​​the second lens field of view using at least two lens distortion parameters, so as to obtain at least two sets of image pairs.

[0197] The recognition module 400 is used to determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs;

[0198] The determination module 500 is used to determine the lens distortion parameters corresponding to the image pair with the smallest difference as the target lens distortion parameters of the shooting device.

[0199] In one embodiment, at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing; the determining module 500 is further configured to: determine that the shooting device is in a protective housing state if the image pair with the smallest difference is obtained based on the lens distortion parameter with a protective housing.

[0200] The image pair with the smallest difference corresponds to the lens distortion parameter of the lens with a protective housing, which is then determined as the target lens distortion parameter of the shooting device.

[0201] In one embodiment, at least two lens distortion parameters include lens distortion parameters corresponding to at least two different protective housing types; the determining module 500 is further configured to: determine the housing type worn by the shooting device based on the housing type corresponding to the lens distortion parameters of the lens with protective housing.

[0202] In one embodiment, the distortion correction module 300 is further configured to reproject the first lens field-of-view overlapping region image and the second lens field-of-view overlapping region image respectively to obtain the reprojected first lens field-of-view overlapping region image and the second lens field-of-view overlapping region image.

[0203] At least two lens distortion parameters are used to perform distortion correction on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection, resulting in at least two sets of image pairs.

[0204] In one embodiment, the recognition module 400 is further configured to calculate image quality evaluation indices for the first lens distortion-free image and the second lens distortion-free image in at least two image pairs, respectively.

[0205] Image quality evaluation metrics are used to characterize the image pair with the highest similarity between the first-shot distortion-free image and the second-shot distortion-free image as the image pair with the smallest difference.

[0206] In one embodiment, the recognition module 400 is further configured to calculate the disparity score between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs, respectively.

[0207] Obtain the image pair with the smallest difference between the first-lens distortion-free image and the second-lens distortion-free image, as represented by the disparity score.

[0208] In one embodiment, the recognition module 400 is further configured to determine the disparity score between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs according to a predefined scoring function; the scoring function uses the disparity of image blocks or pixels in the image pair as a variable and is a functional expression characterizing the magnitude of the difference between the two images in the image pair.

[0209] In one embodiment, the recognition module 400 is further configured to calculate image quality evaluation indices for the first lens distortion-free image and the second lens distortion-free image in at least two image pairs, respectively.

[0210] The image quality evaluation index characterizes the image pair with the highest similarity between the first-lens distortion-free image and the second-lens distortion-free image as the first recognition result;

[0211] Calculate the disparity score between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs;

[0212] The image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image, represented by the disparity score, is taken as the second recognition result;

[0213] Based on the first and second recognition results, determine the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image from at least two image pairs.

[0214] In one embodiment, the acquisition module 100 is further configured to acquire multiple frames of first original images captured by the first lens of the shooting device and multiple frames of second original images captured by the second lens.

[0215] The image extraction module 200 is also used to determine the overlapping area image of the field of view in the first original image and the second original image at multiple frames at the same time, to obtain the overlapping area image of the field of view of the first lens from the first original image at multiple frames at the same time, and to extract the overlapping area image of the field of view of the second lens from the second original image at multiple frames at the same time.

[0216] The distortion correction module 300 is also used to perform distortion correction processing on the first lens field-of-view overlapping region image and the second lens field-of-view overlapping region image at multiple frames simultaneously using at least two lens distortion parameters, to obtain at least two sets of image pairs.

[0217] The recognition module 400 is also used to determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image among at least two image pairs at the same time.

[0218] The module 500 determines the lens distortion parameters corresponding to the image pairs with the smallest differences at the same moment as the candidate lens distortion parameters of the shooting device; it then classifies the candidate lens distortion parameters and determines the candidate lens distortion parameter with the most classification results as the target lens distortion parameter of the shooting device.

[0219] The various modules in the lens distortion parameter determination device of the aforementioned shooting equipment can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0220] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 12 As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements a method for determining lens distortion parameters of a photographic device. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0221] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0222] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0223] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0224] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0225] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0226] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0227] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0228] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for determining lens distortion parameters of an imaging device, characterized in that, The method includes: Acquire a first raw image captured by the first lens of the shooting device and a second raw image captured by the second lens; Determine the overlapping area images of the field of view in the first original image and the second original image to obtain the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image. At least two lens distortion parameters are used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view, resulting in at least two image pairs; the at least two lens distortion parameters include the lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing, and the at least one lens distortion parameter with a protective housing is different from each other; Identify the image pair in the at least two image pairs where the difference between the first lens distortion-corrected image and the second lens distortion-corrected image is the smallest; The lens distortion parameters corresponding to the image pair with the smallest difference are determined as the target lens distortion parameters of the shooting device; The type of housing worn by the shooting device is determined based on the housing type corresponding to the lens distortion parameters of the lens with a protective housing.

2. The method according to claim 1, characterized in that, The at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing. The step of determining the lens distortion parameters corresponding to the image pair with the smallest difference as the target lens distortion parameters of the shooting device further includes: If the image pair with the smallest difference is obtained based on the lens distortion parameters with a protective housing, then the shooting device is determined to be in a state with a protective housing. The lens distortion parameters corresponding to the image pair with the smallest difference, which are fitted with protective housings, are determined as the target lens distortion parameters of the shooting device.

3. The method according to claim 1, characterized in that, The at least two lens distortion parameters include lens distortion parameters corresponding to at least two different protective housing types.

4. The method according to claim 1, characterized in that, The distortion correction process involves applying at least two lens distortion parameters to both the overlapping regions of the first and second lens field of view images, resulting in at least two image pairs, including: The first lens field of view overlap region image and the second lens field of view overlap region image are reprojected respectively to obtain the reprojected first lens field of view overlap region image and second lens field of view overlap region image. At least two lens distortion parameters are used to perform distortion correction on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection, resulting in at least two sets of image pairs.

5. The method according to claim 1, characterized in that, The step of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among the at least two image pairs includes: Calculate the image quality evaluation index for the first lens distortion-free image and the second lens distortion-free image in the at least two image pairs respectively; The image quality evaluation index is used to characterize the image pair with the highest similarity between the first lens distortion-corrected image and the second lens distortion-corrected image as the image pair with the smallest difference.

6. The method according to claim 1, characterized in that, The step of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among the at least two image pairs includes: Calculate the disparity score between the first lens distortion-free image and the second lens distortion-free image in at least two image pairs respectively; Obtain the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image, as represented by the disparity score.

7. The method according to claim 6, characterized in that, The calculation of disparity scores between the first lens-corrected image and the second lens-corrected image in at least two image pairs includes: Based on a pre-defined scoring function, the disparity score between the first lens distortion-corrected image and the second lens distortion-corrected image in at least two image pairs is determined; the scoring function is a functional expression that takes the disparity of image blocks or pixels in the image pair as a variable and characterizes the magnitude of the difference between the two images in the image pair.

8. The method according to claim 1, characterized in that, The step of determining the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image among the at least two image pairs includes: Calculate the image quality evaluation index for the first shot distortion-free image and the second shot distortion-free image in at least two image pairs respectively; The image quality evaluation index characterizes the image pair with the highest similarity between the first-lens distortion-free image and the second-lens distortion-free image as the first recognition result; Calculate the disparity score between the first-shot distortion-free image and the second-shot distortion-free image in at least two image pairs; The image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image, represented by the disparity score, is taken as the second recognition result; Based on the first and second recognition results, determine the image pair with the smallest difference between the first lens distortion-corrected image and the second lens distortion-corrected image from at least two image pairs.

9. A method for determining lens distortion parameters of a shooting device, characterized in that, The method includes: Acquire multiple frames of first raw images captured by the first lens of the shooting device and multiple frames of second raw images captured by the second lens; Determine the overlapping area of ​​the field of view in the first original image and the second original image at multiple frames simultaneously, and obtain the overlapping area of ​​the field of view of the first lens corresponding to the first original image at multiple frames simultaneously, and the overlapping area of ​​the field of view of the second lens corresponding to the second original image at multiple frames simultaneously. At least two lens distortion parameters are used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view at multiple frames simultaneously, to obtain at least two sets of image pairs; the at least two lens distortion parameters include the lens distortion parameters without protective housing and at least one lens distortion parameter with protective housing, and the at least one lens distortion parameter with protective housing is different from each other. Identify the image pair with the smallest difference between the first-shot distortion-free image and the second-shot distortion-free image from at least two image pairs at the same time. The lens distortion parameters corresponding to the images with the smallest differences at the same moment are determined as the alternative lens distortion parameters for the shooting device; Multiple candidate lens distortion parameters are classified, and the candidate lens distortion parameter with the most classification results is determined as the target lens distortion parameter of the shooting device. The type of housing worn by the shooting device is determined based on the housing type corresponding to the lens distortion parameters of the lens with a protective housing.

10. The method according to claim 9, characterized in that, The at least two lens distortion parameters include lens distortion parameters without a protective housing and at least one lens distortion parameter with a protective housing. The step of determining the lens distortion parameters corresponding to the image pairs with the smallest differences at the same moment as the alternative lens distortion parameters for the shooting device also includes: If the image pair with the smallest difference is obtained based on the lens distortion parameters with a protective housing, then the shooting device is determined to be in a state with a protective housing. The lens distortion parameters corresponding to the image pair with the smallest difference, which are protected by a lens housing, are determined as the alternative lens distortion parameters for the shooting device.

11. The method according to claim 9, characterized in that, The at least two lens distortion parameters include lens distortion parameters corresponding to at least two different protective housing types; The method further includes: determining the type of housing worn by the shooting device based on the housing type corresponding to the lens distortion parameters of the lens with a protective housing.

12. The method according to claim 9, characterized in that, The process involves using at least two lens distortion parameters to perform distortion correction on the overlapping regions of the first and second lens field of view images simultaneously across multiple frames, resulting in at least two image pairs, including: The first and second lens field-of-view overlap region images and the second lens field-of-view overlap region images at multiple frames simultaneously are reprojected to obtain the reprojected first and second lens field-of-view overlap region images. At least two lens distortion parameters are used to perform distortion correction on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view after reprojection, resulting in at least two sets of image pairs.

13. A lens distortion parameter determination device for a shooting apparatus, characterized in that, The device includes: The acquisition module is used to acquire a first raw image captured by the first lens and a second raw image captured by the second lens of the shooting device; The image extraction module is used to determine the overlapping area images of the field of view in the first original image and the second original image, and to obtain the overlapping area image of the first lens field of view corresponding to the first original image and the overlapping area image of the second lens field of view corresponding to the second original image. The distortion correction module is used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view using at least two lens distortion parameters to obtain at least two sets of image pairs; the at least two lens distortion parameters include lens distortion parameters without protective housing and at least one lens distortion parameter with protective housing, and the at least one lens distortion parameter with protective housing is different from each other; The identification module is used to determine the image pair in the at least two image pairs where the difference between the first lens distortion-corrected image and the second lens distortion-corrected image is the smallest; The determination module is used to determine the lens distortion parameters corresponding to the image pair with the smallest difference as the target lens distortion parameters of the shooting device; and to determine the type of housing worn by the shooting device according to the housing type corresponding to the lens distortion parameters with protective housing.

14. The apparatus according to claim 13, characterized in that, The acquisition module is also used to acquire multiple frames of first original images captured by the first lens of the shooting device and multiple frames of second original images captured by the second lens; The image extraction module is also used to determine the overlapping area image of the field of view in the first original image and the second original image at multiple frames at the same time, so as to obtain the overlapping area image of the first lens field of view corresponding to the first original image at multiple frames at the same time, and the overlapping area image of the second lens field of view corresponding to the second original image at multiple frames at the same time. The distortion correction module is also used to perform distortion correction processing on the overlapping area images of the first lens field of view and the overlapping area images of the second lens field of view at multiple frames simultaneously using at least two lens distortion parameters, so as to obtain at least two sets of image pairs. The recognition module is also used to determine the image pair with the smallest difference between the first lens distortion-free image and the second lens distortion-free image among at least two image pairs at the same moment; The determining module is further configured to determine the lens distortion parameters corresponding to the image pairs with the smallest differences at the same moment as the candidate lens distortion parameters of the shooting device; classify the candidate lens distortion parameters, and determine the candidate lens distortion parameter with the most classification results as the target lens distortion parameter of the shooting device.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

Images