External parameter calibration method, device and computer program product for a photographing device

Abstract

An embodiment of the present application provides an external parameter calibration method, device, and computer program product for a photographing device. The method may include: determining an overlapping area between a first input image and a second input image, where the first input image is a reference image and the second input image is an image at a preset viewing angle of the target photographing device; determining target feature points in the overlapping area; obtaining at least two scales of calibration ranges for each external parameter of the target photographing device; and, in the order from the largest scale to the smallest scale, for each scale, calibrating each external parameter of the target photographing device according to the target feature points and the calibration ranges of the external parameters at the scale, where the calibration result of each external parameter value at the previous scale is the calibration reference for each external parameter value at the next scale, and the calibration range of the next scale is a part of the calibration range of the previous scale. In this way, the inconvenience caused to the user by returning the target photographing device to the factory for re-calibration can be avoided.

Description

Technical Field

[0001] This document relates to the field of camera extrinsic parameter calibration, and in particular, to an extrinsic parameter calibration method, device, and computer program product for a photographing device. Background Art

[0002] In the scenario of using multiple cameras, such as in a vehicle surround-view imaging system during long-term driving of a vehicle, there is a problem that the cumulative error of the camera extrinsic parameters is caused by external forces and the external environment.

[0003] A common method for re-calibrating (error correction) the extrinsic parameters of multiple cameras is to return the hardware device to the manufacturer, and the manufacturer uses a calibration board for re-calibration. Such methods have great limitations and are very inconvenient for users, seriously affecting the user experience. Summary of the Invention

[0004] Embodiments of the present application provide an extrinsic parameter calibration method, device, and computer program product for a photographing device, which are used to avoid the inconvenience brought to users by returning the hardware device to the manufacturer for calibration.

[0005] To solve the above technical problems, the embodiments of the present application are implemented as follows:

[0006] In a first aspect, there is provided an extrinsic parameter calibration method for a photographing device, the method including:

[0007] Determine an overlapping region between a first input image and a second input image, where the first input image is a reference image, and the second input image is an image at a preset viewing angle of a target photographing device;

[0008] Determine target feature points in the overlapping region;

[0009] Obtain at least two scales of calibration ranges for each extrinsic parameter setting of the target photographing device;

[0010] In the order from the largest to the smallest of the scales, for each scale, correct the extrinsic parameter values of the target photographing device according to the target feature points and the calibration ranges of the extrinsic parameters at the scale, where the calibration result of the extrinsic parameter values at the previous scale is the calibration reference for the extrinsic parameter values at the next scale, and the calibration range at the next scale is a part of the calibration range at the previous scale.

[0011] In a second aspect, there is provided an extrinsic parameter calibration method for a photographing device, the method including:

[0012] Determine an overlapping region between a first input image and a second input image, where the first input image is a reference image, and the second input image is an image at a preset viewing angle of a target photographing device;

[0013] Determine the target feature points in the overlapping region;

[0014] Let i = 1, 2, ……, k in sequence, and repeat the external parameter calibration step a after each assignment of i until a preset termination condition is satisfied;

[0015] Wherein, the external parameter calibration step a includes: obtaining the i-th calibration range set for each of the multiple external parameters of the target imaging device, respectively selecting an external parameter compensation value from the i-th calibration ranges corresponding to the multiple external parameters, obtaining multiple i-th calibrated external parameter values corresponding to the multiple external parameters, and in response to the pixel features of the target feature points in the preset viewing angle of the target imaging device calibrated according to the multiple i-th calibrated external parameter values satisfying a preset calibration condition, updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values correspondingly, wherein, the i-th calibrated external parameter value of each external parameter is equal to the sum of the value of this external parameter before updating and the i-th compensation value of this external parameter;

[0016] Wherein, k is an integer greater than or equal to 2, and for i = 2, ……, k, the first external parameter update for the i-th calibration range is based on the result of the last external parameter update corresponding to the i - 1-th calibration range;

[0017] Wherein, for i = 2, ……, k, the i-th calibration range is a part of the i - 1-th calibration range.

[0018] In a third aspect, there is provided an external parameter calibration device for an imaging device, the device includes:

[0019] An overlapping region determination module, configured to determine the overlapping region between a first input image and a second input image, wherein, the first input image is a reference image, and the second input image is an image in the preset viewing angle of the target imaging device;

[0020] A feature point determination module, configured to determine the target feature points in the overlapping region;

[0021] A calibration range acquisition module, configured to acquire at least two scales of calibration ranges set for each external parameter of the target imaging device;

[0022] An external parameter calibration module, configured to calibrate the values of the external parameters of the target imaging device according to the target feature points and the calibration ranges of the external parameters at each scale in the order from large to small of the scales, wherein, the calibration result of the external parameter values at the previous scale is the calibration reference for the external parameter values at the next scale, and the calibration range at the next scale is a part of the calibration range at the previous scale.

[0023] Fourth aspect, there is provided an external parameter calibration device for a photographing device, the device comprising:

[0024] An overlapping area determination module, configured to determine an overlapping area between a first input image and a second input image, wherein the first input image is a reference image, and the second input image is an image at a preset viewing angle of a target photographing device;

[0025] A feature point determination module, configured to determine target feature points in the overlapping area;

[0026] An external parameter correction module, configured to sequentially set i = 1, 2,..., k, and repeat the external parameter correction step a after each assignment of i until a preset termination condition is satisfied;

[0027] Wherein, the external parameter correction step a includes: obtaining a first correction range set for each of a plurality of external parameters of the target photographing device, respectively selecting an external parameter compensation value from the first correction ranges corresponding to the plurality of external parameters, obtaining a plurality of first correction external parameter values corresponding to the plurality of external parameters respectively, in response to the pixel features of the target feature points at the preset viewing angle of the target photographing device corrected according to the plurality of first correction external parameter values satisfying a preset correction condition, updating the values of the plurality of external parameters of the target photographing device to the plurality of first correction external parameter values correspondingly, wherein the first correction external parameter value of each external parameter is equal to the sum of the value of the external parameter before update and the first compensation value of the external parameter;

[0028] Wherein, k is an integer greater than or equal to 2, and for i = 2,..., k, the first external parameter update for the i-th correction range is based on the result of the last external parameter update corresponding to the (i - 1)-th correction range;

[0029] Wherein, for i = 2,..., k, the i-th correction range is a part of the (i - 1)-th correction range.

[0030] Fifth aspect, there is provided an electronic device, comprising:

[0031] A processor;

[0032] A memory for storing instructions executable by the processor;

[0033] Wherein, the processor is configured to execute the instructions to implement the method described in the first aspect or the second aspect.

[0034] Sixth aspect, there is provided a computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the method described in the first aspect or the second aspect.

[0035] In a seventh aspect, there is provided a computer program product including instructions. When a computer runs the instructions of the computer program product, the computer executes the method according to the first aspect or the second aspect.

[0036] In the embodiments of the present application, after determining the overlapping area between the reference image and the image under the preset viewing angle of the target shooting device, the target feature points in the overlapping area can be automatically recognized. Then, according to the target feature points and the calibration ranges of different scales corresponding to each external parameter, multi-scale calibration of each external parameter of the target shooting device is performed, without returning the target shooting device to the manufacturer for re-calibration. Therefore, the calibration process is more convenient for users and can improve the user's device usage experience. Description of the Drawings

[0037] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0038] Figure 1 is a schematic flowchart of a method for calibrating external parameters of a shooting device provided by an embodiment of the present application.

[0039] Figure 2 is a schematic diagram of the field of view relationship of a vehicle-mounted surround-view camera system provided by an embodiment of the present application.

[0040] Figure 3 is a schematic diagram of a pixel window in a method for extracting angular feature points provided by an embodiment of the present application.

[0041] Figure 4 is a schematic structural diagram of a process for a method for calibrating external parameters of a shooting device provided by an embodiment of the present application.

[0042] Figure 5 is a schematic diagram of the process for extracting target feature points provided by an embodiment of the present application.

[0043] Figure 6 is a schematic flowchart of a method for calibrating external parameters of a shooting device provided by another embodiment of the present application.

[0044] Figure 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

[0045] Figure 8 is a schematic structural diagram of a device for calibrating external parameters of a shooting device provided by an embodiment of the present application.

[0046] Figure 9 This is a schematic structural diagram of an external parameter calibration device for a photographing device provided by another embodiment of the present application. Detailed implementation manners

[0047] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in one or more embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0048] The terms "first", "second", etc. in the present application and the claims are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that such used data may be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein. In addition, "and / or" in the present application and the claims means at least one of the connected objects, and the character " / " generally means that the related objects before and after are in an "or" relationship.

[0049] Although the related art has also proposed some camera external parameter line calibration methods that do not require returning to the factory, these methods have their own deficiencies and need to be improved. For example, the multi-external parameter calibration method using road lane lines relies heavily on lane lines and has great limitations; the multi-camera external parameter line calibration method based on image registration uses a large number of features in the overlapping area for registration and correction, and the algorithm has a large amount of calculation and time consumption.

[0050] In order to avoid the inconvenience brought to users by returning the target photographing device to the manufacturer for calibration, and in order to overcome at least one deficiency existing in the camera external parameter line calibration methods that do not require returning to the factory in the related art, the present application proposes an external parameter calibration method, device and computer program product for a photographing device. The method can be executed by an electronic device or software installed in the electronic device. Among them, the electronic device includes but is not limited to any one of intelligent devices such as smart phones, personal computers (PCs), laptop computers, tablet computers, e-readers, Internet TVs, wearable devices, etc.

[0051] An external parameter calibration method for a shooting device proposed in an embodiment of the present application can be applied to, but is not limited to, the following multi-camera scenarios: vehicle-mounted surround-view imaging systems, intelligent driving assistance systems, etc. In the case where the external parameters of the multi-camera system receive error images, the external parameter error correction of the multi-camera system can be performed online without returning to the after-sales workshop. As an example, an external parameter calibration method for a shooting device proposed in an embodiment of the present application can be used as an algorithm application or software program in a vehicle-mounted surround-view software solution and run on hardware such as a vehicle-mounted digital signal processor (DSP), a graphics processing unit (GPU), and a central processing unit (CPU).

[0052] First, an external parameter calibration method for a shooting device provided in an embodiment of the present application will be described with reference to the accompanying drawings.

[0053] As shown in Figure 1 the drawings, an external parameter calibration method for a shooting device provided in an embodiment of the present application may include:

[0054] Step 101: Determine the overlapping area between a first input image and a second input image, where the first input image is a reference image and the second input image is an image at a preset viewing angle of a target shooting device.

[0055] The target shooting device is the device to be calibrated.

[0056] In some embodiments, the preset viewing angle may be a birds-eye view (BEV). The first input image may be a birds-eye view converted from a first image captured by a reference shooting device, and the second input image may be a birds-eye view converted from a second image captured by the target shooting device. The shooting ranges of the reference shooting device and the target shooting device overlap, and the first image and the second image are captured by the reference shooting device and the target shooting device simultaneously from different viewing angles for the same scene.

[0057] Taking a vehicle-mounted surround-view camera system as an example, the reference shooting device may be one of the cameras, and the target shooting device may be an adjacent camera. Figure 2 shows a schematic diagram of the shooting range (field of view) of a vehicle-mounted surround-view camera system including four cameras in the front, rear, left, and right. As shown in Figure 2As shown, there is an overlap in the imaging fields of view between every two of the four cameras. First, taking the front camera as the reference - the reference imaging device, the extrinsic parameters of the right camera or the left camera (the target imaging device) can be calibrated; then, taking the right camera or the left camera as the reference - the reference imaging device, the online extrinsic parameter calibration of the rear camera (the target imaging device) can be performed.

[0058] Optionally, when the reference imaging device and the target imaging device are fisheye cameras, before step 101, the method may further include:

[0059] Undistorting the first image and the second image;

[0060] Converting the undistorted first image into the first input image;

[0061] Converting the undistorted second image into the second input image.

[0062] Further, when the preset viewing angle is a bird's-eye view angle, converting the undistorted first image into an image in the bird's-eye view angle to obtain the first input image; converting the undistorted second image into an image in the bird's-eye view angle to obtain the second input image.

[0063] In some embodiments, the undistorting of the first image and the second image may include: undistorting the first image by using the distortion parameters (usually provided by the camera manufacturer) and the intrinsic parameters of the reference imaging device, and undistorting the second image by using the distortion parameters (usually provided by the camera manufacturer) and the intrinsic parameters of the target imaging device.

[0064] In some embodiments, the converting of the undistorted first image into an image in the bird's-eye view angle to obtain the first input image may include: obtaining a bird's-eye view image with errors - the first input image by using the extrinsic parameters of the reference imaging device and the bird's-eye view initialization parameters.

[0065] In some embodiments, the converting of the undistorted second image into an image in the bird's-eye view angle to obtain the second input image may include: obtaining a bird's-eye view image with errors - the second input image by using the initial extrinsic parameters of the target imaging device and the bird's-eye view initialization parameters.

[0066] Specifically, the process of converting the undistorted image into an image in the bird's-eye view angle includes: based on the undistorted image, the camera intrinsic parameters , the camera extrinsic parameters and the bird's-eye view transformation matrix , transforming all the pixels of the undistorted image into pixels in the bird's-eye view angle to obtain a bird's-eye view image.

[0067] Among them, for any pixel I of the undistorted fisheye image , the pixel coordinates B of this pixel in the bird's-eye view can be calculated , and the calculation formula is as follows:

[0068]

[0069] Among them, z can take a constant, for example is the internal parameter of the fisheye camera, is the external parameter of the fisheye camera, is the transformation parameter for converting the bird's-eye view in the world coordinate system to the camera shooting view.

[0070] Optionally, for the first input image and the second input image converted to the bird's-eye view, the required image area is cropped, and then the overlapping area between the two is calculated. Among them, the pixels in the overlapping area of the two cropped images are not zero, so based on this principle, the area where the pixels in the first input image and the second input image in the bird's-eye view are not zero can be calculated to obtain the overlapping area. Optionally, preprocessing such as binarization and morphological operations is performed on this overlapping area, and noise points with pixel values that may be zero are removed to obtain a mask (Mask) of this overlapping area; then an erosion operation is performed on the mask image, and the edge area is removed to obtain an image of the overlapping area that is clear and noise-free.

[0071] Step 102, determine the target feature points in the overlapping area.

[0072] Optionally, before step 102, the method may further include: preprocessing the overlapping area of the first input image, such as filtering processing, to further eliminate the noise in the overlapping area. Specifically, a 3×3 Gaussian filter kernel is used to perform Gaussian filtering on the overlapping area of the first input image to reduce the noise in the overlapping area.

[0073] Among them, the target feature points are sparse feature points in the overlapping area rather than all pixel points. For example, the target feature points can be at least one of the edge feature points and the corner feature points in the overlapping area.

[0074] In some embodiments, the target feature points include edge feature points and corner feature points, and step 102 may include: extracting edge feature points from the overlapping area of the first input image based on the Sobel operator; and extracting corner feature points from the overlapping area of the first input image based on the Harris corner feature point detection algorithm.

[0075] Among them, the edge feature extraction based on the Sobel operator includes: gradient calculation in the horizontal direction (X direction) and gradient calculation in the vertical direction (Y direction). The first-order Sobel operator in the X direction and the first-order operator in the Y direction are respectively:

[0076]

[0077]

[0078] Among them, the Harris corner feature point detection algorithm determines whether a pixel is a corner feature point by calculating the content difference within the pixel neighborhood area, usually referring to a pixel window with a large content difference from its surroundings. The difference value obtained by moving the pixel window in any direction on the image is called the window response value E, and its calculation formula is:

[0079]

[0080] As Figure 3 shown, the pixel window 31 is the window before movement and the pixel window 32 is the window after movement . represents the window movement direction, represents the weights of different pixels, usually using a Gaussian weighting function. The above formula calculates the square of the difference in gray values of each pixel before and after movement, and sums the weights of each pixel point as the content difference between the two windows . Based on the set threshold, the E calculated for a certain pixel position is judged. If it exceeds the set threshold, the current pixel is the position of the corner feature point, otherwise the current pixel is not the position of the corner feature point. The position of the corner feature point should have the following characteristics: the difference value E calculated by moving the current window in any direction is relatively large.

[0081] After calculating the coordinates of the edge feature points and the coordinates of the corner feature points, removing the duplicate coordinates can obtain the coordinate set R of the sparse target feature points.

[0082] It can be understood that: 1) Compared with the prior art solution for calibrating the external parameters of the surround-view camera using lane line features, the embodiments of the present application can use any natural scene for feature extraction in the overlapping view area of the camera, not limited to lane lines, so the limitations of external parameter calibration can be further reduced; 2) Instead of using all pixels in the overlapping view for external parameter calibration, sparse edge feature points and corner feature points with large gradients are extracted for external parameter calibration, so the overall computational amount in the entire calibration process can be reduced, the consumption of computing resources can be reduced, and the calibration efficiency can be improved.

[0083] In some embodiments, an external parameter calibration method for a photographing device proposed by the present application may further include: determining the brightness modulation coefficient according to the brightness information of the overlapping region in the first input image and the brightness information of the overlapping region in the second input image. This brightness modulation coefficient can be used to eliminate the brightness difference between the two images subsequently, reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the brightness difference, and thus avoid affecting the calibration accuracy.

[0084] In some embodiments, the determining the brightness modulation coefficient according to the brightness information of the overlapping region in the first input image and the brightness information of the overlapping region in the second input image may include: determining a first statistical result of the brightness of each pixel in the overlapping region of the first input image; determining a second statistical result of the brightness of each pixel in the overlapping region of the second input image; and determining the brightness modulation coefficient according to the first statistical result and the second statistical result.

[0085] Wherein, the first statistical result and the second statistical result are of the same type of statistical results. For example, if the first statistical result is the median, then the second statistical result is also the median; if the first statistical result is the mean value, then the second statistical result is also the mean value.

[0086] In an example, the first statistical result is the mean value of the brightness of each pixel in the overlapping region of the first input image, and the second statistical result is the mean value of the brightness of each pixel in the overlapping region of the second input image. Specifically, for each pixel in the overlapping region of the masked first input image or the second input image, the brightness modulation coefficient Luma_ration is calculated by the following formula:

[0087] Luma_ration = Mean(image_f × Mask) / Mean(image_r × Mask)

[0088] Wherein, image_f is the first input image, image_r is the second input image, Mask is the overlapping region mask, and Mean() is the operation of calculating the mean value of all pixels in the image of this region.

[0089] Step 103, obtain at least two scales of calibration ranges for each external parameter setting of the target photographing device.

[0090] Wherein, at the same scale, the calibration ranges of the same type of external parameters may be the same or different.

[0091] Step 104: In the order from the largest to the smallest of the said scales, for each said scale, correct the external parameter values of the target imaging device according to the target feature points and the correction ranges of the external parameters at the said scale, wherein the correction result of the external parameter values at the previous scale is the correction reference for the external parameter values at the next scale, and the correction range at the next scale is a part of the correction range at the previous scale.

[0092] For example, for a camera, its external parameters include three translation parameter vectors and three rotation parameter vectors. Among them, the three translation parameter vectors belong to the same category, and the three rotation parameter vectors belong to the same category. These six parameters can be expressed as , where represents a vector containing three translation parameters, represents a vector containing three rotation parameters. In an example, the "external parameters" in Step 103 refer to these six external parameters.

[0093] To improve the correction effect, the embodiments of the present application separately set correction ranges for at least two scales for each external parameter, and adopt multiple iterative corrections from coarse to fine (scales from large to small), which can make the external parameter corrections with coarse granularity in the early stage ensure the robustness to large external parameter errors, and the external parameter corrections with fine granularity in the later stage ensure the accuracy of the correction results.

[0094] As an example, assume that correction ranges for two scales are separately set for the above six external parameters. First, perform the first external parameter correction at the first scale with a larger scale (one external parameter correction includes multiple rounds of iteration), and then perform the second external parameter correction at the second scale with a smaller scale. Among them, at the first scale, the correction ranges of the three rotation parameters are all (-3, +3) degrees, and the correction ranges of the three translation parameters are all (-0.1, 0.1) meters; at the second scale, the correction ranges of the three rotation parameters are all (-0.5, +0.5) degrees, and the correction ranges of the three translation parameters are all (-0.02, 0.02) meters. It can be seen that for the same external parameter, the correction range at the second scale is a part of the correction range at the first scale. Then, for one external parameter correction process at the first scale, the compensation value of any translation external parameter , and the compensation value of any rotation external parameter (-3, 3). After the iterative search is completed, update the external parameter values of the target imaging device with the final result (the optimal corrected values of the external parameters), and then perform the iterative search for the external parameters at the second scale, that is, the correction result of the external parameter values at the previous scale is the correction reference for the external parameter values at the next scale; for one external parameter correction process at the second scale, the compensation value of any translation external parameter , and the compensation value of any rotation external parameter (-0.5, +0.5). After the iterative search is completed, the external parameter values of the target imaging device are updated using the final result (the optimal corrected values of each external parameter). Through the calibration strategy with a gradually refined scale, the robustness of the algorithm to large errors can be enhanced while ensuring the calibration accuracy.

[0095] In some embodiments, for each of the scales, calibrating the external parameter values of the target imaging device according to the target feature points and the calibration ranges of the external parameters at the scale includes: for each of the scales, repeatedly executing a first specified step until a preset termination condition is met.

[0096] The preset termination condition may include but is not limited to at least one of the following: reaching the upper limit of the preset number of iterations, reaching the upper limit of the preset iteration time, and exhaustion of the compensation values in the multiple compensation value lists.

[0097] The first specified step includes:

[0098] Selecting an external parameter compensation value from each of the calibration ranges of the external parameters at the scale to obtain a set of external parameter compensation values;

[0099] Determining the calibrated external parameter values of the target imaging device according to the uncalibrated external parameter values of the target imaging device and the set of external parameter compensation values;

[0100] In response to the pixel features of the target feature points at the preset viewing angle of the calibrated target imaging device satisfying a preset calibration condition, updating the uncalibrated external parameter values of the target imaging device to the calibrated external parameter values.

[0101] Selecting an external parameter compensation value from each of the calibration ranges of the external parameters at the scale includes: randomly extracting an external parameter compensation value from each of the multiple compensation value lists of the external parameters at the scale to obtain a set of external parameter compensation values. At one scale, one external parameter corresponds to a compensation value list determined according to the calibration range of the external parameter. Alternatively, according to the number of iterations at the scale, an external parameter compensation value can be sequentially extracted from the compensation value lists of the external parameters at the scale to obtain a set of external parameter compensation values.

[0102] In some embodiments, before selecting an external parameter compensation value from each of the calibration ranges of the external parameters at the scale, the method further includes: respectively setting external parameter compensation values according to a preset interval under the calibration range corresponding to each external parameter at the scale to obtain the multiple compensation value lists.

[0103] For example, as previously assumed, calibration ranges of two scales are set for each external parameter. At the first scale, the calibration ranges of the three rotation parameters are all (-3, +3) degrees, and the calibration ranges of the three translation parameters are all (-0.1, 0.1) meters; at the second scale, the calibration ranges of the three rotation parameters are all (-0.5, +0.5) degrees, and the calibration ranges of the three translation parameters are all (-0.02, 0.02) meters. Then, for the first scale, an equally divided preset interval can be set. If the preset interval is 0.1, a list of compensation values corresponding to the rotation parameters can be calculated as , and the calculation of the list of compensation values corresponding to the translation parameters is the same. Among the six lists of compensation values composed of 3 rotation compensation values and 3 translation compensation values, one compensation value is randomly selected from each list each time to form a set of external parameter compensation values. Taking the uncalibrated external parameter value as , and a set of external parameter compensation values obtained by randomly selecting one compensation value from the six lists of compensation values as as an example, the calibrated external parameter value can be expressed as .

[0104] In some embodiments, the updating the uncalibrated external parameter value of the target imaging device to the calibrated external parameter value in response to the pixel feature of the target feature point satisfying a preset calibration condition in the preset viewing angle of the calibrated target imaging device may include:

[0105] Obtaining a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device before updating the external parameter value;

[0106] Based on the calibrated external parameter value, determining a second pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device;

[0107] In response to the second pixel feature distance being less than the first pixel feature distance, updating the uncalibrated external parameter value of the target imaging device to the calibrated external parameter value.

[0108] Optionally, Figure 1 the method further includes: in response to the second pixel feature distance being greater than or equal to the first pixel feature distance, ignoring and / or recording the calibrated external parameter value, and re - executing the first specified step, that is, continuing the next round of iteration.

[0109] In some embodiments, when the first specified step is first executed for the maximum scale, the obtaining the first pixel feature distance may include:

[0110] Determining the pixel value of the target feature point in the first input image as the first pixel value;

[0111] Determine the pixel value of the target feature point in the second input image as the second pixel value;

[0112] Determine the first pixel feature distance according to the first pixel value and the second pixel value.

[0113] It should be noted that for the first correction at the maximum scale, the first pixel feature distance can be calculated directly based on the pixel values of the target feature points in the first input image and the second input image, and it may not be calculated based on the uncorrected external parameter values. Of course, it can also be calculated based on the uncorrected external parameter values; in the subsequent correction processes, if the first pixel feature distance was updated to the second pixel feature distance in the previous correction process, then the first pixel feature distance is the second pixel feature distance calculated in the previous time, and there is no need to recalculate.

[0114] In some embodiments, determining the second pixel feature distance of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device based on the corrected external parameter value may include:

[0115] Determine the third pixel value of the target feature point at the preset viewing angle of the corrected target imaging device according to the corrected external parameter value;

[0116] Determine the second pixel feature distance of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

[0117] As an example, the absolute value of the difference between the pixel values of two pixels can be used as the feature distance between the two pixels. That is to say, the first pixel feature distance can be the absolute value of the difference between the first pixel value and the second pixel value, and the second pixel feature distance can be the absolute value of the difference between the first pixel value and the third pixel value. It can be understood that the pixel feature distance can represent the error between two pixels. That is, the first pixel feature distance can represent the pixel value error of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device before correction, and the second pixel feature distance can represent the pixel value error of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device after correction. If the error decreases (i.e., the loss decreases) before and after correction, it indicates that the correction is effective.

[0118] In some other embodiments, determining the first pixel feature distance according to the first pixel value and the second pixel value includes: determining the first pixel feature distance according to the first pixel value, the brightness modulation coefficient, and the second pixel value, where the brightness modulation coefficient is determined according to the brightness difference of the overlapping region in the first input image and the second input image. Correspondingly, determining the second pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value includes: determining the second pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device according to the first pixel value, the brightness modulation coefficient, and the third pixel value.

[0119] It can be understood that introducing the brightness modulation coefficient when determining the first pixel feature distance and the second pixel feature distance can reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the brightness difference, and thus avoid affecting the calibration accuracy.

[0120] In some embodiments, the first specifying step may further include: in response to the second pixel feature distance being less than the first pixel feature distance, updating the first pixel feature distance to the second pixel feature distance.

[0121] Continuing with the above example, the iteration logic of each external parameter value is that first, based on the uncalibrated external parameter value , calculate the first pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device. The calculation formula can be:

[0122]

[0123] where represents the coordinates of the target feature point in the first input image, represents the coordinates of the target feature point under the preset viewing angle of the target imaging device, represents the pixel value of the target feature point in the first input image - the first pixel value, represents the pixel value of the target feature point under the preset viewing angle of the target imaging device - the second pixel value, is the normalized weight coefficient, is the brightness modulation coefficient calculated previously.

[0124] Similarly, after obtaining each externally calibrated value , map the coordinates of the target feature point under the preset viewing angle of the calibrated target imaging device to obtain the coordinates , and then the second pixel feature distance is calculated using the following formula:

[0125]

[0126] After calculating the first pixel feature distance and the second pixel feature distance , the logic for updating each extrinsic parameter correction value is that if , then update each extrinsic parameter value of the target imaging device to the corrected extrinsic parameter value , and then update , and let ; if , ignore and / or record , and continue with the next iteration.

[0127] During the set number of calibration iterations, the extrinsic parameter calibration and feature distance comparison are repeated. Finally, a set of updated extrinsic parameter values is calculated, and the combination of extrinsic parameter values with the minimum feature distance is the optimal result of the current round of parameter correction.

[0128] An extrinsic parameter calibration method for an imaging device proposed in an embodiment of the present application can, after determining the overlapping area between the reference image and the image under the preset viewing angle of the target imaging device, automatically identify the target feature points in the overlapping area, and then perform multi-scale calibration on each extrinsic parameter of the target imaging device according to the target feature points and the calibration ranges of different scales corresponding to each extrinsic parameter, without having to return the target imaging device for re-calibration. Therefore, the calibration process is more convenient for the user and can improve the user's device usage experience.

[0129] In addition, an extrinsic parameter calibration method for an imaging device proposed in an embodiment of the present application: 1) Compared with the extrinsic parameter calibration scheme for a surround-view camera using lane line features in the related art, the embodiment of the present application can use any natural scene for feature extraction in the overlapping viewing angle area of the camera, not limited to lane lines, so it can further reduce the limitations of extrinsic parameter calibration; 2) Instead of using all pixels in the overlapping viewing angle for extrinsic parameter calibration, sparse edge feature points and corner feature points with larger gradients are extracted for extrinsic parameter calibration, so it can reduce the overall calculation amount in the entire calibration process, reduce the consumption of computing resources, and improve the calibration efficiency; 3) When determining the first pixel feature distance and the second pixel feature distance, a brightness modulation coefficient is introduced, which can reduce the error introduced by different camera exposures to prevent the distortion of the pixel feature distance caused by brightness differences, thereby avoiding affecting the calibration accuracy.

[0130] Figure 4 shows the process framework of an extrinsic parameter calibration method for an imaging device proposed in an embodiment of the present application. The following is combined with Figure 4The following further describes an external parameter calibration method for a photographing device proposed in this application.

[0131] Taking a fisheye camera, which is common in a surround view system, as an example of a multi-camera, as Figure 4 shown, the method may include: 1) multi-camera image input, such as inputting a first image captured by a reference camera and a second image captured by a target camera; 2) importing distortion parameters, internal parameters, and initial external parameters of the multi-camera; 3) undistorting the first image and the second image, and converting the undistorted first image and the image into a bird's-eye view image to obtain a first input image and a second input image; 4) extracting target feature points, as Figure 5 shown, this part may specifically include: determining an overlapping area, image exposure coordination (such as determining a brightness modulation coefficient), preprocessing of the overlapping area, calculation of edge feature points and corner feature points, and feature table screening (such as removing duplicate coordinates); 5) multi-scale external parameter search and correction, and loss (such as pixel features of the target feature points under a preset viewing angle of the corrected target photographing device) and iterative update of the external parameters; 6) result output. The detailed implementation processes of parts 1) to 6) described here may refer to the corresponding descriptions above and will not be elaborated here.

[0132] The following Figure 6 shows an embodiment to further describe an external parameter calibration method for a photographing device proposed in this application from another perspective.

[0133] As Figure 6 shown, an external parameter calibration method for a photographing device proposed in another embodiment of this application may include:

[0134] Step 601, determining an overlapping area between a first input image and a second input image, where the first input image is a reference image and the second input image is an image under a preset viewing angle of a target photographing device;

[0135] Step 602, determining target feature points in the overlapping area.

[0136] Among them, the specific implementation processes of step 601 and step 602 correspond to and are consistent with the specific implementation processes of step 101 and step 102 in the Figure 1 shown embodiment, and the same technical effects can be achieved, so the description will not be repeated here.

[0137] Step 603, sequentially setting i = 1, 2,..., k, and repeating external parameter correction step a each time after assigning a value to i until a preset termination condition is met.

[0138] Among them, the preset termination condition may include but is not limited to at least one of the following: reaching a preset upper limit of the number of iterations, reaching a preset upper limit of the iteration time, and exhaustion of the compensation values in the multiple compensation value lists, etc.

[0139] Among them, the external parameter calibration step a may include: obtaining a first calibration range set for each of a plurality of external parameters of the target imaging device, respectively selecting an external parameter compensation value from the first calibration ranges corresponding to the plurality of external parameters, obtaining a plurality of first calibration external parameter values corresponding to the plurality of external parameters, and in response to the pixel features of the target feature points satisfying a preset calibration condition under the preset viewing angle of the target imaging device calibrated according to the plurality of first calibration external parameter values, correspondingly updating the values of the plurality of external parameters of the target imaging device to the plurality of first calibration external parameter values, where the first calibration external parameter value of each external parameter is equal to the sum of the value of the external parameter before update and the first compensation value of the external parameter.

[0140] Among them, k is an integer greater than or equal to 2, and for i = 2,..., k, the first external parameter update for the first calibration range is based on the result of the last external parameter update corresponding to the (i - 1)th calibration range;

[0141] Among them, for i = 2,..., k, the first calibration range is a part of the (i - 1)th calibration range.

[0142] For example, for a camera, its external parameters include three translation parameter vectors and three rotation parameter vectors. Among them, the three translation parameter vectors belong to the same category, and the three rotation parameter vectors belong to the same category. These six parameters can be expressed as , where represents a vector containing three translation parameters, represents a vector containing three rotation parameters. In an example, the "each external parameter" in step 103 refers to these six external parameters.

[0143] To improve the calibration effect, in the embodiments of the present application, at least two scales of calibration ranges are respectively set for each external parameter, and multiple iterative calibrations from coarse to fine (scales from large to small) are adopted, which can enable coarse-grained external parameter calibration in the early stage to ensure robustness to large external parameter errors, and fine-grained external parameter calibration in the later stage to ensure the accuracy of the calibration result.

[0144] As an example, assume k = 2, that is, correction ranges of two scales are set for the above six extrinsic parameters respectively. First, let i = 1, and perform the first extrinsic parameter correction in the first correction range with a larger scale (one extrinsic parameter correction includes multiple rounds of iteration); then, let i = 2, and perform the second extrinsic parameter correction in the second correction range with a smaller scale. Among them, the first correction range of the three rotation parameters is (-3, +3) degrees, and the correction range of the three translation parameters is (-0.1, 0.1) meters; the second correction range of the three rotation parameters is (-0.5, +0.5) degrees, and the second correction range of the three translation parameters is (-0.02, 0.02) meters. It can be seen that for the same extrinsic parameter, the second correction range is a part of the first correction range. Then, for each extrinsic parameter correction process in the first correction range, the compensation value of any translation extrinsic parameter , and the compensation value of any rotation extrinsic parameter (-3, 3). After the iterative search is completed, use the final result (the optimal corrected values of each extrinsic parameter) to update the extrinsic parameter values of the target imaging device, and then perform the iterative search of each extrinsic parameter in the second correction range, that is, the i-th correction range is a part of the (i - 1)-th correction range; for each extrinsic parameter correction process in the second correction range, the compensation value of any translation extrinsic parameter , and the compensation value of any rotation extrinsic parameter (-0.5, +0.5). After the iterative search is completed, use the final result (the optimal corrected values of each extrinsic parameter) to update the extrinsic parameter values of the target imaging device. Through the correction strategy with the scale of the correction range from coarse to fine, the robustness of the algorithm to large errors can be enhanced, and at the same time, the accuracy of the correction can be guaranteed.

[0145] Among them, selecting an extrinsic parameter compensation value from the i-th correction ranges corresponding to the multiple extrinsic parameters respectively may include: randomly extracting an extrinsic parameter compensation value from the multiple i-th compensation value lists corresponding to the multiple extrinsic parameters to obtain multiple i-th corrected extrinsic parameter values corresponding to each of the multiple extrinsic parameters, where one of the extrinsic parameters corresponds to one i-th compensation value list determined according to the i-th correction range of this extrinsic parameter.

[0146] In some embodiments, for one i-th compensation value list corresponding to one of the extrinsic parameters determined according to the i-th correction range of this extrinsic parameter, the i-th compensation value list is an arithmetic sequence determined according to the i-th correction range.

[0147] For example, assuming k = 2 as previously mentioned, that is, two calibration ranges of scales are set for each external parameter. At the first scale, the first calibration range of the three rotation parameters is (-3, +3) degrees, and the first calibration range of the three translation parameters is (-0.1, 0.1) meters; at the second scale, the second calibration range of the three rotation parameters is (-0.5, +0.5) degrees, and the second calibration range of the three translation parameters is (-0.02, 0.02) meters. Then, for the first calibration range, an equally divided preset interval can be set. If the preset interval is 0.1, a list of compensation values corresponding to the rotation parameters can be calculated as , and the calculation of the list of compensation values corresponding to the translation parameters is the same. Among the six lists of compensation values composed of 3 rotation compensation values and 3 translation compensation values, one compensation value is randomly selected each time to obtain six corrected external parameter values of the six external parameters respectively. Taking the uncorrected external parameter value as , and the six corrected external parameter values obtained by randomly selecting one compensation value from the six lists of compensation values are as an example, the corrected external parameter value can be expressed as .

[0148] Among them, the step of updating the values of the multiple external parameters of the target imaging device to the multiple corrected external parameter values of the i-th corresponding to the pixel feature of the target feature point satisfying a preset calibration condition under the preset view angle of the target imaging device corrected according to the multiple corrected external parameter values of the i-th includes:

[0149] Obtain a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature point under the preset view angle of the reference imaging device and the target imaging device before updating the external parameter values;

[0150] Based on the multiple corrected external parameter values of the i-th, determine a second pixel feature distance of the target feature point under the preset view angle of the reference imaging device and the target imaging device;

[0151] In response to the second pixel feature distance being less than the first pixel feature distance, update the values of the multiple external parameters of the target imaging device to the multiple corrected external parameter values of the i-th corresponding.

[0152] For i = 1 and when the external parameter calibration step a is executed for the first time, the obtaining of the first pixel feature distance includes:

[0153] Determine the pixel value of the target feature point in the first input image as the first pixel value;

[0154] Determine the pixel value of the target feature point in the second input image as the second pixel value;

[0155] Determine a first pixel feature distance according to the first pixel value and the second pixel value.

[0156] Among them, determining the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device based on the multiple i-th calibration extrinsic parameter values includes:

[0157] Determine a third pixel value of the target feature point under the preset viewing angle of the target imaging device according to the multiple i-th calibration extrinsic parameter values;

[0158] Determine the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

[0159] As an example, the absolute value of the difference between the pixel values of two pixels can be used as the feature distance between the two pixels. That is to say, the first pixel feature distance can be the absolute value of the difference between the first pixel value and the second pixel value, and the second pixel feature distance can be the absolute value of the difference between the first pixel value and the third pixel value. It can be understood that the pixel feature distance can characterize the error between two pixels, that is, the first pixel feature distance can characterize the pixel value error of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device before calibration, and the second pixel feature distance can characterize the pixel value error of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device after calibration. If the error decreases (i.e., the loss decreases) before and after calibration, it means that the calibration is effective.

[0160] In some other embodiments, determining the first pixel feature distance according to the first pixel value and the second pixel value includes: determining the first pixel feature distance according to the first pixel value, the luminance modulation coefficient, and the second pixel value, where the luminance modulation coefficient is determined according to the luminance difference of the overlapping region in the first input image and the second input image. Correspondingly, determining the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value includes: determining the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value, the luminance modulation coefficient, and the third pixel value.

[0161] It can be understood that introducing the luminance modulation coefficient when determining the first pixel feature distance and the second pixel feature distance can reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the luminance difference, and thus avoid affecting the calibration accuracy.

[0162] Optionally, the external parameter calibration step a may further include: in response to the second pixel feature distance being less than the first pixel feature distance, updating the first pixel feature distance to the second pixel feature distance.

[0163] An external parameter calibration method for a shooting device provided by an embodiment of the present application can automatically identify target feature points in the overlapping area after determining the overlapping area between the reference image and the image under the preset viewing angle of the target shooting device, and then perform multi-scale calibration on the external parameters of the target shooting device according to the target feature points and the calibration ranges of different scales corresponding to each external parameter. There is no need to return the target shooting device for re-calibration. Therefore, the calibration process is more convenient for users and can improve the user experience of using the device.

[0164] In addition, an external parameter calibration method for a shooting device provided by an embodiment of the present application: 1) Compared with the external parameter calibration scheme of a surround view camera using lane line features in the related art, the embodiment of the present application can use any natural scene for feature extraction in the camera overlapping viewing angle area, not limited to lane lines, so the limitation of external parameter calibration can be further reduced; 2) Instead of using all pixels in the overlapping viewing angle for external parameter calibration, sparse edge feature points and corner feature points with larger gradients are extracted for external parameter calibration, so the overall calculation amount in the entire calibration process can be reduced, the consumption of computing resources can be reduced, and the calibration efficiency can be improved; 3) When determining the first pixel feature distance and the second pixel feature distance, a brightness modulation coefficient is introduced, which can reduce the error introduced by different camera exposures to prevent the distortion of the pixel feature distance caused by brightness differences, thereby avoiding affecting the calibration accuracy.

[0165] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0166] Figure 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Please refer to Figure 7 , at the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory, such as at least one disk memory, etc. Of course, the electronic device may also include other hardware required for other services.

[0167] The processor, network interface, and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 only a bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0168] Memory, used to store programs. Specifically, the program can include program code, and the program code includes computer operation instructions. The memory can include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0169] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an external parameter calibration device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0170] Determine the overlapping area between the first input image and the second input image, where the first input image is a reference image and the second input image is an image at a preset viewing angle of the target imaging device;

[0171] Determine the target feature points in the overlapping area;

[0172] Obtain at least two scales of calibration ranges for each external parameter setting of the target imaging device;

[0173] In the order from the largest scale to the smallest scale, for each scale, correct the external parameter values of the target imaging device according to the target feature points and the calibration ranges of the external parameters at the scale, where the calibration result of the external parameter values at the previous scale is the calibration reference for the external parameter values at the next scale, and the calibration range of the next scale is a part of the calibration range of the previous scale.

[0174] Alternatively, the processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an external parameter calibration device at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0175] Determine the overlapping region between the first input image and the second input image, where the first input image is a reference image and the second input image is an image captured from a preset perspective of the target imaging device;

[0176] Determine the target feature points in the overlapping region;

[0177] Successively set i = 1, 2,..., k, and repeat the external parameter calibration step a after each assignment of i until a preset termination condition is satisfied;

[0178] Wherein, the external parameter calibration step a includes: obtaining the i-th calibration range set for each of the multiple external parameters of the target imaging device, respectively selecting an external parameter compensation value from the i-th calibration ranges corresponding to the multiple external parameters to obtain multiple i-th calibrated external parameter values corresponding to the multiple external parameters, and in response to the pixel features of the target feature points in the preset perspective of the target imaging device calibrated according to the multiple i-th calibrated external parameter values satisfying a preset calibration condition, correspondingly updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values, where the i-th calibrated external parameter value of each external parameter is equal to the sum of the value of the external parameter before update and the i-th compensation value of the external parameter;

[0179] Wherein, k is an integer greater than or equal to 2, and for i = 2,..., k, the first external parameter update for the i-th calibration range is based on the result of the last external parameter update corresponding to the (i - 1)-th calibration range;

[0180] Wherein, for i = 2,..., k, the i-th calibration range is a part of the (i - 1)-th calibration range.

[0181] The above as in this application Figure 7The method executed by the external parameter calibration device disclosed in the illustrated embodiment can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit in the hardware of the processor or the instructions in software form. The above-mentioned processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by the combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0182] The electronic device can also execute Figure 1 or Figure 6 the method, and implement the functions of the external parameter calibration device in Figure 1 or Figure 6 the illustrated embodiment. The embodiments of the present application will not be elaborated herein.

[0183] Of course, in addition to the software implementation method, the electronic device of the present application does not exclude other implementation methods, such as the way of logic devices or the combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, and can also be hardware or logic devices.

[0184] The embodiments of the present application also propose a computer-readable storage medium. The computer-readable storage medium stores one or more programs. The one or more programs include instructions. When the instructions are executed by a portable electronic device including a plurality of target application programs, the portable electronic device can be enabled to execute Figure 1 or Figure 6 the method of the illustrated embodiment.

[0185] The embodiment of the present application also provides a computer program product including instructions. When a computer runs the instructions of the computer program product, the computer executes the external parameter calibration method as Figure 1 or Figure 6 shown.

[0186] Figure 8 is a schematic structural diagram of an external parameter calibration device 800 according to an embodiment of the present application. Please refer to Figure 8 , in a software implementation manner, the external parameter calibration device 800 may include: an overlapping area determination module 801, a feature point determination module 802, a calibration range acquisition module 803, and an external parameter calibration module 804.

[0187] The overlapping area determination module 801 is configured to determine the overlapping area between a first input image and a second input image, where the first input image is a reference image, and the second input image is an image at a preset viewing angle of a target imaging device.

[0188] Wherein, the target imaging device is the device to be calibrated.

[0189] In some embodiments, the preset viewing angle may be a Birds Eyes View (BEV). The first input image may be a bird's-eye view obtained by converting a first image captured by a reference imaging device, and the second input image may be a bird's-eye view obtained by converting a second image captured by the target imaging device. Wherein, the shooting ranges of the reference imaging device and the target imaging device overlap, and the first image and the second image are captured by the reference imaging device and the target imaging device for the same scene from different viewing angles simultaneously.

[0190] Taking the vehicle surround view camera system as an example, the reference imaging device may be one of the cameras, and the target imaging device may be an adjacent camera.

[0191] Optionally, the device 800 may further include: a distortion removal module and an image viewing angle conversion module.

[0192] The distortion removal module is configured to remove the distortion of the first image and the second image before determining the overlapping area between the first input image and the second input image.

[0193] The image viewing angle conversion module is configured to convert the de-distorted first image into the first input image; and convert the de-distorted second image into the second input image.

[0194] Further, the image perspective conversion module is used to: when the preset perspective is the bird's-eye view perspective, convert the undistorted first image into an image in the bird's-eye view perspective to obtain the first input image; convert the undistorted second image into an image in the bird's-eye view perspective to obtain the second input image.

[0195] In some embodiments, the undistortion module may specifically be used to: undistort the first image by using the distortion parameters (usually provided by the camera manufacturer) and the internal parameters of the reference imaging device, and undistort the second image by using the distortion parameters (usually provided by the camera manufacturer) and the internal parameters of the target imaging device.

[0196] In some embodiments, the image perspective conversion module may specifically be used to:

[0197] Use the external parameters of the reference imaging device and the initialization parameters of the bird's-eye view perspective to obtain a bird's-eye view image with errors - the first input image;

[0198] Use the initial external parameters of the target imaging device and the initialization parameters of the bird's-eye view perspective to obtain a bird's-eye view image with errors - the second input image.

[0199] Optionally, the apparatus 800 may further include: a cropping module, configured to crop the first input image and the second input image converted into the bird's-eye view perspective to obtain the required image regions, and then calculate the overlapping region between the two. Among them, the pixels in the overlapping region of the two cropped images are not zero, so based on this principle, the region where the pixels in the first input image and the second input image in the bird's-eye view perspective are not zero can be calculated to obtain the overlapping region. Optionally, the apparatus 800 may further include: an overlapping region enhancement module, configured to perform processing such as binarization and morphological operations on the overlapping region, and remove the noise points whose pixel values may be zero to obtain a mask (Mask) of the overlapping region; then perform an erosion operation on the Mask image and remove the edge region to obtain an image of the overlapping region that is clear and free of noise.

[0200] The feature point determination module 802 is configured to determine the target feature points in the overlapping region.

[0201] Optionally, the apparatus 800 may further include: an overlapping region preprocessing module, configured to perform preprocessing on the overlapping region of the first input image, such as filtering processing, to further eliminate the noise in the overlapping region before determining the target feature points in the overlapping region.

[0202] Wherein, the target feature points are sparse feature points in the overlapping region. For example, the target feature points may be at least one of the edge feature points and the corner feature points in the overlapping region.

[0203] In some embodiments, the target feature points include edge feature points and corner feature points. The feature point determination module 802 can be used to: extract edge feature points from the overlapping region of the first input image based on the Sobel operator; and extract corner feature points from the overlapping region of the first input image based on the Harris corner feature point detection algorithm.

[0204] In some embodiments, the apparatus 800 may further include: an exposure coordination module, configured to determine the luminance modulation coefficient according to the luminance information of the overlapping region in the first input image and the luminance information of the overlapping region in the second input image. The luminance modulation coefficient can be used to subsequently eliminate the luminance difference between the two images, reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the luminance difference, and thus avoid affecting the calibration accuracy.

[0205] In some embodiments, the determining the luminance modulation coefficient according to the luminance information of the overlapping region in the first input image and the luminance information of the overlapping region in the second input image may include: determining a first statistical result of the luminance of each pixel in the overlapping region of the first input image; determining a second statistical result of the luminance of each pixel in the overlapping region of the second input image; and determining the luminance modulation coefficient according to the first statistical result and the second statistical result.

[0206] Wherein, the first statistical result and the second statistical result are of the same type of statistical results. For example, if the first statistical result is the median, then the second statistical result is also the median; if the first statistical result is the mean, then the second statistical result is also the mean.

[0207] The calibration range acquisition module 803 is configured to acquire at least two scales of calibration ranges for each external parameter setting of the target imaging device.

[0208] Wherein, at the same scale, the calibration ranges of the same type of external parameters may be the same or different.

[0209] The external parameter calibration module 804 is configured to, in the order from the largest scale to the smallest scale, for each scale, calibrate each external parameter value of the target imaging device according to the target feature points and the calibration range of each external parameter at the scale. Wherein, the calibration result of each external parameter value at the previous scale is the calibration reference for each external parameter value at the next scale, and the calibration range of the next scale is a part of the calibration range of the previous scale.

[0210] For example, for a camera, its external parameters include three translation parameter vectors and three rotation parameter vectors. Among them, the three translation parameter vectors belong to the same category, and the three rotation parameter vectors belong to the same category. These six parameters can be expressed as , where represents a vector containing three translation parameters, and represents a vector containing three rotation parameters. In one example, the "external parameters" in step 103 refer to these six external parameters.

[0211] To improve the calibration effect, the embodiments of the present application respectively set at least two scales of calibration ranges for each external parameter, and adopt multiple iterative calibrations from coarse to fine (scales from large to small), which can perform coarse-grained external parameter calibration in the early stage, so as to ensure the robustness to large external parameter errors, and perform fine-grained external parameter calibration in the later stage, which can ensure the accuracy of the calibration result.

[0212] In some embodiments, the external parameter calibration module 804 can be used to: for each of the scales, repeatedly execute the first specified step until a preset termination condition is met.

[0213] Among them, the preset termination condition may include but is not limited to at least one of the following: reaching the upper limit of the preset number of iterations, reaching the upper limit of the preset iteration time, etc.

[0214] Among them, the first specified step may include:

[0215] Select an external parameter compensation value from the calibration ranges of each external parameter at the scale to obtain a set of external parameter compensation values;

[0216] Determine the calibrated external parameter value of the target imaging device according to the uncalibrated external parameter value of the target imaging device and the set of external parameter compensation values;

[0217] In response to the pixel features of the target feature points in the preset viewing angle of the calibrated target imaging device satisfying the preset calibration condition, update the uncalibrated external parameter value of the target imaging device to the calibrated external parameter value.

[0218] Among them, selecting an external parameter compensation value from the calibration ranges of each external parameter at the scale includes: randomly extracting an external parameter compensation value from multiple compensation value lists of each external parameter at the scale to obtain a set of external parameter compensation values, where at one scale, one external parameter corresponds to a compensation value list determined according to the calibration range of the external parameter. It is also possible to sequentially extract an external parameter compensation value from the compensation value lists of each external parameter at the scale according to the number of iterations at the scale to obtain a set of external parameter compensation values.

[0219] In some embodiments, the device further includes: a compensation value list setting module, configured to, before respectively selecting an external parameter compensation value from the calibration ranges of the respective external parameters at the scale, set external parameter compensation values according to a preset interval respectively under the calibration range corresponding to each external parameter at the scale, so as to obtain the multiple compensation value lists.

[0220] In some embodiments, the updating the uncalibrated external parameter value of the target imaging device to the calibrated external parameter value in response to the pixel feature of the target feature point satisfying a preset calibration condition under the preset viewing angle of the calibrated target imaging device may include:

[0221] Obtaining a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device before updating the external parameter value;

[0222] Based on the calibrated external parameter value, determining a second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device;

[0223] In response to the second pixel feature distance being less than the first pixel feature distance, updating the uncalibrated external parameter value of the target imaging device to the calibrated external parameter value.

[0224] Optionally, the external parameter calibration module 804 may further be configured to: in response to the second pixel feature distance being greater than or equal to the first pixel feature distance, ignore and / or record the calibrated external parameter value, and re-execute the first specified step, that is, continue the next round of iteration.

[0225] In some embodiments,

[0226] In the case of first executing the first specified step for the maximum scale, the obtaining the first pixel feature distance may include:

[0227] Determining a first pixel value as the pixel value of the target feature point in the first input image;

[0228] Determining a second pixel value as the pixel value of the target feature point in the second input image;

[0229] Determining a first pixel feature distance according to the first pixel value and the second pixel value.

[0230] In some embodiments, the determining, based on the calibrated external parameter value, a second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device may include:

[0231] Determine a third pixel value of the target feature point under the preset viewing angle of the corrected target imaging device according to the corrected external reference value;

[0232] Determine a second pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

[0233] As an example, the absolute value of the difference between the pixel values of two pixels can be used as the feature distance between the two pixels. That is, the first pixel feature distance can be the absolute value of the difference between the first pixel value and the second pixel value, and the second pixel feature distance can be the absolute value of the difference between the first pixel value and the third pixel value. It can be understood that the pixel feature distance can represent the error between two pixels. That is, the first pixel feature distance can represent the pixel value error of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device before correction, and the second pixel feature distance can represent the pixel value error of the target feature point under the preset viewing angles of the reference imaging device and the corrected target imaging device. If the error decreases before and after correction, it indicates that the correction is effective.

[0234] In some other embodiments, the determining the first pixel feature distance according to the first pixel value and the second pixel value includes: determining the first pixel feature distance according to the first pixel value, a brightness modulation coefficient, and the second pixel value, where the brightness modulation coefficient is determined according to the brightness difference of the overlapping region in the first input image and the second input image. Correspondingly, the determining the second pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value includes: determining the second pixel feature distance of the target feature point under the preset viewing angles of the reference imaging device and the target imaging device according to the first pixel value, the brightness modulation coefficient, and the third pixel value.

[0235] It can be understood that introducing the brightness modulation coefficient when determining the first pixel feature distance and the second pixel feature distance can reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the brightness difference, and thus avoid affecting the accuracy of the correction.

[0236] In some embodiments, the first specifying step may further include: in response to the second pixel feature distance being less than the first pixel feature distance, updating the first pixel feature distance to the second pixel feature distance.

[0237] The external parameter calibration device 800 for an imaging device provided in the embodiments of the present application can also execute Figure 1 the method and achieve Figure 1The functions of the illustrated embodiments are the same and achieve the same technical effects, and the embodiments of the present application will not be elaborated one by one here.

[0238] Figure 9 FIG. is a schematic structural diagram of an external parameter calibration device 900 according to an embodiment of the present application. Please refer to Figure 9 In a software implementation manner, the external parameter calibration device 900 may include: an overlapping area determination module 901, a feature point determination module 902, and an external parameter correction module 903.

[0239] The overlapping area determination module 901 is configured to determine an overlapping area between a first input image and a second input image, where the first input image is a reference image, and the second input image is an image at a preset viewing angle of a target imaging device.

[0240] The feature point determination module 902 is configured to determine target feature points in the overlapping area.

[0241] Wherein, the specific implementation processes of the overlapping area determination module 901 and the feature point determination module 902 correspond to and are consistent with the specific implementation processes of the overlapping area determination module 801 and the feature point determination module 802 in the Figure 8 illustrated embodiments, and can achieve the same technical effects, and will not be described repeatedly here.

[0242] The external parameter correction module 903 is configured to sequentially set i = 1, 2,..., k, and repeat the external parameter correction step a after each assignment to i until a preset termination condition is met.

[0243] Wherein, the preset termination condition may include but is not limited to at least one of the following: reaching the upper limit of the preset number of iterations, reaching the upper limit of the preset iteration time, and exhaustion of the compensation values in the multiple compensation value lists, etc.

[0244] Wherein, the external parameter correction step a may include: obtaining a first correction range set for each of the multiple external parameters of the target imaging device, respectively selecting an external parameter compensation value from the first correction ranges corresponding to the multiple external parameters to obtain multiple first correction external parameter values corresponding to the multiple external parameters, and in response to the pixel features of the target feature points in the preset viewing angle of the target imaging device corrected according to the multiple first correction external parameter values satisfying a preset correction condition, updating the values of the multiple external parameters of the target imaging device to the multiple first correction external parameter values, where the first correction external parameter value of each external parameter is equal to the sum of the value of the external parameter before update and the first compensation value of the external parameter.

[0245] Wherein, k is an integer greater than or equal to 2, and for i = 2,..., k, the first external parameter update for the i-th correction range is based on the result of the last external parameter update corresponding to the (i - 1)-th correction range;

[0246] Among them, for i = 2, ……, k, the i-th calibration range is a part of the (i - 1)-th calibration range.

[0247] For example, for a camera, its external parameters include three translation parameter vectors and three rotation parameter vectors. Among them, the three translation parameter vectors belong to the same category, and the three rotation parameter vectors belong to the same category. These six parameters can be expressed as , where represents a vector containing three translation parameters, represents a vector containing three rotation parameters. In one example, the "each external parameter" in step 103 refers to these six external parameters.

[0248] To improve the calibration effect, in the embodiments of the present application, at least two scales of calibration ranges are respectively set for each external parameter, and multiple iterative calibrations from coarse to fine (scales from large to small) are adopted, which can enable coarse-grained external parameter calibration in the early stage to ensure robustness to large external parameter errors, and fine-grained external parameter calibration in the later stage to ensure the accuracy of the calibration result.

[0249] Among them, selecting one external parameter compensation value from the i-th calibration ranges corresponding to the multiple external parameters may include: randomly extracting one external parameter compensation value from the multiple i-th compensation value lists corresponding to the multiple external parameters to obtain multiple i-th calibrated external parameter values corresponding to each of the multiple external parameters, where one external parameter corresponds to one i-th compensation value list determined according to the i-th calibration range of this external parameter.

[0250] In some embodiments, for one i-th compensation value list corresponding to one external parameter determined according to the i-th calibration range of this external parameter, the i-th compensation value list is an arithmetic progression determined according to the i-th calibration range.

[0251] Among them, the step of updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values in response to the pixel features of the target feature point in the preset viewing angle of the target imaging device calibrated according to the multiple i-th calibrated external parameter values satisfying the preset calibration condition includes:

[0252] Obtaining a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device before updating the external parameter values;

[0253] Based on the multiple i-th calibrated external parameter values, determining a second pixel feature distance of the target feature point in the preset viewing angles of the reference imaging device and the target imaging device;

[0254] In response to the second pixel feature distance being less than the first pixel feature distance, the values of the multiple extrinsic parameters of the target imaging device are correspondingly updated to the multiple i-th corrected extrinsic parameter values.

[0255] For i = 1 and when the extrinsic parameter correction step a is executed for the first time, the obtaining of the first pixel feature distance includes:

[0256] Determine the pixel value of the target feature point in the first input image as the first pixel value;

[0257] Determine the pixel value of the target feature point in the second input image as the second pixel value;

[0258] Determine the first pixel feature distance according to the first pixel value and the second pixel value.

[0259] Wherein, the determining of the second pixel feature distance of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device based on the multiple i-th corrected extrinsic parameter values includes:

[0260] Determine the third pixel value of the target feature point at the preset viewing angle of the target imaging device according to the multiple i-th corrected extrinsic parameter values;

[0261] Determine the second pixel feature distance of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

[0262] As an example, the absolute value of the difference between the pixel values of two pixels can be used as the feature distance between the two pixels. That is to say, the first pixel feature distance can be the absolute value of the difference between the first pixel value and the second pixel value, and the second pixel feature distance can be the absolute value of the difference between the first pixel value and the third pixel value. It can be understood that the pixel feature distance can characterize the error between two pixels. That is, the first pixel feature distance can characterize the pixel value error of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device before correction, and the second pixel feature distance can characterize the pixel value error of the target feature point at the preset viewing angle of the reference imaging device and the target imaging device after correction. If the error decreases (i.e., the loss decreases) before and after correction, it indicates that the correction is effective.

[0263] In some other embodiments, determining the first pixel feature distance according to the first pixel value and the second pixel value includes: determining the first pixel feature distance according to the first pixel value, a brightness modulation coefficient, and the second pixel value, where the brightness modulation coefficient is determined according to the brightness difference of the overlapping area in the first input image and the second input image. Correspondingly, determining the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value includes: determining the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value, the brightness modulation coefficient, and the third pixel value.

[0264] It can be understood that introducing the brightness modulation coefficient when determining the first pixel feature distance and the second pixel feature distance can reduce the error introduced by different camera exposures, prevent the distortion of the pixel feature distance caused by the brightness difference, and thus avoid affecting the calibration accuracy.

[0265] Optionally, the external parameter calibration step a may further include: in response to the second pixel feature distance being less than the first pixel feature distance, updating the first pixel feature distance to the second pixel feature distance.

[0266] The external parameter calibration device 900 for an imaging device provided in the embodiments of the present application can also execute Figure 6 the method and implement Figure 6 the functions of the embodiments shown and achieve the same technical effects. The embodiments of the present application will not be described in detail herein.

[0267] In summary, the above are only the preferred embodiments of the present application and are not intended to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0268] The systems, devices, modules, or units described in the above embodiments can be specifically implemented by a computer chip or an entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0269] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0270] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0271] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. An external parameter calibration method for a photographing device, characterized in that The method includes: Determine the overlapping region between a first input image and a second input image, where the first input image is a reference image and the second input image is an image captured from a preset perspective of a target imaging device; Determine target feature points in the overlapping region; Obtain correction ranges for k scales of each external parameter setting of the target imaging device, where k is an integer greater than or equal to 2, and the correction range is the value range of the external parameter compensation value of the target imaging device; In the order from the largest scale to the smallest scale, for each scale, correct the external parameter values of the target imaging device according to the target feature points and the correction range of each external parameter at the scale. Among them, the correction result of the external parameter values at the previous scale is the correction reference for the external parameter values at the next scale, and the correction range of the next scale is a part of the correction range of the previous scale.

2. The method according to claim 1, wherein The step of correcting the external parameter values of the target imaging device according to the target feature points and the correction range of each external parameter at the scale for each scale includes: For each scale, repeatedly execute a first specified step until a preset termination condition is met; Among them, the first specified step includes: Respectively select an external parameter compensation value from the correction ranges of each external parameter at the scale to obtain a set of external parameter compensation values; According to the uncorrected external parameter values of the target imaging device and the set of external parameter compensation values, determine the corrected external parameter values of the target imaging device; In response to the pixel features of the target feature points in the preset perspective of the corrected target imaging device satisfying a preset correction condition, update the uncorrected external parameter values of the target imaging device to the corrected external parameter values.

3. The method according to claim 2, wherein The step of respectively selecting an external parameter compensation value from the correction ranges of each external parameter at the scale includes: Randomly select an external parameter compensation value from multiple compensation value lists of each external parameter at the scale respectively to obtain a set of external parameter compensation values. Among them, at one scale, one external parameter corresponds to a compensation value list determined according to the correction range of the external parameter.

4. The method according to claim 3, wherein Before respectively selecting an external parameter compensation value from the correction ranges of each external parameter at the scale, the method further includes: Under the correction range corresponding to each external parameter at the scale, set external parameter compensation values respectively according to a preset interval to obtain the multiple compensation value lists.

5. The method according to claim 3, wherein The preset termination condition includes at least one of the following: Reaching the upper limit of the preset number of iterations; Reaching the upper limit of the preset iteration time; All compensation values in the multiple compensation value lists are used up.

6. The method according to claim 2, characterized in that The step of, in response to the pixel features of the target feature points in the preset perspective of the corrected target imaging device satisfying a preset correction condition, updating the uncorrected external parameter values of the target imaging device to the corrected external parameter values includes: Obtain a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature points in the preset perspective of a reference imaging device and the target imaging device before updating the external parameter values; Based on the calibrated extrinsic parameter value, determine a second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device; In response to the second pixel feature distance being less than the first pixel feature distance, update the uncalibrated extrinsic parameter value of the target imaging device to the calibrated extrinsic parameter value.

7. The method according to claim 6, wherein When the first specified step is first executed for the maximum scale, the obtaining of the first pixel feature distance includes: Determine a first pixel value of the target feature point in the first input image; Determine a second pixel value of the target feature point in the second input image; Determine a first pixel feature distance according to the first pixel value and the second pixel value.

8. The method according to claim 7, wherein The determining of the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device based on the calibrated extrinsic parameter value includes: Determine a third pixel value of the target feature point under the preset viewing angle of the calibrated target imaging device according to the calibrated extrinsic parameter value; Determine a second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

9. The method according to claim 8, wherein The determining of the first pixel feature distance according to the first pixel value and the second pixel value includes: Determine a first pixel feature distance according to the first pixel value, a brightness modulation coefficient, and the second pixel value, where the brightness modulation coefficient is determined according to the brightness difference of the overlapping region in the first input image and the second input image.

10. The method according to claim 9, characterized in that The determining of the second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value includes: Determine a second pixel feature distance of the target feature point under the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value, the brightness modulation coefficient, and the third pixel value.

11. The method according to claim 9, wherein Before determining the first pixel feature distance according to the first pixel value, the brightness modulation coefficient, and the second pixel value, the method further includes: Determine the brightness modulation coefficient according to the brightness information of the overlapping region in the first input image and the brightness information of the overlapping region in the second input image.

12. The method according to claim 11, wherein, The determining of the brightness modulation coefficient according to the brightness information of the overlapping region in the first input image and the brightness information of the overlapping region in the second input image includes: Determine a first statistical result of the pixel brightnesses of the overlapping region in the first input image; Determine a second statistical result of the pixel brightnesses of the overlapping region in the second input image; Determine the brightness modulation coefficient according to the first statistical result and the second statistical result.

13. The method according to any one of claims 6-12, characterized in that, The first specified step further includes: In response to the second pixel feature distance being less than the first pixel feature distance, update the first pixel feature distance to the second pixel feature distance.

14. The method according to any one of claims 1 to 12, characterized in that, The preset perspective is a bird's-eye view perspective. The first input image is a bird's-eye view image obtained by converting a first image captured by a reference imaging device, and the second input image is a bird's-eye view image obtained by converting a second image captured by the target imaging device.

15. The method according to claim 14, wherein The reference imaging device and the target imaging device are fisheye cameras. Before determining the overlapping region between the first input image and the second input image, the method further includes: Undistorting the first image and the second image; Converting the undistorted first image into the first input image; Converting the undistorted second image into the second input image.

16. The method according to any one of claims 1-12, characterized in that, The target feature points include edge feature points and corner feature points. Among them, determining the target feature points in the overlapping region includes: Extracting edge feature points from the overlapping region of the first input image based on the Sobel operator; Extracting corner feature points from the overlapping region of the first input image based on the Harris corner feature point detection algorithm.

17. An external parameter calibration method for a photographing device, characterized in that, The method includes: Determining the overlapping region between the first input image and the second input image, where the first input image is a reference image and the second input image is an image under the preset perspective of the target imaging device; Determining the target feature points in the overlapping region; Sequentially setting i = 1, 2,..., k, and repeating the external parameter calibration step a after each assignment of i until a preset termination condition is satisfied; Among them, the external parameter calibration step a includes: obtaining the i-th calibration range set for each of the multiple external parameters of the target imaging device, respectively selecting an external parameter compensation value from the i-th calibration ranges corresponding to the multiple external parameters to obtain multiple i-th calibrated external parameter values corresponding to the multiple external parameters, and in response to the pixel features of the target feature points in the preset perspective of the target imaging device calibrated according to the multiple i-th calibrated external parameter values satisfying the preset calibration condition, correspondingly updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values, where the i-th calibrated external parameter value of each external parameter is equal to the sum of the value of the external parameter before update and the i-th compensation value of the external parameter; Among them, k is an integer greater than or equal to 2, and for i = 2,..., k, the first external parameter update for the i-th calibration range is based on the result of the last external parameter update corresponding to the (i - 1)-th calibration range; Among them, for i = 2,..., k, the i-th calibration range is a part of the (i - 1)-th calibration range.

18. The method according to claim 17, wherein Respectively selecting an external parameter compensation value from the i-th calibration ranges corresponding to the multiple external parameters to obtain multiple i-th calibrated external parameter values corresponding to the multiple external parameters, including: Randomly extracting an external parameter compensation value from the multiple i-th compensation value lists corresponding to the multiple external parameters to obtain multiple i-th calibrated external parameter values corresponding to the multiple external parameters, where one external parameter corresponds to one i-th compensation value list determined according to the i-th calibration range of the external parameter.

19. According to the method of claim 18, wherein For an external parameter corresponding to an i-th compensation value list determined according to an i-th calibration range of the external parameter, the i-th compensation value list is an arithmetic progression determined according to the i-th calibration range.

20. The method according to claim 17, wherein Responding to the pixel feature of the target feature point in the preset viewing angle of the target imaging device after being calibrated according to the multiple i-th calibrated external parameter values satisfying a preset calibration condition, updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values includes: Obtaining a first pixel feature distance, where the first pixel feature distance is the pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device before updating the external parameter values. Based on the multiple i-th calibrated external parameter values, determining a second pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device. Responding to the second pixel feature distance being less than the first pixel feature distance, updating the values of the multiple external parameters of the target imaging device to the multiple i-th calibrated external parameter values.

21. The method according to claim 20, wherein For i = 1 and when the external parameter calibration step a is executed for the first time, the obtaining of the first pixel feature distance includes: Determining a first pixel value of the target feature point in the first input image. Determining a second pixel value of the target feature point in the second input image. Determining the first pixel feature distance according to the first pixel value and the second pixel value.

22. The method according to claim 21, wherein The determining of the second pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device based on the multiple i-th calibrated external parameter values includes: Determining a third pixel value of the target feature point in the preset viewing angle of the target imaging device according to the multiple i-th calibrated external parameter values. Determining the second pixel feature distance of the target feature point in the preset viewing angle of the reference imaging device and the target imaging device according to the first pixel value and the third pixel value.

23. The method according to any one of claims 20 - 22, characterized in that, The external parameter calibration step a further includes: Responding to the second pixel feature distance being less than the first pixel feature distance, updating the first pixel feature distance to the second pixel feature distance.

24. An electronic device, characterized in that, Including: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the method according to any one of claims 1 to 23.

25. A computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 23.

26. A computer program product comprising instructions, characterized in that, When the computer runs the instructions of the computer program product, the computer executes the method according to any one of claims 1 to 23.

Images