External parameter calibration method and device

Abstract

The invention provides an external parameter calibration method and device, and the method comprises the steps: obtaining a plurality of feature copies of an original image corresponding to a target main body, and carrying out the translation on the feature copies, and obtaining a first image feature corresponding to the feature copies; determining a feature relationship between feature points in the second image feature and the first image feature, and obtaining a matching cost graph corresponding to the feature copy; the second image feature is obtained based on a projection depth map corresponding to the target body; and determining a cost matrix corresponding to the target subject based on the plurality of matching cost maps corresponding to the target subject, so as to determine external parameters of the target subject based on the cost matrix. According to the embodiment, the applicability of the calibration method can be improved, and the efficiency and accuracy of external parameter calibration are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and more specifically, to an external parameter calibration method and apparatus. Background Technology

[0002] In existing technologies, most commonly used calibration methods rely heavily on the scene. For example, the scene needs to contain orthogonal vertical features such as vertical flagpoles and horizontal lane lines for extrinsic parameter calibration, which reduces the applicability of the calibration methods. In addition, since different vehicles in the field of autonomous driving are equipped with different types of sensors, traditional calibration methods need to model and calibrate each sensor or camera model separately, and are highly dependent on the initial extrinsic parameters, resulting in low efficiency and accuracy of extrinsic parameter calibration. Summary of the Invention

[0003] In view of this, the embodiments of the present invention provide at least one external parameter calibration method, apparatus, electronic device and storage medium, which can improve the applicability of the calibration method and improve the efficiency and accuracy of external parameter calibration.

[0004] In a first aspect, embodiments of the present invention provide an external parameter calibration method, including:

[0005] Obtain multiple feature copies of the original image corresponding to the target subject, apply translation to the feature copies to obtain the first image feature corresponding to the feature copies;

[0006] Determine the feature relationships between each feature point in the second image feature and the first image feature to obtain the matching cost map corresponding to the feature copy; the second image feature is obtained based on the projection depth map corresponding to the target subject.

[0007] Based on multiple matching cost maps corresponding to the target subject, a cost matrix corresponding to the target subject is determined, and the external parameters of the target subject are determined based on the cost matrix.

[0008] Optionally, before obtaining multiple feature copies of the original image corresponding to the target subject, the method further includes:

[0009] The initial extrinsic parameters of the target body are rotated or translated to generate deviation extrinsic parameters;

[0010] Based on the internal parameters of the target subject involved in the deviation, the point cloud data of the target subject is projected onto a two-dimensional image to obtain the projection depth map corresponding to the target subject, and the second image features corresponding to the target subject are determined based on the projection depth map.

[0011] Optionally, multiple feature copies of the original image corresponding to the target subject are obtained, including:

[0012] The number of copies corresponding to the feature copies is determined based on the degree of rotation or translation transformation applied to the initial extrinsic parameters of the target entity.

[0013] The third image features of the original image corresponding to the target subject are copied according to the number of copies, resulting in multiple feature copies of the original image corresponding to the target subject; the third image features are obtained by feature extraction of the original image.

[0014] Optionally, a translation is applied to the feature copy to obtain the first image feature corresponding to the feature copy, including:

[0015] A preset direction is applied to the feature copy in a two-dimensional space; the translation is a unit translation; the preset direction includes any one of the vertical direction, horizontal direction and combined direction, and the preset direction is different for different feature copies; wherein, the translation in the combined direction includes translating one unit distance in two orthogonal directions respectively;

[0016] For the gaps created after the feature copy is shifted, zero-filling is used to fill them in, thus obtaining the first image feature corresponding to the feature copy.

[0017] Optionally, based on multiple matching cost maps corresponding to the target subject, a cost matrix corresponding to the target subject is determined, including:

[0018] Multiple matching cost maps are multiplied by the second image features of the target subject to obtain the cost matrix corresponding to the target subject.

[0019] Optionally, the external parameters of the target entity are determined based on the cost matrix, including:

[0020] The cost matrix is ​​encoded to obtain a high-dimensional feature representation corresponding to the cost matrix;

[0021] Based on high-dimensional feature representation, the external parameters corresponding to the target are determined; the external parameters include position parameters and attitude parameters.

[0022] Secondly, embodiments of the present invention provide an external parameter calibration device, comprising:

[0023] The translation module is used to acquire multiple feature copies of the original image corresponding to the target subject, apply translation to the feature copies, and obtain the first image feature corresponding to the feature copy.

[0024] The determination module is used to determine the feature relationship between each feature point in the second image feature and the first image feature, and to obtain a matching cost map corresponding to the feature copy; the second image feature is obtained based on the projection depth map corresponding to the target subject.

[0025] The calibration module is used to determine the cost matrix corresponding to the target subject based on multiple matching cost maps, so as to determine the external parameters of the target subject based on the cost matrix.

[0026] Thirdly, embodiments of the present invention also provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the first aspect or any optional implementation of the first aspect are performed.

[0027] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the first aspect or any optional implementation thereof.

[0028] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0029] In any of the above aspects or any implementation of any aspect, by constructing multiple feature copies corresponding to the target subject and applying translation in the feature space to generate the first image feature, diversity is introduced into the image feature space, thereby enhancing the robustness of feature matching. By establishing feature point-level feature relationships between the first image feature and the second image feature extracted based on the target subject's projection depth map, a matching cost map can be effectively constructed, reflecting the similarity between the two sets of features at different spatial locations. Using matching cost maps generated under multiple different translation conditions to comprehensively construct the cost matrix helps to integrate information from multiple translation perspectives during feature matching, thereby significantly improving matching accuracy. Unlike traditional methods that rely on a single feature or a single image pair for matching, this embodiment of the invention introduces multiple feature copies and their translation transformations, improving the adaptability to target pose changes at the matching level and enhancing robustness to complex scenes such as local errors, occlusion, and weak texture regions. Furthermore, the cost matrix, as the fusion result of multiple matching cost maps, provides a more comprehensive and stable optimization objective in subsequent extrinsic parameter estimation, thereby making the final solved extrinsic parameters more accurate.

[0030] The beneficial effects of the aforementioned external parameter calibration device, electronic equipment, and storage medium are described in the description of the external parameter calibration method above, and will not be repeated here. Attached Figure Description

[0031] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to the present invention and, together with the specification, serve to explain the technical solutions of the present invention. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0032] Figure 1 A flowchart of an external parameter calibration method provided by an embodiment of the present invention is shown;

[0033] Figure 2 A flowchart illustrating an external parameter calibration method provided by an embodiment of the present invention is shown;

[0034] Figure 3 A schematic diagram of an external parameter calibration device provided in an embodiment of the present invention is shown;

[0035] Figure 4 An exemplary system architecture in which embodiments of the present invention can be applied is shown;

[0036] Figure 5 A schematic diagram of the structure of a computer system used to implement embodiments of the present invention is shown. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0038] It should be noted that the collection, use, storage, sharing and transfer of user personal information involved in the technical solution of the present invention all comply with the provisions of relevant laws and regulations, and require notification to users and obtaining their consent or authorization. When applicable, user personal information is subjected to de-identification and / or anonymization and / or encryption technical processing.

[0039] The above problems and solutions are the result of the inventor's practice and careful research. The discovery process of the above problems and the solutions proposed for the above problems should be considered as the inventor's contribution to the invention.

[0040] The technical solutions of this invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, of the embodiments of this invention. The components of this invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0041] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0042] To facilitate understanding of this embodiment, a detailed description of the extrinsic parameter calibration method disclosed in this invention will be provided first. The execution entity of the extrinsic parameter calibration method provided in this invention is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this extrinsic parameter calibration method can be implemented by the processor calling computer-readable instructions stored in memory.

[0043] See Figure 1 The diagram shows a flowchart of an external parameter calibration method provided in an embodiment of the present invention. The method includes steps S101 to S103, wherein:

[0044] S101: Obtain multiple feature copies of the original image corresponding to the target subject, apply translation to the feature copies to obtain the first image feature corresponding to the feature copies.

[0045] In this embodiment of the invention, the target subject typically refers to the sensor body or observation target involved in determining the spatial relationship between multiple sensors in the perception system. It may include the sensor device itself, such as a camera, LiDAR, depth camera, etc.; it may also include a sensor mounting platform, such as a vehicle body, robotic arm, drone, etc. It should be noted that the above-mentioned possible forms of the target subject are merely illustrative examples of feasible implementation methods in this embodiment of the invention and do not constitute an improper limitation of the invention. This embodiment of the invention does not specifically limit the form of the target subject, and the appropriate form can be selected based on actual circumstances in practical applications.

[0046] In this embodiment of the invention, before obtaining multiple feature copies of the original image corresponding to the target subject, the method further includes: performing rotation or translation transformation on the initial extrinsic parameters of the target subject to generate deviation extrinsic parameters; based on the deviation extrinsic parameters and the internal parameters of the target subject, projecting the point cloud data of the target subject onto a two-dimensional image to obtain a projection depth map corresponding to the target subject, so as to determine the second image features corresponding to the target subject based on the projection depth map.

[0047] In practical implementation, before extrinsic parameter calibration, to enhance the robustness of the calibration process to initial errors, the initial extrinsic parameters of the target subject can be perturbed, i.e., rotational or translational transformations can be applied to construct multiple bias extrinsic parameters. Initial extrinsic parameters refer to the initial pose estimation between sensors or between a sensor and the target subject, typically represented as a combination of a rotation matrix and a translation vector, describing the spatial transformation relationship from the target subject's coordinate system to the camera or other observation sensor's coordinate system. Specifically, to simulate potential attitude errors, perturbations within a certain angular range can be applied to the rotation matrix, such as rotating a few degrees around the X, Y, and Z axes, and a certain magnitude of offset can be applied to the translation vector, such as translating a few centimeters along each axis, thus obtaining a set of bias extrinsic parameters. Then, based on each bias extrinsic parameter and the target subject's known internal parameters, such as the camera's focal length, principal point position, and distortion parameters, the point cloud data of the target subject in three-dimensional space can be projected onto a two-dimensional image plane to generate a corresponding projection depth map. Each pixel value in the projection depth map represents the depth information of the projection point from the camera at that pixel location. After obtaining the projected depth map, a second image feature can be extracted based on it. This second image feature not only includes the depth values ​​at the image pixel locations but also multidimensional attributes reflecting spatial structural features, such as depth gradient, edge information, local geometry, normal vector distribution, and surface continuity. It is a set of descriptive vectors that reflects the geometric shape of the target subject. Therefore, by introducing a bias extrinsic parameter, we can simulate the situation where extrinsic parameters have errors in real-world scenarios. Furthermore, the extracted second image feature helps improve the accuracy and efficiency of solving the target subject's extrinsic parameters.

[0048] In this embodiment of the invention, to enhance the pose robustness and matching accuracy of the target subject during the extrinsic parameter calibration process, multiple feature copies associated with the original image can be generated. Specifically, obtaining multiple feature copies of the original image corresponding to the target subject includes: determining the number of copies corresponding to the feature copies based on the degree of rotation or translation transformation applied to the initial extrinsic parameters of the target subject; copying the third image feature of the original image corresponding to the target subject according to the number of copies, thereby obtaining multiple feature copies of the original image corresponding to the target subject; the third image feature is obtained by feature extraction from the original image.

[0049] In practical implementation, the required number of feature copies can be determined based on the degree of rotation or translation transformation applied to the initial extrinsic parameters. For example, when the transformation amplitude is small, such as a rotation angle within ±2 degrees and a translation distance within ±2 centimeters, 3 to 5 feature copies can be generated to cover a small range of disturbances. However, when the rotation angle increases to ±10 degrees and the translation distance increases to ±10 centimeters, indicating a potentially larger error in the initial extrinsic parameters, more copies, such as 10 to 20, can be generated to more fully simulate various possible viewpoint changes in the feature space, thereby improving the robustness of subsequent matching cost calculations. It should be noted that the above correspondence between different transformation degrees and different copy numbers is only illustrative of feasible implementation methods in this invention and does not constitute an improper limitation of the invention. In practical applications, the correspondence between the two can be set according to actual conditions and needs. This invention does not impose specific limitations in this regard, but only aims to achieve its function. After the number of copies is determined, feature copying operations can be performed on the original image according to the number of copies. The original image can be processed by feature extraction networks or operators, such as the backbone network in a convolutional neural network or ResNet, to extract its semantic and geometric information and generate third-party image features. These third-party image features are essentially a representation of the original image in feature space, typically represented as feature maps across multiple channels, containing key information about the target's edges, textures, and contours. Multiple feature copies are obtained through copying operations, each corresponding to a specific rotation or translation perturbation, used for the subsequent generation of the matching cost map.

[0050] In another possible implementation, besides determining the number of feature copies based on the degree of rotation or translation transformation, other criteria can also be used to determine the number of copies. For example, the number of copies can be determined based on the uncertainty estimate of the initial extrinsic parameters. If the extrinsic parameters come from low-precision sensors or coarse initialization, more copies can be generated to enhance the tolerance to attitude deviations. Adjustments can also be made based on the scale changes or occlusion risk of the target subject in the image: when the target is small or located at the edge of the image, the system may increase sampling density to avoid feature loss. It should be noted that the above methods for determining the number of copies are only illustrative examples of possible implementations in this invention and do not constitute an improper limitation of the invention. In practical applications, the corresponding method can be selected according to the actual situation. This invention does not impose specific limitations on this, but only on achieving its function.

[0051] In this embodiment of the invention, applying a translation to a feature copy to obtain a first image feature corresponding to the feature copy includes: applying a translation in a preset direction to the feature copy in a two-dimensional spatial position; the translation is a unit translation; the preset direction includes any one of a vertical direction, a horizontal direction, and a combined direction, and the preset direction is different for different feature copies; wherein, the translation in the combined direction includes translating one unit distance in two orthogonal directions respectively; for the empty spaces generated after the translation of the feature copy, zero-filling is used to fill them to obtain the first image feature corresponding to the feature copy.

[0052] In practical implementation, assuming there are 9 copies, after obtaining 9 feature copies corresponding to the target subject, a translation operation in a preset direction can be applied to the feature copies to generate the first image feature. Specifically, the translation operation is to translate the feature map of the feature copy in two-dimensional space by one unit, that is, move it one unit in a specified direction. The preset direction can include three categories: vertical direction, horizontal direction, and combined direction. The vertical direction can be up or down, the horizontal direction can be left or right, and the combined direction is to translate one unit in each of two orthogonal directions, such as "upper left" and "lower right", forming a diagonal movement. The system assigns different translation directions to each feature copy to enhance the image feature's responsiveness to perturbations in different directions. For example, if there are 9 feature copies, the following 9 translations can be applied respectively: up, down, left, right, upper left, upper right, lower left, lower right, and no translation, that is, keeping it in place for comparison. During the translation process, since the feature map is moved as a whole in a certain direction, empty spaces will be generated at the original image edge positions. To keep the size of the feature map constant, zero-padding can be used to fill in the gaps, that is, fill the edge gaps exposed after movement with all zero values ​​to ensure the continuity of position information and the consistency of calculation.

[0053] In another possible implementation, besides the unit translation method described above, local affine transformation can also be used. This involves applying a small-scale affine transformation, such as a small rotation, scaling, or shearing, to the feature copy to simulate the geometric changes of the image under different viewing angles or imaging conditions. This can also improve the sensitivity of the features to external parameter perturbations, and interpolation compensation can be used for edge processing. It should be noted that the above method for obtaining the first image features based on the feature copy is only illustrative of feasible implementation methods in this invention and does not constitute an improper limitation of the invention. In practical applications, the method can be selected according to the actual situation. This invention does not impose specific limitations on this method, but rather focuses on achieving its function.

[0054] S102: Determine the feature relationship between each feature point in the second image feature and the first image feature to obtain the matching cost map corresponding to the feature copy.

[0055] In embodiments of the present invention, such as Figure 2 As shown, after constructing the first and second image features, the feature relationship between them can be further determined to generate a matching cost map corresponding to each feature copy. Specifically, the similarity or difference between the feature points at corresponding positions of the second image features and each first image feature can be calculated to construct a cost space, which is used to characterize the matching quality under different extrinsic perturbations.

[0056] In practical implementation, a feature point can refer to a local feature vector at a two-dimensional coordinate position in an image feature map. It typically consists of values ​​from multiple channels, representing high-dimensional semantic information such as texture, edges, and shape of a local region of the image. For example, if a feature map has dimensions H×W×C, then the feature point at position (i, j) is a C-dimensional vector. Feature relationships refer to the similarity relationships between these corresponding feature points, which can be measured using distance functions or similarity functions, such as Euclidean distance, cosine similarity, and Manhattan distance. By calculating and organizing these feature relationship values ​​at each spatial location into a two-dimensional matrix, a matching cost map can be obtained. The matching cost map can be understood as a heatmap consistent with the spatial location of the image, reflecting the degree of matching between the feature copy of the target subject and the projected features of the current viewpoint under different extrinsic perturbation conditions. Each feature copy can generate a corresponding matching cost map, thus forming multiple matching cost maps. These can be used in subsequent steps to construct the cost matrix and estimate the optimal extrinsic parameters, i.e., by finding the spatial pattern with the minimum cost, the extrinsic parameters that best match the actual original image can be deduced.

[0057] S103: Based on multiple matching cost maps corresponding to the target subject, determine the cost matrix corresponding to the target subject, and determine the external parameters of the target subject based on the cost matrix.

[0058] In this embodiment of the invention, determining the cost matrix corresponding to the target subject based on multiple matching cost maps corresponding to the target subject includes: performing matrix multiplication of the multiple matching cost maps with the second image features of the target subject to obtain the cost matrix corresponding to the target subject.

[0059] In practical implementation, continuing the example of generating nine feature copies and their corresponding matching cost maps, as described above, such as... Figure 2 As shown, after obtaining multiple matching cost maps, a cost matrix corresponding to the target subject can be further constructed based on the matching cost maps. Specifically, each matching cost map can be first subjected to matrix operations with the second image features. The matching cost map is a set of two-dimensional matrices, each position of which records the similarity or difference value between the first image feature and the second image feature at that position. Its size is consistent with the size of the image feature map space, assumed to be H×W. The second image feature is a three-dimensional tensor of H×W×C, where C represents the channel dimension of each feature point, i.e., the feature description dimension at that position. When constructing the cost matrix, each matching cost map can be regarded as a weight map, and the second image features are weighted and summed. Specifically, each matching cost map can be expanded to the same dimension as the second image feature, that is, the H×W cost map is copied C times to form an H×W×C weight tensor. Then, this tensor and the second image feature can be multiplied channel by channel at each spatial position, and the product is accumulated (or pooled) along the spatial dimension (H×W) to obtain a C-dimensional feature vector. This process can be viewed as a global response of the second image feature map to the feature relationship weights represented by each matching cost map, reflecting the overall feature matching situation under the extrinsic perturbation represented by that matching map. Since there are nine feature copies and nine corresponding matching cost maps, the above operation can be performed on each of these nine matching cost maps, ultimately resulting in nine C-dimensional vectors. Combining these vectors by row or column forms a 9×C cost matrix. Each row of the cost matrix corresponds to a feature copy, i.e., an extrinsic parameter assumption under a specific directional perturbation, while each column corresponds to the matching response of a certain feature dimension in the second image features under that extrinsic parameter perturbation.

[0060] In another possible implementation, the nine matching cost maps can be directly stacked along the new dimension to obtain a cost tensor of shape 𝐻×𝑊×9. Then, high-level features can be extracted using global average pooling, max pooling, or a convolutional network to reduce the dimensionality to a cost matrix. Alternatively, the similarity between feature points of each first image feature and second image feature can be measured along the channel dimension, and these similarities can be normalized or encoded to form a cost vector corresponding to each perturbation, thus combining them into a cost matrix. It should be noted that the above methods for determining the cost matrix are merely illustrative examples of feasible implementations of this invention and do not constitute an improper limitation of the invention. In practical applications, the method can be selected according to the actual situation and needs. This invention does not impose specific limitations on this, but rather focuses on achieving its function.

[0061] In this embodiment of the invention, determining the external parameters of the target subject based on the cost matrix includes: encoding the cost matrix to obtain a high-dimensional feature representation corresponding to the cost matrix; and determining the external parameters corresponding to the target subject based on the high-dimensional feature representation; the external parameters include position parameters and attitude parameters.

[0062] In practical implementation, the cost matrix can first be encoded to transform it into a more discriminative and expressive high-dimensional feature representation. The cost matrix is ​​typically a two-dimensional structure representing the matching error or similarity information between different image locations or feature points. To more effectively extract the geometric relationships implied by this information, the cost matrix can be encoded using a set of convolutional layers, attention mechanisms, or multilayer perceptrons, mapping it to a feature space with a higher semantic level, forming a high-dimensional feature vector or feature map. This high-dimensional representation integrates the matching trends, relative spatial distributions, and matching confidence among all image features. Then, based on this high-dimensional feature representation, the extrinsic parameters of the target subject can be estimated using a regression network or decoding module. Extrinsic parameters can include position parameters and pose parameters; position parameters describe the target subject's position in a three-dimensional world coordinate system, typically represented as three-dimensional coordinate values ​​(x, y, z); pose parameters describe the target subject's rotational state in space, commonly represented by Euler angles, rotation vectors, or quaternions. By mapping high-dimensional features through neural networks, these rotation and translation amounts can be directly predicted, thus obtaining the complete spatial position and orientation of the target subject in the camera coordinate system or world coordinate system.

[0063] According to a second aspect of the embodiments of the present invention, such as Figure 3 As shown, an external parameter calibration device 300 is provided, comprising:

[0064] Translation module 301 is used to acquire multiple feature copies of the original image corresponding to the target subject, apply translation on the feature copies, and obtain the first image feature corresponding to the feature copy;

[0065] The determination module 302 is used to determine the feature relationship between each feature point in the second image feature and the first image feature, and to obtain a matching cost map corresponding to the feature copy; the second image feature is obtained based on the projection depth map corresponding to the target subject.

[0066] The calibration module 303 is used to determine the cost matrix corresponding to the target subject based on multiple matching cost maps corresponding to the target subject, so as to determine the external parameters of the target subject based on the cost matrix.

[0067] Optionally, the translation module 301 is also used for:

[0068] The initial extrinsic parameters of the target body are rotated or translated to generate deviation extrinsic parameters;

[0069] Based on the internal parameters of the target subject involved in the deviation, the point cloud data of the target subject is projected onto a two-dimensional image to obtain the projection depth map corresponding to the target subject, and the second image features corresponding to the target subject are determined based on the projection depth map.

[0070] Optionally, the translation module 301 is specifically used for:

[0071] The number of copies corresponding to the feature copies is determined based on the degree of rotation or translation transformation applied to the initial extrinsic parameters of the target entity.

[0072] The third image features of the original image corresponding to the target subject are copied according to the number of copies, resulting in multiple feature copies of the original image corresponding to the target subject; the third image features are obtained by feature extraction of the original image.

[0073] Optionally, the translation module 301 is specifically used for:

[0074] A preset direction is applied to the feature copy in a two-dimensional space; the translation is a unit translation; the preset direction includes any one of the vertical direction, horizontal direction and combined direction, and the preset direction is different for different feature copies; wherein, the translation in the combined direction includes translating one unit distance in two orthogonal directions respectively;

[0075] For the gaps created after the feature copy is shifted, zero-filling is used to fill them in, thus obtaining the first image feature corresponding to the feature copy.

[0076] Optionally, the calibration module 303 is specifically used for:

[0077] Multiple matching cost maps are multiplied by the second image features of the target subject to obtain the cost matrix corresponding to the target subject.

[0078] Optionally, the calibration module 303 is specifically used for:

[0079] The cost matrix is ​​encoded to obtain a high-dimensional feature representation corresponding to the cost matrix;

[0080] Based on high-dimensional feature representation, the external parameters corresponding to the target are determined; the external parameters include position parameters and attitude parameters.

[0081] According to a third aspect of the present invention, an electronic device for extrinsic parameter calibration is provided, comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the present invention.

[0082] According to a fourth aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method provided in the first aspect of the present invention.

[0083] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0084] Figure 4 An exemplary system architecture 400 is shown that can be applied to the extrinsic parameter calibration method or extrinsic parameter calibration device implemented in this invention.

[0085] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, and 403, a network 404, and a server 405. Network 404 serves as the medium for providing communication links between terminal devices 401, 402, and 403 and server 405. Network 404 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0086] Users can use terminal devices 401, 402, and 403 to interact with server 405 via network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, and 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0087] Terminal devices 401, 402, and 403 can be various electronic devices with displays that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0088] Server 405 can be a server that provides various services, such as a backend management server that supports shopping websites browsed by users using terminal devices 401, 402, and 403 (for example only). The backend management server can process received external parameter calibration requests and feed back the processing results (for example only) to the terminal devices.

[0089] It should be noted that the external parameter calibration method provided in this embodiment of the invention is generally executed by server 405, and correspondingly, the external parameter calibration device is generally set in server 405. The external parameter calibration method provided in this embodiment of the invention can also be executed by terminal devices 401, 402, and 403, and correspondingly, the external parameter calibration device can be set in terminal devices 401, 402, and 403.

[0090] It should be understood that Figure 4 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0091] The following is for reference. Figure 5 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing a terminal device of the present invention. Figure 5 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0092] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0093] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0094] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0095] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0097] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be located in a processor. For example, a processor includes a translation module, a determination module, and a calibration module. The names of these modules do not necessarily limit the module itself. For example, the translation module can also be described as "acquiring multiple feature copies of the original image corresponding to the target subject, applying translation on the feature copies, and obtaining a first image feature module corresponding to the feature copies".

[0098] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, implement the following method: acquiring multiple feature copies of an original image corresponding to a target subject; applying a translation to the feature copies to obtain a first image feature corresponding to the feature copies; determining the feature relationship between a second image feature and each feature point in the first image feature to obtain a matching cost map corresponding to the feature copies; the second image feature is obtained based on a projection depth map corresponding to the target subject; and determining a cost matrix corresponding to the target subject based on the multiple matching cost maps corresponding to the target subject, so as to determine the external parameters of the target subject based on the cost matrix.

[0099] Finally, it should be noted that the above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for calibrating external parameters, characterized in that, include: Multiple feature copies of the original image corresponding to the target subject are obtained, and a translation is applied to the feature copies to obtain the first image feature corresponding to the feature copies; Determine the feature relationships between each feature point in the second image feature and the first image feature to obtain a matching cost map corresponding to the feature copy; the second image feature is obtained based on the projection depth map corresponding to the target subject. Based on the multiple matching cost maps corresponding to the target subject, a cost matrix corresponding to the target subject is determined, and the external parameters of the target subject are determined based on the cost matrix.

2. The method according to claim 1, characterized in that, Before obtaining multiple feature copies of the original image corresponding to the target subject, the process also includes: The initial extrinsic parameters of the target body are subjected to rotation or translation transformation to generate deviation extrinsic parameters; Based on the deviation externally involved in the internal parameters of the target subject, the point cloud data of the target subject is projected onto a two-dimensional image to obtain a projection depth map corresponding to the target subject, so as to determine the second image feature corresponding to the target subject based on the projection depth map.

3. The method according to claim 2, characterized in that, Obtain multiple feature copies of the original image corresponding to the target subject, including: The number of copies corresponding to the feature copy is determined based on the degree of rotation or translation transformation applied to the initial extrinsic parameters of the target body. The third image features of the original image corresponding to the target subject are copied according to the specified number of copies to obtain multiple feature copies of the original image corresponding to the target subject; the third image features are obtained by feature extraction of the original image.

4. The method according to claim 1, characterized in that, Applying a translation to the feature copy to obtain the first image feature corresponding to the feature copy includes: A preset direction is applied to the feature copy in a two-dimensional spatial position; the translation is a unit translation; the preset direction includes any one of the vertical direction, horizontal direction and combined direction, and the preset direction is different for different feature copies; wherein, the translation in the combined direction includes translating one unit distance in two orthogonal directions respectively; For the empty spaces generated after the feature copy is shifted, zero-filling is used to fill them in, so as to obtain the first image feature corresponding to the feature copy.

5. The method according to claim 1, characterized in that, Based on the multiple matching cost maps corresponding to the target entity, a cost matrix corresponding to the target entity is determined, including: The multiple matching cost maps are multiplied by the second image features of the target subject to obtain the cost matrix corresponding to the target subject.

6. The method according to claim 1, characterized in that, Determining the external parameters of the target entity based on the cost matrix includes: The cost matrix is ​​encoded to obtain a high-dimensional feature representation corresponding to the cost matrix; Based on the high-dimensional feature representation, the external parameters corresponding to the target entity are determined; the external parameters include position parameters and attitude parameters.

7. An external parameter calibration device, characterized in that, include: The translation module is used to acquire multiple feature copies of the original image corresponding to the target subject, and apply translation on the feature copies to obtain the first image feature corresponding to the feature copy; The determination module is used to determine the feature relationship between each feature point in the second image feature and the first image feature, and to obtain a matching cost map corresponding to the feature copy; the second image feature is obtained based on the projection depth map corresponding to the target subject. The calibration module is used to determine a cost matrix corresponding to the target subject based on multiple matching cost maps corresponding to the target subject, so as to determine the external parameters of the target subject based on the cost matrix.

8. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images