Calibration method and system of adaptive feature extraction and multi-level matching under degenerative scene

Abstract

The present application belongs to the technical field of multi-sensor fusion and calibration, and provides a calibration method and system of adaptive feature extraction and multi-level matching in a degenerative scene to solve the problem of calibration result invalidation caused by feature matching failure or geometric difference expansion. The calibration method of adaptive feature extraction and multi-level matching in a degenerative scene comprises obtaining images and radar point clouds in the same degenerative scene, and adaptively extracting image features and radar features; corresponding image depth feature descriptors and radar depth feature descriptors are generated; preliminary matching is performed according to similarity, a multi-modal weight adaptive objective function with a space-time transformation matrix as an optimization target is constructed, and an optimized space-time transformation matrix is obtained under the consistency constraint of local geometric structures of the radar and the image; image features and radar features are matched, and the calibration of the images and the radar point clouds in the same degenerative scene is realized, which can realize space-time calibration with strong robustness, high precision and high efficiency in a degenerative scene.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of multi-sensor fusion and calibration, and particularly relates to a calibration method and system for adaptive feature extraction and multi-level matching in a degenerated scene. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] In a complex degenerated scene (such as insufficient light, rainy and foggy weather, or texture missing conditions), the data collected by the camera and radar sensor usually shows feature sparsity, instability and noise interference, making it difficult for traditional calibration methods relying on fixed calibration targets or prior scenes to guarantee accuracy and adaptability.

[0004] Existing calibration methods usually rely on specific calibration templates or scene features, so when processing multi-modal sensor data alignment, the problem of calibration result failure caused by feature matching failure or geometric difference expansion is prone to occur. SUMMARY

[0005] In order to solve the technical problems existing in the background art, the present application provides a calibration method and system for adaptive feature extraction and multi-level matching in a degenerated scene, which can realize robust, high-precision and efficient spatio-temporal calibration in a degenerated scene.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides a calibration method for adaptive feature extraction and multi-level matching in a degenerated scene.

[0008] A calibration method for adaptive feature extraction and multi-level matching in a degenerated scene comprises:

[0009] Obtain an image and a radar point cloud in the same degenerated scene, and extract image features and radar features according to a local feature response function of the image and a local density function of the radar point cloud, in combination with the threshold values corresponding to the local feature response function and the local density function, respectively;

[0010] Use a pre-trained convolutional neural network to process the image features and the radar features correspondingly, to generate image depth feature descriptors and radar depth feature descriptors; and preliminarily match the image depth feature descriptors and the radar depth feature descriptors according to their similarity, to obtain candidate matching pairs;

[0011] Based on the candidate matching pair and the predefined space-time transformation matrix, a multi-modal weight adaptive objective function with the space-time transformation matrix as the optimization target is constructed, and under the consistency constraint of the local geometric structure of the radar and the image, an optimized space-time transformation matrix is obtained.

[0012] Based on the optimized space-time transformation matrix, the image features and the radar features are matched to realize the calibration of the image and the radar point cloud under the same degenerative scene.

[0013] The second aspect of the application provides a calibration system for adaptive feature extraction and multi-level matching under a degenerative scene.

[0014] A calibration system for adaptive feature extraction and multi-level matching under a degenerative scene comprises:

[0015] An adaptive feature extraction module is configured to acquire an image and a radar point cloud under the same degenerative scene, extract image features and radar features according to a local feature response function of the image and a local density function of the radar point cloud, and combine the threshold values corresponding to the local feature response function and the local density function, respectively.

[0016] A preliminary matching module is configured to use a pre-trained convolutional neural network to process the image features and the radar features, generate image depth feature descriptors and radar depth feature descriptors, and perform preliminary matching according to the similarity between the image depth feature descriptors and the radar depth feature descriptors to obtain a candidate matching pair.

[0017] A space-time transformation matrix optimization module is configured to construct a multi-modal weight adaptive objective function with the space-time transformation matrix as the optimization target based on the candidate matching pair and the predefined space-time transformation matrix, and obtain an optimized space-time transformation matrix under the consistency constraint of the local geometric structure of the radar and the image.

[0018] A multi-modal calibration module is configured to match the image features and the radar features based on the optimized space-time transformation matrix to realize the calibration of the image and the radar point cloud under the same degenerative scene.

[0019] The third aspect of the application provides a computer-readable storage medium.

[0020] A computer-readable storage medium stores a computer program, and the program is executed by a processor to realize the steps in the calibration method for adaptive feature extraction and multi-level matching under a degenerative scene.

[0021] The fourth aspect of the application provides a computer program product.

[0022] A computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the method for adaptive feature extraction and multi-level matching in a degenerated scene as described above.

[0023] A fifth aspect of the application provides an electronic device.

[0024] An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method for adaptive feature extraction and multi-level matching in a degenerated scene as described above when executing the program.

[0025] Compared with the prior art, the application has the following beneficial effects:

[0026] (1) According to the local feature response function of the image and the local density function of the radar point cloud, and in combination with the threshold values corresponding to the local feature response function and the local density function respectively, the image features and the radar features are extracted, without relying on fixed calibration targets, and the effective features in the image and radar data can be automatically screened, thereby improving the data utilization rate in a degenerated scene.

[0027] (2) The image features and the radar features are processed by using a convolutional neural network, high-dimensional descriptors are generated, the interoperability between different modal features is enhanced, and in combination with a hierarchical matching mechanism, noise and geometric differences are gradually eliminated, and global alignment from coarse to fine is realized.

[0028] (3) A multi-modal weight adaptive objective function with a spatio-temporal transformation matrix as an optimization target, under the consistency constraint of the local geometric structures of the radar and the image, adaptively accelerates the optimization and solution of the spatio-temporal transformation matrix, guarantees real-time and high-precision calibration, provides effective support for multi-sensor fusion in a complex environment, and can be widely applied to multi-sensor data fusion in the fields of autonomous driving, robot navigation, intelligent monitoring, and the like.

[0029] The advantages of the additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be learned through the practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings, which form a part of the specification, are included to provide a further understanding of the application and are incorporated herein by reference. The illustrations are shown for the purpose of enabling those skilled in the art to implement the application and are not intended to limit the present application unduly.

[0031] Figure 1 is a flowchart of the method for adaptive feature extraction and multi-level matching in a degenerated scene according to an embodiment of the application;

[0032] Figure 2is a calibration system structure diagram of adaptive feature extraction and multi-level matching under a degradation scene of an embodiment of the present application.

[0033] Figure 3 is a calibration result diagram of adaptive feature extraction and multi-level matching under a degradation scene of an embodiment of the present application. DETAILED DESCRIPTION

[0034] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed description is illustrative only, and is intended to provide further description of the present application. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0036] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments in accordance with the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, components, and / or groups thereof, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0037] Embodiment One

[0038] Figure 1 A calibration method flow chart of adaptive feature extraction and multi-level matching under a degradation scene of an embodiment of the present application is given. In combination with Figure 1 , the calibration method of adaptive feature extraction and multi-level matching under a degradation scene of an embodiment of the present application comprises:

[0039] S101: Obtain images and radar point clouds under the same degradation scene, and according to a local feature response function of the images and a local density function of the radar point clouds, in combination with threshold values corresponding to the local feature response function and the local density function respectively, extract image features and radar features.

[0040] Specifically, according to the local feature response function of the camera image, filter out pixel points corresponding to local feature response function values greater than or equal to a first preset threshold value as camera image features; and according to the local density function of the radar point cloud, filter out points corresponding to local density function values greater than or equal to a second preset threshold value as radar features.

[0041] In the degradation scene, both the images and the radar data have the problem of scarce feature information. This embodiment does not need prior targets for adaptive feature extraction, and respectively performs feature filtering on the images and the radar point clouds, thereby ensuring sufficient representation of effective geometric and texture information.

[0042] The process of image feature extraction is as follows:

[0043] Let I(x) and I(y) be respectively x and y The pixel at the location; To quantify local texture and geometric changes, a local feature response function is defined: ;

[0044] in, Indicated by x The neighborhood set centered on; Represents the neighborhood centered at x; The scale parameters are used to control the influence range of the neighborhood; I(x) and I(y) are respectively x and y The pixel at the location; Use Gaussian weights.

[0045] In degraded scenes (such as low light, noise interference, and scarce textures), fixed It cannot adapt well to the feature distribution of different regions. High-texture regions require less detail capture, while smooth regions require greater enhancement of response stability. The following strategy is set to dynamically adjust it to adapt to the local image feature complexity.

[0046] Image gradient intensity Adjusting using gradient strength :

[0047] ;

[0048] in, It is the initial value; Gradient pairs The influence coefficient; The rate of change of a pixel in the horizontal direction. The rate of change of a pixel in the vertical direction; This represents the gradient intensity of the image.

[0049] Meanwhile, a fixed-size neighborhood It cannot adapt to scale variations in local textures within an image. For example, a larger neighborhood is needed to capture information in sparsely textured regions, while a smaller neighborhood is needed in detailed regions to avoid over-smoothing. Let the neighborhood range be... Therefore Centered on, with radius In a local area, ;in It is a dynamically adjusted neighborhood radius.

[0050] By adjusting The neighborhood range can be dynamically changed. The size of the number of pixels or feature points contained in the neighborhood affects the number of pixels or feature points contained in the neighborhood.

[0051] Based on local gradient variance Adjustment of local gradient variance The formula for measuring the degree of gradient change within a region is: ;

[0052] in, It is a location The gradient value. It is the mean of the gradient within the neighborhood.

[0053] The neighborhood radius is dynamically adjusted as follows: ;

[0054] in, It is the initial neighborhood radius; It is the coefficient of influence of the gradient on the neighborhood radius.

[0055] pass The calculation, It shrinks when the gradient changes significantly (such as in areas with rich texture) and increases when the gradient changes slightly (such as in areas with smooth texture).

[0056] Finally, based on the response function Size, choose the one that satisfies: The pixels are used as image features, where are The set threshold.

[0057] Based on pixel value differences To measure local intensity changes, since the application scenario is in degraded images (low light, noise, or blur), the absolute value term is used to enhance robustness to feature points, while Gaussian weights are added. This is used to measure the distance influence of neighboring pixels on the center point. Essentially, it combines a dual weighting of intensity difference and spatial location. Dynamically adjusting σ(x) enhances the formula's adaptability to different texture regions, especially its robustness in degraded scenes. Dynamically adjusting the range of N(x) enables the algorithm to effectively extract features in both dense and sparse texture scenes. Incorporating gradients as an adjustment criterion further enhances the formula's adaptability to various application scenarios.

[0058] The process of radar point cloud feature extraction is as follows:

[0059] Let the input radar point cloud be , in Represents the i-th radar point cloud;

[0060] Considering the local density of radar data, a local density function is defined:

[0061] ;

[0062] ;

[0063] wherein, is a set of fields centered at with radius r; is a scale parameter; g() is a weighting function of the reflectivity; denotes the reflectivity of the radar point cloud , denotes the gradient of the reflectivity in the horizontal direction, denotes the gradient of the reflectivity in the vertical direction, y direction, and Z is a normalization coefficient.

[0064] By introducing the weighting function of the reflectivity, the extended feature is a multi-modal joint density function, which enhances the comprehensive representation of the local attributes of the point cloud, improves the discrimination and robustness of feature matching, and adapts to complex degradation scenarios. Similarly, the points that satisfy are selected as radar features, wherein is a preset threshold.

[0065] The above image and radar feature extraction formulas realize adaptive filtering of noise and only retain points with significant changes or density in the local, thereby improving the stability and matchability of the features.

[0066] S102: Utilize the pre-trained convolutional neural network to process the image features and radar features, respectively, to generate image depth feature descriptors and radar depth feature descriptors; preliminarily match according to the similarity between the image depth feature descriptors and the radar depth feature descriptors to obtain candidate matching pairs .

[0067] The pre-trained convolutional neural network of the present embodiment is a deep learning model based on a cross-modal feature sharing architecture, which is used to generate high-dimensional feature descriptors to ensure that the features can fully represent local structures, semantic information, and inter-modal correlations, and ensure that the features can fully represent local structures and semantic information.

[0068] For each extracted feature contains image or radar, define the descriptor generation function:

[0069] ;

[0070] wherein, denotes the pre-trained convolutional neural network; is the image feature and the radar feature; is the convolutional neural network parameter; is the generated d-dimensional descriptor vector.

[0071] In the training process, a cross-modal feature alignment loss function is designed , by minimizing the distribution difference of different modal features in the shared feature space, to ensure the consistency of the feature descriptors:

[0072] ;

[0073] wherein, is a feature embedding network, is a set of image and radar matching pairs. At the same time, in order to enhance the discriminability of the descriptors, a constraint based on the triplet loss is introduced :

[0074] ;

[0075] wherein, respectively represent the anchor feature, the positive sample feature and the negative sample feature, is the distance boundary threshold, which is a preset constant.

[0076] In an embodiment, the Euclidean distance is used as the descriptor similarity measure in the matching process: ; when is less than a set threshold, it is considered that the feature has a preliminary corresponding relationship with the feature , and it is saved as a candidate matching pair as a low-level preliminary matching. By introducing the cross-modal feature alignment loss and the discriminability constraint mechanism, the generated descriptors can not only extract local texture and geometric features, but also capture implicit correlation information between modalities, thereby improving the robustness and matching accuracy under different modalities. In the generation of deep feature descriptors, by introducing a cross-modal feature sharing architecture, the feature alignment constraint mechanism is used to reduce the difference between the feature distributions of different modalities, and the discriminability of the descriptors is improved by combining the triplet loss, thereby realizing the modal-independent high-dimensional feature embedding, enhancing the consistency and robustness of the features, and effectively adapting to the multi-modal calibration task in complex degenerative scenes.

[0077] S103: Based on the candidate matching pairs and the pre-defined space-time transformation matrix, a multi-modal weight adaptive objective function is constructed with the space-time transformation matrix as the optimization objective, and under the consistency constraint of the local geometric structures of the radar and the image, an optimized space-time transformation matrix is obtained.

[0078] Specifically, the space-time transformation matrix T (which can usually be decomposed into a rotation matrix R and a translation vector t to realize fine alignment of the two modal data. At the same time, a multi-modal weight adaptive mechanism is introduced, and the importance weights of different modal features are defined, which are integrated into the multi-modal weight adaptive objective function :

[0079] ;

[0080] According to the modal source of the feature (such as image or radar) and the matching confidence dynamic adjustment, the formula is: ;

[0081] is the adjustment parameter, represents the matching confidence of the current feature (such as the descriptor similarity score).

[0082] Secondly, the inter-modal geometric structure constraint is established:

[0083] According to the geometric characteristics of radar and image, the inter-modal geometric consistency constraint is added:

[0084] ;

[0085] wherein, and are the local geometric centers of radar and image respectively. The final optimization target combines the geometric consistency: ; wherein, is the optimized multi-modal weight adaptive target function; is the trade-off parameter.

[0086] Finally, the dynamic adjustment of the robust loss function is established:

[0087] ;

[0088] wherein, is dynamically adjusted with the number of iteration steps to adapt to the distribution characteristics of the matching pairs, and the initial value is larger to tolerate more outliers, and gradually converges later to fine optimization, so as to generate a time-space transformation matrix initial value close to the true value.

[0089] In this embodiment, a multi-modal weight adaptive mechanism is introduced, which dynamically adjusts the modal weight according to the feature source and the matching confidence, so that the feature optimization contribution proportion of different modalities is more reasonable, and the matching accuracy is significantly improved. Secondly, by embedding the consistency constraint of the local geometric structure of radar and image in the optimization target, the geometric information of multi-modal data is fully utilized, which makes up for the limitations of relying solely on feature point matching pairs, and further improves the accuracy of modal alignment. Finally, the robust loss function is dynamically adjusted, which gradually weakens the influence of outliers in the optimization process, allows more fault tolerance range in the early stage, and fine-tunes the error in the later stage, to ensure the stability and global convergence effect of the optimization process.

[0090] To further improve the real-time performance of the calibration process, an adaptive acceleration algorithm based on a nonlinear optimization framework is used to solve the space-time transformation parameters. Let the transformation matrix T be represented by a parameter vector , that is, represented as , and the optimization function of the space-time transformation parameters is denoted as: ;

[0091] the intermediate parameters ; are the image depth feature descriptors and radar depth feature descriptors in the candidate matching pair, respectively; is a dynamically adjusted robust loss function; N is the number of candidate matching pairs.

[0092] The Levenberg-Marquardt algorithm is used to iteratively optimize the parameter vector , and the update formula is:

[0093]

[0094] where J is the residual vector , and the Jacobian matrix of ; I is the identity matrix, and the damping factor is dynamically adjusted at each step of the iteration process, taking into account the advantages of gradient descent and Newton's method, accelerating convergence while ensuring the stability of the solution, thereby achieving fast and accurate space-time calibration. The subscript k of is the kth iteration.

[0095] S204: Based on the optimized space-time transformation matrix, the image features and radar features are matched to realize the calibration of the image and radar point cloud in the same degenerate scene, and the calibration results are as shown in (a) of Figure 3 , (d) of Figure 3 . By projecting the lidar point cloud onto the camera image plane, the alignment effect of the two can be observed intuitively. The point cloud projection result and the target contour in the image are highly coincident in spatial position, especially in the edge and contour regions of the object, the projection points and the boundaries of the object in the image are consistent, which indicates that the mapping relationship between the two in the geometric space is accurate. When there is a significant depth change at the boundary of the vehicle or object, the distribution of laser points on the image also clearly outlines these changes, which closely corresponds to the structural lines in the image, thereby providing a more intuitive alignment basis.

[0096] The degenerate scene adaptive feature extraction and multi-level matching calibration method of the application has high robustness, strong adaptability, and excellent real-time performance, and is a fast space-time calibration method, which can be widely applied in multi-sensor data fusion in the fields of autonomous driving, robot navigation, intelligent monitoring, etc.

[0097] Embodiment Two

[0098] As shown in the figure, the calibration system of adaptive feature extraction and multi-level matching under the degenerative scene of the embodiment of the application comprises: Figure 2

[0099] The adaptive feature extraction module 201 is configured to acquire the image and the radar point cloud under the same degenerative scene, extract the image feature and the radar feature according to the local feature response function of the image and the local density function of the radar point cloud, and combine the threshold values corresponding to the local feature response function and the local density function, respectively.

[0100] The preliminary matching module 202 is configured to perform corresponding processing on the image feature and the radar feature by using a pre-trained convolutional neural network, generate an image depth feature descriptor and a radar depth feature descriptor, and perform preliminary matching according to the similarity between the image depth feature descriptor and the radar depth feature descriptor to obtain a candidate matching pair.

[0101] The spatio-temporal transformation matrix optimization module 203 is configured to construct a multi-modal weight adaptive objective function with the spatio-temporal transformation matrix as an optimization target based on the candidate matching pair and a predefined spatio-temporal transformation matrix, and obtain an optimized spatio-temporal transformation matrix under the consistency constraint of the local geometric structures of the radar and the image.

[0102] The multi-modal calibration module 204 is configured to match the image feature and the radar feature based on the optimized spatio-temporal transformation matrix to realize the calibration of the image and the radar point cloud under the same degenerative scene.

[0103] It should be noted that each module of the embodiment of the application corresponds to each step in the above embodiment one by one, and the specific implementation process is the same, which will not be described in detail here.

[0104] Embodiment Three

[0105] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene as described above.

[0106] Embodiment Four

[0107] A computer program product comprises computer programs / instructions, which are executed by a processor to realize the steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene as described above.

[0108] Embodiment Five

[0109] ​The embodiment provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps in the calibration method of adaptive feature extraction and multi-level matching in a degradation scenario when executing the program.

[0110] The electronic device provided by the embodiment comprises a central processing unit (CPU) which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage part to a random access memory (RAM). In the RAM, various programs and data required for system operation are also stored. The central processing unit, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0111] The following components are connected to the I / O interface: an input part comprising a keyboard, a mouse, and the like; an output part comprising a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part comprising a hard disk, and the like; and a communication part comprising a network interface card such as a local area network (LAN) card, a modem, and the like. The communication part performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as needed. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive as needed, so that a computer program read therefrom is installed in the storage part as needed.

[0112] In particular, the processes described above with reference to the flowcharts can be implemented as a computer software program in accordance with embodiments of the present application. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via the communication part, and / or installed from a removable medium. When the computer program is executed by the central processing unit, various functions defined in the apparatus of the present application are executed.

[0113] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (system) and computer program products of embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one flow or multiple flows and / or blocks Figure 1means for performing the function specified in the block or blocks.

[0114] The above merely provides the preferred embodiment of the present application, and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of protection of the present application.

Claims

1. A calibration method of adaptive feature extraction and multi-level matching under degenerative scenes, characterized in that, The method comprises the following steps: obtaining images and radar point clouds under the same degenerative scene, extracting image features and radar features according to a local feature response function of the images and a local density function of the radar point clouds, and combining the local feature response function and the local density function with thresholds corresponding to the local feature response function and the local density function respectively; The image features and the radar features are processed by using a pre-trained convolutional neural network to generate image depth feature descriptors and radar depth feature descriptors; preliminary matching is performed according to the similarity between the image depth feature descriptors and the radar depth feature descriptors to obtain candidate matching pairs; wherein the expression of the image depth feature descriptors and the radar depth feature descriptors is: for each extracted feature, whether it is from an image or a radar, a descriptor generation function is defined as follows: ; represents a pre-trained convolutional neural network; is an image feature and a radar feature; is a convolutional neural network parameter; is a generated d-dimensional descriptor vector; based on the candidate matching pairs and a predefined space-time transformation matrix, constructing a multi-modal weight adaptive objective function with the space-time transformation matrix as the optimization objective, and obtaining an optimized space-time transformation matrix under the consistency constraint of local geometric structures of the radar and the images; based on the optimized space-time transformation matrix, matching the image features and the radar features to realize the calibration of the images and the radar point clouds under the same degenerative scene; wherein the multi-modal weight adaptive objective function with the space-time transformation matrix as the optimization objective is: ； ； ； is a multi-modal weight adaptive target function; is a weight coefficient which dynamically adjusts according to the modal source of the feature and the matching confidence; N is the number of candidate matching pairs; T is a space-time transformation matrix; is an adjustment parameter, represents the matching confidence of the current feature; is a dynamically adjusted robust loss function; respectively represent the image depth feature descriptor and the radar depth feature descriptor in the candidate matching pair; x represents the position of the image pixel; is a scale parameter.

2. The method of claim 1, wherein the method further comprises: Local feature response function of an image is: ； wherein, denotes a neighborhood set centered at x ; denotes a neighborhood centered at x; is a scale parameter to control the influence range of the neighborhood; I(x) and I(y) are the pixels at x and y positions, respectively; is a Gaussian weight.

3. The method of claim 2, wherein the method further comprises: Adjusting a scale parameter of a neighborhood influence range using gradient intensity of an image : ； ； wherein: is an initial value; is an influence coefficient of the gradient pair ; is a rate of change of the pixel in the horizontal direction, is a rate of change of the pixel in the vertical direction; is a gradient intensity of the image.

4. The method of claim 1, wherein the method further comprises: the expression of the local density function of the radar point clouds is: ； wherein, represents the i-th radar point cloud; represents the j-th radar point cloud; is a set of regions centered at with radius r; is a scale parameter; g() is a weighting function of the reflectance intensity; is a local density function of the radar point cloud; represents the reflectance intensity of the radar point cloud .

5. A calibration system of adaptive feature extraction and multi-level matching under degenerated scenes, characterized in that, The steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene according to any one of claims 1-4 are realized, comprising: an adaptive feature extraction module for obtaining images and radar point clouds under the same degenerative scene, extracting image features and radar features according to a local feature response function of the images and a local density function of the radar point clouds, and combining the local feature response function and the local density function with thresholds corresponding to the local feature response function and the local density function respectively; a preliminary matching module for processing the image features and the radar features by using a pre-trained convolutional neural network to correspondingly generate image depth feature descriptors and radar depth feature descriptors, and performing preliminary matching according to the similarity between the image depth feature descriptors and the radar depth feature descriptors to obtain candidate matching pairs; a space-time transformation matrix optimization module for constructing a multi-modal weight adaptive objective function with the space-time transformation matrix as the optimization objective based on the candidate matching pairs and a predefined space-time transformation matrix, and obtaining an optimized space-time transformation matrix under the consistency constraint of local geometric structures of the radar and the images; a multi-modal calibration module for matching the image features and the radar features based on the optimized space-time transformation matrix to realize the calibration of the images and the radar point clouds under the same degenerative scene.

6. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to realize the steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene according to any one of claims 1-4.

7. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to realize the steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene according to any one of claims 1-4.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to realize the steps in the calibration method of adaptive feature extraction and multi-level matching under the degenerative scene according to any one of claims 1-4.

Images