An adaptive composition method and system based on feature fusion

Abstract

The present invention relates to the field of image processing technology, and in particular to an adaptive composition method and system based on feature fusion. The method of the present invention comprises the following steps: acquiring a real-time image around a vehicle, extracting similar areas of the real-time image, and then obtaining feature points of the similar areas; performing three-level pairing using the feature points to obtain an adaptive composition feature point set; obtaining a first transformation model and a second transformation model based on the adaptive composition feature point set; fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle. The present invention obtains a panoramic image of the vehicle by extracting feature points of similar areas, performing three-level pairing on the feature points, and then performing two projection transformations, and finally performing image fusion on the projection results, thereby obtaining the panoramic image of the vehicle. This solves the problem of poor uniformity of the scale of the composition of the 360-degree panoramic image of the vehicle and poor overall quality of the panoramic image caused by blurred boundaries and ghosting.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an adaptive composition method and system based on feature fusion. Background Art

[0002] Digital images are the main source of visual image information for human analysis, and they play an extremely important role in people's daily lives. For example, the on-board panoramic assistance system uses four cameras installed on the front, back, left and right sides of the vehicle to collect images around the vehicle at the same time. After distortion restoration, perspective conversion, image stitching and image enhancement, it finally forms a seamless 360-degree panoramic bird's-eye view of the vehicle's surroundings, eliminating driving blind spots and greatly improving the driver's control of the vehicle's surroundings. However, in complex driving environments, this 360-degree panoramic image still has problems in composition synthesis, such as poor uniformity of composition scale and blurred boundaries and ghosting, which lead to poor overall quality of the panoramic image. Therefore, further improving the image quality of 360-degree panoramic images is an issue that needs to be solved urgently. Summary of the invention

[0003] In view of the shortcomings of the existing methods and the needs of practical applications, in order to solve the problem that the 360-degree panoramic bird's-eye view still has poor uniformity of composition scale and blurred boundary ghosting in the composition of the complex driving environment, the overall quality of the panoramic image is poor. On the one hand, the present invention provides an adaptive composition method based on feature fusion, which comprises the following steps: acquiring a real-time image around the vehicle, extracting similar areas of the real-time image, and then obtaining feature points of the similar areas; performing three-level pairing using the feature points to obtain an adaptive composition feature point set; obtaining a first transformation model and a second transformation model according to the adaptive composition feature point set; fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle. The present invention reduces the number of feature point extractions by extracting feature points of similar areas and performing three-level pairing on the feature points, improves the matching efficiency and accuracy of feature points, and is conducive to improving the speed and efficiency of image fusion. Secondly, the pairing results are projected twice, and then the projection results are image fused to obtain a panoramic image of the vehicle, which solves the problem that the 360-degree panoramic image of the vehicle has poor uniformity of composition scale and blurred boundary ghosting, resulting in poor overall quality of the panoramic image.

[0004] Optionally, the extracting the similar area of ​​the real-time image comprises the following steps:

[0005] Preliminarily determine the position of the real-time image in the vehicle panoramic image, and set a template image according to the determination result; use the template image to obtain a similarity determination value of the real-time image; and determine a similar area of ​​the real-time image according to the similarity determination value. The present invention obtains a similar area of ​​the real-time image according to the similarity determination value after preliminarily determining the position of the real-time image to be fused in the final vehicle panoramic image, thereby avoiding the problem of global search for feature points in subsequent steps and further improving the fusion speed of the present invention.

[0006] Optionally, the step of performing three-level pairing using the feature points to obtain an adaptive composition feature point set comprises the following steps:

[0007] Calculate the similarity distance of the feature points, perform first-level pairing according to the similarity distance, and obtain a first-level feature point set; construct a second-level pairing model, combine the second-level pairing model and the first-level feature point set, perform second-level pairing, and obtain a second-level feature point set; introduce a grid edge distance parameter to improve the grid motion statistical matching algorithm, and correct the second-level feature point set through the improved grid motion statistical matching algorithm to obtain the adaptive composition feature point set. The present invention ensures the quality and speed of feature point matching through three-level pairing, and further improves the composition quality of the present invention.

[0008] Optionally, the secondary pairing model satisfies the following formula:

[0009]

[0010] in, Represents a collection, Indicates The spatial similarity of feature points, Indicates In real-time image The horizontal coordinate of the feature point, Indicates In real-time image The vertical coordinate of the feature point, Indicates the number of feature point pairs in the first-level feature point set. The present invention can concretize the spatial similarity constraint through the secondary pairing model, which is conducive to improving the screening rate of correct feature point matching pairs.

[0011] Optionally, the grid edge distance parameter satisfies the following formula:

[0012]

[0013] in, represents the grid edge distance, Indicates the probability that the feature point is correctly matched, Represents the length of the grid, Indicates the width of the grid. The grid edge distance parameter introduced in the present invention uses the probability of feature points being correctly matched to adaptively adjust the grid edge distance parameter, thereby improving the adaptive ability of feature point matching, further facilitating the solution of the boundary fuzziness problem, and improving the smoothness of the final composition.

[0014] Optionally, obtaining the first transformation model and the second transformation model according to the adaptive composition feature point set comprises the following steps:

[0015] According to the adaptive composition feature point set, a homography transformation model is obtained; through the homography transformation model, a first transformation model and a second transformation model are obtained. The present invention performs two transformation projections on the adaptive composition feature point set, effectively solving the problems of poor uniformity of composition scale and composition ghosting.

[0016] Optionally, the homography transformation model obtained according to the adaptive composition feature point set satisfies the following formula:

[0017]

[0018]

[0019] ,in, 、 Represents the feature points The horizontal and vertical coordinates of 、 Represents the feature points The horizontal and vertical coordinates of represents the homography matrix, represents a constant;

[0020] The first transformation model and the second transformation model are obtained by the homography transformation model, satisfying the following formula:

[0021]

[0022] in, represents the first transformation model, Indicates the maximum area of ​​the figure formed by the feature points, represents the area of ​​similar regions, represents the similarity matrix;

[0023]

[0024] in, Represents the second transformation model. The present invention uses the homography transformation model and the first transformation model obtained by the similarity matrix to perform the first projection change on the image to be fused, constructs the weight coefficient according to the maximum area of ​​the figure formed by the feature points and the area of ​​the similar area to improve the smoothness of the projection, and then uses the homography transformation model and the second transformation model obtained by the first transformation model to perform the first projection change on another image to be fused, further improves the alignment, and is conducive to improving the projection fusion speed and quality of the image to be fused with the projection change.

[0025] Optionally, fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle includes the following steps:

[0026] Based on the first transformation model and the second transformation model, a first projection image of the real-time image is obtained; low-frequency components and high-frequency components of the first projection image are obtained by wavelet decomposition; an image fusion model is constructed, and the low-frequency components and the high-frequency components are respectively fused by the image fusion model; based on inverse wavelet decomposition, a panoramic image of the vehicle is obtained according to the fused low-frequency components and high-frequency components. The present invention utilizes wavelet decomposition and inverse decomposition to fuse images that have undergone the first projection change, unify the composition scale, and improve the efficiency of fusion composition.

[0027] Optionally, the image fusion model satisfies the following formula:

[0028]

[0029] in, represents the fused component, represents the fusion coefficient, , represents the width of the similarity region, Indicates the distance between the pixel of the reference image and the left boundary of the fusion area. and The present invention uses the fusion coefficients obtained from similar regions to adjust the image fusion weights, thereby improving the composition adaptability and image resolution, and further improving the natural coordination and quality of the final composition.

[0030] In the second aspect, in order to efficiently execute the adaptive composition method based on feature fusion provided by the present invention, the present invention also provides an adaptive composition system based on feature fusion, including a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the adaptive composition method based on feature fusion as described in the first aspect of the present invention. The adaptive composition system based on feature fusion of the present invention has a compact structure and stable performance, and can stably execute the adaptive composition method based on feature fusion provided by the present invention, further improving the overall applicability and practical application ability of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 A flow chart of an adaptive composition method based on feature fusion provided by an embodiment of the present invention;

[0032] Figure 2 A framework diagram of an adaptive composition system based on feature fusion provided by an embodiment of the present invention;

[0033] Figure 3 A schematic diagram of the structure of an adaptive patterning device based on feature fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not need to be adopted to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, software or methods are not specifically described.

[0035] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or subcombination. In addition, it should be understood by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale.

[0036] See also Figure 1 , Figure 1 A flow chart of an adaptive composition method based on feature fusion provided by an embodiment of the present invention. To solve the problem, the present invention provides an adaptive composition method based on feature fusion, such as Figure 1 As shown, the method comprises the following steps:

[0037] S1. Acquire a real-time image around a vehicle, extract a similar area of ​​the real-time image, and then obtain feature points of the similar area.

[0038] In an embodiment, four 180-degree wide-angle cameras installed at the front, rear, left, and right sides of the vehicle are used to obtain real-time images around the vehicle, and four frames of images at the same moment are treated as a group for subsequent processing.

[0039] Furthermore, after acquiring the real-time image around the vehicle, the real-time image is also subjected to distortion restoration processing and image noise reduction processing. The original image is corrected through the distortion restoration processing to obtain an image without geometric distortion, which helps to improve the quality of the original image and make the image clearer and more realistic.

[0040] It is understandable that images taken with a camera have more or less noise points. If the noise is not suppressed, the noise points may be mistaken for feature points when extracting image feature points, thus affecting the accuracy of image registration. Image denoising mainly includes two types: spatial domain filtering denoising and transform domain filtering denoising. Spatial domain filtering denoising directly operates on image pixels, mainly including mean filtering, median filtering and Gaussian filtering. Transform domain filtering denoising converts the image in the spatial domain to the transform domain, and reduces noise by adjusting the coefficients in the transform domain. It is an indirect denoising method. The main methods include Fourier transform, cosine transform and wavelet transform.

[0041] In the embodiment, the step S1 of extracting the similar area of ​​the real-time image comprises the following steps:

[0042] S11. Preliminarily determine the position of the real-time image in the vehicle panoramic image, and set a template image according to the determination result.

[0043] The real-time images provided by the front camera and the right camera must be fused in such a way that the right part of the front camera image and the left part of the right camera image overlap. The template image can be set to different-sized images of the right part of the front camera image and the left part of the right camera image. Similarly, the template images between other cameras can be obtained.

[0044] S12: using the template image to obtain a similarity judgment value of the real-time image.

[0045] The template image is used to traverse another camera image to obtain a similarity judgment value between the two real-time images. Specifically, one tenth of the right side of the front camera image is set as the template image, and the template image is used to traverse the right camera image, and then one tenth of the left side of the right camera image is set as the template image, and the template image is used to traverse the front camera image to obtain a similarity judgment value between the front camera image and the right camera image; similarly, similarity judgment values ​​between other cameras are obtained.

[0046] Furthermore, the template image is used to obtain a similarity judgment value of the real-time image, which satisfies the following formula:

[0047]

[0048] in, Represents the similarity judgment value of the real-time image, Respectively represent the image grayscale values ​​of the corresponding area positions of the template image and another camera image, The similarity distribution function represents the position of the corresponding area between the template image and another camera image. In this embodiment, the similarity distribution function is the Euclidean distance. In other embodiments, the Manhattan distance, Chebyshev distance, angle cosine, and Michaelis distance can also be used as the similarity distribution function.

[0049] S13: Determine a similar area of ​​the real-time image according to the similarity judgment value.

[0050] To determine the similar area of ​​the real-time image through the similarity judgment value, a similarity judgment threshold needs to be set. When the similarity judgment value is greater than the similarity judgment threshold, the template image at this time is the similar area of ​​the real-time image.

[0051] Furthermore, the similarity judgment threshold may be set to 70% of the similarity judgment value calculated between the template image and itself.

[0052] In the embodiment, the feature points of the similar area obtained in step S1 can be extracted using the SURF (Speed-Up Robust Feature) algorithm. The SURF algorithm performs appropriate simplification and approximation while ensuring correctness, and uses integral images for calculation multiple times. The integral image refers to the value at a point defined as the sum of all pixel values ​​in the rectangular area from the upper left corner point of the original image to the original image point.

[0053] In some other embodiments, SIFT (Scale-Invariant Feature Transform) algorithm, ORB (Oriented FAST and Rotated BRIEF) algorithm, Harris corner detection algorithm, Hessian-Laplace algorithm and convolutional neural network (CNN) can also be used to extract feature points of similar areas. SIFT algorithm includes constructing DOG (Difference of Gaussians) scale space to simulate the multi-scale features of image data, which can maintain invariance to rotation, scale scaling, brightness change, and maintain a certain degree of stability to perspective change, affine transformation, and noise; ORB algorithm uses FAST corner detection algorithm to detect image corners and uses BRIEF algorithm to describe feature points; Harris corner detection algorithm extracts corner feature points by calculating the corner response function of each point in the image; Hessian-Laplace algorithm detects local extreme points of the image by calculating the Hessian matrix of the image, and then uses Laplace operator to extract feature points; convolutional neural network extracts spatial features of data through convolution operation.

[0054] S2. Perform three-level pairing using the feature points to obtain an adaptive composition feature point set.

[0055] The step S2 of performing three-level pairing using the feature points to obtain an adaptive composition feature point set includes the following steps:

[0056] S21, calculating the similarity distance of the feature points, performing first-level pairing according to the similarity distance, and obtaining a first-level feature point set.

[0057] In the embodiment, the Euclidean distance is used as the similarity distance to perform the first-level pairing and obtain the first-level feature point set.

[0058] S22: construct a secondary pairing model, and perform a second-level pairing in combination with the secondary pairing model and the first-level feature point set to obtain a second-level feature point set.

[0059] In an embodiment, the constructed secondary pairing model satisfies the following formula:

[0060]

[0061] in, Represents a collection, Indicates The spatial similarity of feature points, Indicates In real-time image The horizontal coordinate of the feature point, Indicates In real-time image The vertical coordinate of the feature point, Indicates the number of feature point pairs in the first-level feature point set.

[0062] Statistics of all feature points After that, the points are arranged from large to small, and the median is used as the spatial similarity constraint. The second-level feature point set is obtained by screening according to the spatial similarity constraint and the actual situation and requirements. In this embodiment, the feature points within 0.1 above and below the spatial similarity constraint are considered to meet the requirements and are included in the second-level feature point set.

[0063] S23, introducing a grid edge distance parameter to improve the grid motion statistical matching algorithm, and using the improved grid motion statistical matching algorithm to correct the second-level feature point set to obtain the adaptive composition feature point set.

[0064] The grid edge distance parameter introduced in step S23 satisfies the following formula:

[0065]

[0066] in, represents the grid edge distance, Indicates the probability that the feature point is correctly matched, Represents the length of the grid, Indicates the width of the grid. Indicates the number of grids.

[0067] Furthermore, the support points on the edge of the grid are distributed to multiple surrounding neighborhood grids, so that the score of the correct matching point will be increased, thereby widening the distance from the score of the incorrect matching point, increasing the distinguishability of correct matches and incorrect matches, and then correcting the second-level feature point set through the improved grid motion statistical matching algorithm to obtain the adaptive composition feature point set.

[0068] S3. Obtain a first transformation model and a second transformation model according to the adaptive composition feature point set.

[0069] Specifically, according to the adaptive composition feature point set, a first transformation model and a second transformation model are obtained, which satisfy the following formula:

[0070]

[0071]

[0072] ,in, 、 Respectively represent the feature points in the adaptive composition feature point set The horizontal and vertical coordinates of 、 Respectively represent the feature points in the adaptive composition feature point set The horizontal and vertical coordinates of represents the homography matrix, represents a constant;

[0073]

[0074] in, represents the first transformation model, Indicates the maximum area of ​​the figure formed by the feature points, represents the area of ​​similar regions, represents the similarity matrix;

[0075]

[0076] in, Represents the second transformation model.

[0077] Furthermore, the similarity matrix can be obtained through similarity transformation. The coordinates of a pixel point in the original image are , the coordinates of the point after similarity transformation are , then the similarity transformation is expressed in matrix as:

[0078] ,in, Indicates the scale factor, that is, in addition to translation and rotation, the image will also produce a certain degree of scaling. represents the rotation angle, and Respectively and The offset of the direction.

[0079] S4. Fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle.

[0080] In the embodiment, the step S4 of fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle includes the following steps:

[0081] S41. Obtain a first projection image of the real-time image based on the first transformation model and the second transformation model.

[0082] The image A to be fused is subjected to a first projection change through the first transformation model, and the image B to be fused is subjected to a first projection change through the second transformation model, and both are projected uniformly into an image domain coordinate system.

[0083] S42, obtaining low-frequency components and high-frequency components of the first projection image by using wavelet decomposition.

[0084] Perform wavelet decomposition on the image to be fused, select a suitable wavelet basis function, such as Haar wavelet, Daubechies wavelet, and perform multi-layer wavelet decomposition on the image to obtain sub-images of different frequencies, namely low-frequency components and high-frequency components.

[0085] It can be understood that after N wavelet decompositions, a low-frequency component and 3N high-frequency components will be generated, where the low-frequency component represents the overall contour and brightness information of the image, retains the overall contour information of the original image, and is a reduced version of the original image. The high-frequency component represents the detail information of the image. Since there are as many as 3N high-frequency components, the sizes of each high-frequency component are not exactly the same under the wavelet decomposition framework. The high-frequency component actually reflects the detail information of the original image at each scale, but there is a certain degree of distinction. Therefore, the high-frequency components must be fused by region.

[0086] S43: construct an image fusion model, and use the image fusion model to fuse the low-frequency components and the high-frequency components respectively.

[0087] The low-frequency part and the high-frequency part are fused respectively using the fusion model. Furthermore, the image fusion model satisfies the following formula:

[0088]

[0089] in, represents the fused component, represents the fusion coefficient, , represents the width of the similarity region, Indicates the distance between the pixel of the reference image and the left boundary of the fusion area. and They represent the components before fusion.

[0090] S44, obtaining a panoramic image of the vehicle based on inverse wavelet decomposition combined with the fused low-frequency component and high-frequency component.

[0091] The fused sub-images are subjected to inverse wavelet transform to obtain the final fused image. Inverse wavelet transform synthesizes sub-images of different frequencies and restores them to the original image.

[0092] See also Figure 2In an embodiment, in order to efficiently execute the adaptive composition method based on feature fusion provided by the present invention, the present invention also provides an adaptive composition system based on feature fusion, including: an input device, an output device, a processor, and a memory, wherein the input device, the output device, the processor, and the memory are interconnected, and the memory contains program instructions, and the program instructions are used for the steps of the adaptive composition method based on feature fusion. The adaptive composition system based on feature fusion of the present invention has a compact structure and stable performance, and can stably execute the adaptive composition method based on feature fusion of the present invention, further improving the overall applicability and practical application ability of the present invention.

[0093] In an embodiment, the processor may be a central processing unit (CPU), which may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. An input device may be used to obtain data information. An output device may be used to output the results obtained by storing program instructions contained in a computer program in a memory provided by the present invention. The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the type of device.

[0094] In yet another alternative embodiment, see Figure 3 In order to efficiently execute the adaptive composition method based on feature fusion provided by the present invention, this embodiment also provides an adaptive composition device based on feature fusion, such as Figure 3 As shown, including:

[0095] The memory 10 is used to store computer programs; the processor 20 is used to execute the computer programs to implement the above-mentioned adaptive composition method based on feature fusion.

[0096] Memory 10 , processor 20 , communication interface 31 and communication bus 32 . Memory 10 , processor 20 , communication interface 31 all communicate with each other via communication bus 32 .

[0097] In an embodiment, the memory 10 is used to store one or more program instructions. The memory 10 may store program instructions for implementing the following functions: acquiring a real-time image around the vehicle, extracting similar areas of the real-time image, and then obtaining feature points of the similar areas; performing three-level pairing using the feature points to obtain an adaptive composition feature point set; obtaining a first transformation model and a second transformation model based on the adaptive composition feature point set; and fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle.

[0098] In a possible implementation, the memory 10 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function, etc.; the data storage area may store data created during use.

[0099] In addition, the memory 10 may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include an NVRAM. The memory stores an operating system and operating instructions, executable modules or data structures, or a subset thereof, or an extended set thereof, wherein the operating instructions may include various operating instructions for implementing various operations. The operating system may include various system programs for implementing various basic tasks and processing hardware-based tasks.

[0100] The processor 20 may be a central processing unit (CPU), an application specific integrated circuit, a digital signal processor, a field programmable gate array or other programmable logic device, a microprocessor or any conventional processor, etc. The processor 20 may call a program stored in the memory 10 .

[0101] The communication interface 31 may be an interface of a communication module, and is used to connect to other devices or systems.

[0102] Of course, it should be noted that Figure 3 The structure shown does not constitute a limitation on the adaptive imaging device based on feature fusion in this embodiment. In practical applications, the adaptive imaging device based on feature fusion may include: Figure 3 More or fewer components than shown, or combinations of certain components.

[0103] The embodiment further provides a storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned adaptive composition method based on feature fusion are implemented.

[0104] The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0105] In summary, the present invention reduces the number of feature point extractions and improves the matching efficiency and accuracy of feature points by extracting feature points in similar areas and performing three-level pairing on the feature points, which is beneficial to improving the speed and efficiency of image fusion. Secondly, the pairing results are projected twice, and then the projection results are image-fused to obtain a panoramic image of the vehicle, solving the problem of poor uniformity of the composition scale of the 360-degree panoramic image of the vehicle and poor overall quality of the panoramic image caused by blurred boundaries and ghosting. Therefore, the present invention effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. An adaptive composition method based on feature fusion, characterized in that: The adaptive composition method based on feature fusion comprises the following steps: Acquire a real-time image around the vehicle, extract a similar area of ​​the real-time image, and then obtain feature points of the similar area; Performing three-level pairing using the feature points to obtain an adaptive composition feature point set; According to the adaptive composition feature point set, obtaining a first transformation model and a second transformation model; Fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle; The method of performing three-level pairing using the feature points to obtain an adaptive composition feature point set comprises the following steps: Calculating the similarity distance of the feature points, performing first-level pairing according to the similarity distance, and obtaining a first-level feature point set; Constructing a secondary pairing model, combining the secondary pairing model with the first-level feature point set, performing a second-level pairing, and obtaining a second-level feature point set; The grid edge distance parameter is introduced to improve the grid motion statistical matching algorithm, and the second-level feature point set is corrected by the improved grid motion statistical matching algorithm to obtain the adaptive composition feature point set; The secondary pairing model satisfies the following formula: ,in, Represents a collection, Indicates The spatial similarity of feature points, Indicates In real-time image The horizontal coordinate of the feature point, Indicates In real-time image The vertical coordinate of the feature point, Indicates the number of feature point pairs in the first-level feature point set; The grid edge distance parameter satisfies the following formula: ,in, represents the grid edge distance, Indicates the probability that the feature point is correctly matched, Represents the length of the grid, Indicates the width of the grid. Indicates the number of grids.

2. The adaptive composition method based on feature fusion according to claim 1, characterized in that: The extracting of the similar area of ​​the real-time image comprises the following steps: Preliminarily determining the position of the real-time image in the vehicle panoramic image, and setting a template image according to the determination result; Using the template image, obtaining a similarity judgment value of the real-time image; The similar area of ​​the real-time image is determined by the similarity judgment value.

3. The adaptive composition method based on feature fusion according to claim 1, characterized in that: The step of obtaining a first transformation model and a second transformation model according to the adaptive composition feature point set comprises the following steps: Obtaining a homography transformation model according to the adaptive composition feature point set; Through the homography transformation model, a first transformation model and a second transformation model are obtained.

4. The adaptive composition method based on feature fusion according to claim 3, characterized in that: According to the adaptive composition feature point set, the homography transformation model obtained satisfies the following formula: ,in, 、 Represents the feature points The horizontal and vertical coordinates of 、 Represents the feature points The horizontal and vertical coordinates of represents the homography matrix, represents a constant; The first transformation model and the second transformation model are obtained by the homography transformation model, satisfying the following formula: ,in, represents the first transformation model, Indicates the maximum area of ​​the figure formed by the feature points, represents the area of ​​similar regions, represents the similarity matrix; ,in, Represents the second transformation model.

5. The adaptive composition method based on feature fusion according to claim 1, characterized in that: The step of fusing the real-time image based on the first transformation model and the second transformation model to obtain a panoramic image of the vehicle comprises the following steps: Based on the first transformation model and the second transformation model, obtaining a first projection image of the real-time image; Obtaining low-frequency components and high-frequency components of the first projection image by using wavelet decomposition; Constructing an image fusion model, and using the image fusion model to fuse the low-frequency component and the high-frequency component respectively; Based on inverse wavelet decomposition, the vehicle panoramic image is obtained according to the fused low-frequency components and high-frequency components.

6. The adaptive composition method based on feature fusion according to claim 5, characterized in that: The image fusion model satisfies the following formula: ,in, represents the fused component, represents the fusion coefficient, , represents the width of the similarity region, Indicates the distance between the pixel of the reference image and the left boundary of the fusion area. and They represent the components before fusion.

7. An adaptive composition system based on feature fusion, characterized in that: The adaptive composition system based on feature fusion comprises: an input device, an output device, a processor, and a memory, wherein the input device, the output device, the processor, and the memory are interconnected, and the memory comprises program instructions, and the program instructions are used to execute the adaptive composition method based on feature fusion according to any one of claims 1 to 6.

Images