Around-looking aerial view splicing implementation method and system, vehicle and storage medium

Abstract

The invention relates to the technical field of automobile auxiliary driving, in particular to a panoramic aerial view splicing implementation method and system, a vehicle and a storage medium. The panoramic aerial view splicing implementation method comprises the following steps: S1, establishing a three-dimensional coordinate system of a vehicle, and calibrating internal and external parameters of a plurality of camera devices; s2, constructing a virtual camera right above the vehicle based on the calibrated internal and external parameters; s3, performing overlook projection on the image data acquired by the camera device based on the virtual camera to obtain a plurality of overlook area images; s4, respectively screening reference points of two adjacent overlook area images to obtain a first reference point set and a second reference point set; s5, performing texture matching on the first reference point set and the second reference point set to obtain a matching point set; s6, self-adaptive texture transformation is conducted through the matching point sets, and the mapping relation of the matching point sets is established. According to the method, the problem of texture dislocation on the two sides of the splicing seam of the look-around aerial view can be solved.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile assisted driving, and in particular to a method, system, vehicle and storage medium for realizing stitching of a surround bird's-eye view. Background Art

[0002] As cars continue to become more intelligent, electrified, lightweight, networked and shared, car consumers are paying more and more attention to safety during driving. Eliminating the driver's blind spots is an important safety requirement during driving. The on-board surround view system, also known as the panoramic image monitoring system, uses the images collected by the fisheye cameras installed around the car body to stitch a 360° panoramic image without blind spots. The body surround view images provided by the on-board surround view system will provide the driver with a larger external field of view of the car body, eliminating the blind spots around the car body to achieve the function of assisted driving.

[0003] At present, most in-vehicle surround view systems are composed of two parts: a bird's-eye view and a specific perspective view. The bird's-eye view uses multiple cameras for two-dimensional projection imaging, which is mainly used to display the relative position relationship between the vehicle and the surrounding environment, so that the driver can have an overall understanding and control of the surrounding environment. However, the real world is three-dimensional. When the camera shoots real scenes, the pictures taken only have two-dimensional coordinate information, lacking the three-dimensional coordinate information of the scene. In order to realize the projection of the scene to the surround view virtual camera, it is necessary to find a way to complete the three-dimensional coordinate information of the scene.

[0004] Specifically, when there are high objects next to the vehicle (such as curbs on both sides of the road, other vehicles, etc.), when they cross the seam, due to the lack of height information, most of them use the method of assuming that the scene height Z=0 to complete the three-dimensional coordinate information, at the expense of the display effect, to achieve surround stitching and projection. In order to capture a complete bird's-eye view of the entire vehicle body, it is necessary to stitch the two-dimensional projection information of each camera to form a seam. Since different cameras are installed at different positions and heights, the degree and direction of the stretching of high objects on both sides of the seam are also very different. This makes it impossible to align the high textures on both sides of the seam, affecting the driver's driving judgment. The principle of distortion of multiple cameras projecting the same position is as follows. Figure 1 As shown, from Figure 1 It can be seen that the two-dimensional projection of the object is at the center of the imaging range of camera one, but at the lower part of the imaging range of camera two. This is the projection misalignment problem of the same object in different imaging ranges caused by the difference in camera positions.

[0005] The above problems may cause confusion or misjudgment while driving, and it is difficult to provide the driver with clear and accurate information about the vehicle's surroundings, especially on both sides of the seam. Due to the different installation positions of the fisheye cameras, the stretching ratios are different, which makes the projected texture of high objects crossing the seam misaligned, causing a sense of fragmentation in the display. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method, system, vehicle and storage medium for realizing stitching of a surrounding bird's-eye view. The method for realizing stitching of a surrounding bird's-eye view can solve the problem of texture misalignment on both sides of the stitching seam of a surrounding bird's-eye view. The system, vehicle and storage medium for realizing stitching of a surrounding bird's-eye view are used to realize the method for realizing stitching of a surrounding bird's-eye view.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides a method for stitching a bird's-eye view of a panoramic view, comprising the following steps:

[0009] S1: Establishing a three-dimensional coordinate system of the vehicle and calibrating the internal and external parameters of multiple camera devices;

[0010] S2: Based on the calibrated internal and external parameters, a virtual camera is constructed directly above the vehicle;

[0011] S3: performing a top-down projection on the image data acquired by the camera device based on a virtual camera to obtain a plurality of top-down area images;

[0012] S4: respectively screening reference points of two adjacent overhead area images to obtain a first reference point set and a second reference point set;

[0013] S5: Perform texture matching on the first reference point set and the second reference point set to obtain a matching point set;

[0014] S6: Use the matching point set to perform adaptive texture transformation and establish a mapping relationship of the matching point set.

[0015] By adopting the above technical solution, the processor can establish a three-dimensional coordinate system of the vehicle based on the vehicle position, and the calibration of the internal and external parameters of multiple camera devices is a pre-work, which can effectively improve the accuracy of the data. The processor can build a virtual camera directly above the vehicle based on the internal and external parameters of multiple camera devices after calibration, and then the processor can use the virtual camera to perform a bird's-eye view projection on the image data obtained by the camera device to obtain multiple bird's-eye view images. A single bird's-eye view image is part of the panoramic bird's-eye view. Multiple bird's-eye view images are spliced ​​in sequence to obtain a panoramic bird's-eye view. After that, the processor will screen the reference points of the two adjacent bird's-eye view images respectively to obtain the first reference point set of one of the bird's-eye view images and the second reference point set of the adjacent bird's-eye view image. The processor will then perform texture matching on the first reference point and the second reference point to obtain the corresponding matching point set. Finally, the processor can establish a mapping relationship of the matching point set through adaptive texture transformation to match the corresponding points in the matching point set, thereby realizing the splicing between the two adjacent bird's-eye view images. After stitching all the overhead area images using the above method, a surround bird's-eye view can be obtained. Since the texture pixels of two adjacent overhead area images are aligned, the texture misalignment problem on both sides of the stitching seam of the surround bird's-eye view can be solved, thereby providing the driver with clear and accurate vehicle surrounding information.

[0016] Optionally, step S3 includes:

[0017] Multiple camera devices respectively acquire image data, and the processor performs a top-down projection on the image data acquired by each camera device through a virtual camera, and the processor can obtain the top-down area images F1, F2, F3, ..., F corresponding to the n camera devices. n .

[0018] By adopting the above technical solution, multiple camera devices can respectively obtain image data, and the processor can perform a bird's-eye view projection of the image data through a virtual camera, thereby obtaining a corresponding bird's-eye view area image.

[0019] Optionally, step S4 includes the following steps:

[0020] S41: Obtain any point P(u i , v j ), and obtain any point P(u i , v j )'s first color representation value Get any point Q(w a , x b ), and obtain any point Q(w a , x b )'s second color representation value

[0021] S42: N points are respectively selected from the vehicle body position along the splicing line segment from near to far in the two top-view area images, and based on the plurality of first color representation values Get the first color representation value curve about the splicing line segment Based on a plurality of second color representation values The second color representation value curve about the splicing line segment is obtained

[0022] S43: First color representation value curve for the splicing line segment By taking the first-order derivative, we can get the first first-order derivative point set about the first color representation value curve. Second color representation value curve for the splicing line segment Calculate the first-order derivative and obtain the second first-order derivative point set about the second color representation value curve

[0023] S44: According to the preset screening method, respectively select the first first-order derivative point set F set and the second first-order derivative point set G set Select m first-order derivative points from the first first-order derivative point set F set The selected m first-order derivative points are used as the first reference point set F′ set , from the second first-order derivative point set G set The selected m first-order derivative points are used as the second reference point set G′ set .

[0024] By adopting the above technical solution, when the processor obtains any points on the splicing line segment of two adjacent overhead area images respectively, it can calculate the color representation values ​​of the two arbitrary points. When the processor obtains N points from near to far on the splicing line segment extending from the vehicle body position of two adjacent overhead area images respectively, it can calculate the color representation value curves of two groups of points, that is, the color representation curve is obtained through multiple color representation values. After that, the processor calculates the first-order derivatives of the two color representation curves respectively to obtain the corresponding two first-order derivative point sets. Finally, the processor selects any first-order derivative points from the first-order derivative point set to obtain the corresponding two reference point sets.

[0025] Optionally, the step S5 includes the following steps:

[0026] S51: In the first reference point set F′ set Select a feature point P from f , take c pixels before and after, a total of 2c+1 pixels to form the first one-dimensional matching template, and generate the first template vector

[0027]

[0028] Among them, the feature point P f On the splicing line segment, Represents the color representation value of any overlooking area image;

[0029] In the second reference point set G′ set Select a feature point P from l , take c pixels before and after, a total of 2c+1 points to form the second one-dimensional matching template, and generate the second template vector

[0030]

[0031] Among them, the feature point P l On the splicing line segment, Represents the color representation value on the image of the adjacent overlooking area;

[0032] S52: for the second template vector Perform a scaling and translation transformation to obtain the second template vector Transformation of

[0033]

[0034] Among them, α represents the stretch coefficient and β represents the translation coefficient;

[0035] S53: The first template vector With the second template vector Transformation of Perform similarity matching, and its similarity coefficient is defined as:

[0036]

[0037] Among them, C l，f (α, β) represents the second template vector Transformation of With the first template vector The similarity coefficient of

[0038] In the first template vector With the second template vector Transformation of After similarity matching is completed, we can get s pairs of matching point sets CM:

[0039] CM={(B,B′),(k s , k′ s ),...,(k1,k′1),(C,C′)}.

[0040] By adopting the above technical solution, the processor can calculate a template vector through one of the reference point sets, and calculate the transformation of another template vector through another reference point set, and then the processor can perform similarity matching on the two and obtain the corresponding matching point set.

[0041] Optionally, step S6 includes:

[0042] By dividing any two pairs of points in the matching point set CM into source intervals and mapping intervals, and establishing a mapping relationship between the source interval and the mapping interval, the texture pixels in any overlooking area image and the adjacent overlooking area image can be aligned to achieve the stitching of the surround bird's-eye view.

[0043] By adopting the above technical solution, the processor can divide the points in the matching point set into source intervals and mapping intervals, thereby establishing a mapping relationship, so as to align the points of one of the overlooking area images with the points of the adjacent overlooking area images, that is, to align the texture pixels. After the mapping relationship is established, the splicing of the bird's-eye view can be realized, and the texture misalignment problem on both sides of the splicing seam of the bird's-eye view can be solved.

[0044] Optionally, the method for implementing stitching of the panoramic bird's-eye view further includes: a display screen displays the stitched panoramic bird's-eye view.

[0045] By adopting the above technical solution, the display screen can present a surround bird's-eye view to the driver, thereby facilitating the driver to operate the vehicle according to the surround bird's-eye view.

[0046] In a second aspect, the present invention provides a system for realizing stitching of a surrounding bird's-eye view, comprising: a processor and a plurality of camera devices, wherein the processor is communicatively connected to the plurality of camera devices respectively to implement the method for realizing stitching of a surrounding bird's-eye view as described in the first aspect.

[0047] By adopting the above technical solution, the camera device can obtain image data, and the processor can process the image data and realize the splicing of the surrounding bird's-eye view, thereby solving the problem of texture misalignment on both sides of the splicing seam of the surrounding bird's-eye view.

[0048] Optionally, the surround bird's-eye view stitching implementation system further includes: a display screen, which is communicatively connected to the processor.

[0049] By adopting the above technical solution, the display screen can display the bird's-eye view after splicing under the control of the processor.

[0050] In a third aspect, the present invention provides a vehicle comprising a processor, a memory and a communication bus, wherein the communication bus is used to realize a communication connection between the processor and the memory, and the processor is used to execute a computer program stored in the memory to implement the method for splicing a surround bird's-eye view as described in the first aspect.

[0051] By adopting the above technical solution, a vehicle is provided that realizes the method for splicing a surround bird's-eye view.

[0052] In a fourth aspect, the present invention further provides a computer-readable storage medium storing a computer program; the computer program can be executed by a processor to implement a method for stitching a panoramic bird's-eye view as described in the first aspect.

[0053] By adopting the above technical solution, a computer program carrier for realizing the method of splicing a panoramic bird's-eye view is provided.

[0054] In summary, the present invention at least includes the following beneficial technical effects:

[0055] The present invention can realize the basic functions of a surround bird's-eye view, and can eliminate the texture misalignment problem of objects with height on both sides of the bird's-eye view seam based on adaptive texture alignment, eliminate the sense of display fragmentation, thereby providing the driver with clear and accurate vehicle surrounding information, and effectively solving the problem of visual judgment confusion caused by visual display fragmentation to the driver; compared with ordinary bird's-eye view implementation solutions, the adaptive texture alignment algorithm has a smaller amount of calculation, only relies on the collected surround view images, and does not add additional hardware. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 Schematic diagram of the distortion caused by multiple cameras projecting the same position.

[0057] Figure 2 The present invention is a flowchart of a method for implementing a surround bird's-eye view stitching method in an embodiment of the present invention.

[0058] Figure 3 Schematic diagram of the overlapping area of ​​two adjacent top-view area images.

[0059] Figure 4 It is a mapping relationship diagram between source interval and mapping interval.

[0060] Figure 5 It is a schematic diagram of a system for realizing stitching of a panoramic bird's-eye view according to an embodiment of the present invention.

[0061] Figure 6 It is a schematic diagram of another system for realizing stitching of a surround bird's-eye view in an embodiment of the present invention.

[0062] Figure 7Schematic diagram of a vehicle in an embodiment of the present invention.

[0063] Explanation of the accompanying drawings: 1. Camera device; 2. Processor; 3. Display screen; 4. Memory; 5. Communication bus; 6. Communication interface. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the following Figure 2-7 It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0065] The terms used in the following embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to be used as limitations of the present invention. As used in the specification of the present invention and the appended claims, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear contrary indication in its context. It should also be understood that the term "and / or" used in the present invention refers to and includes any or all possible combinations of one or more listed items. The term "exemplary" means "used as an example, embodiment or illustrative", and any embodiment described as "exemplary" here does not have to be interpreted as being superior to or better than other embodiments. The terms "first" and "second" are only used for descriptive purposes, and cannot be understood as implying or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can expressly or implicitly include one or more of these features, and in the description of the embodiments of the present invention, unless otherwise specified, the meaning of "multiple" is two or more.

[0066] An embodiment of the present invention provides a method for splicing a panoramic bird's-eye view.

[0067] refer to Figure 2 A method for stitching a bird's-eye view of a panoramic view includes the following steps:

[0068] S1: Establish a three-dimensional coordinate system of the vehicle and calibrate the internal and external parameters of multiple camera devices 1.

[0069] The processor establishes a three-dimensional coordinate system of the vehicle and calibrates the internal and external parameters of the multiple camera devices 1 .

[0070] Among them, multiple camera devices 1 are installed on the vehicle, preferably four camera devices 1, and the four camera devices 1 are installed in the front, left, right and rear of the vehicle respectively. The four camera devices 1 are used as the external scene acquisition and photography of the vehicle-mounted surround view system to acquire the image data of the environment around the vehicle. The installation position of the camera device 1 is the existing technology and will not be repeated here.

[0071] The positions of multiple camera devices 1 are calibrated by the checkerboard calibration method, and the internal and external parameters of the multiple camera devices 1 can be obtained. The internal and external parameters include camera intrinsic parameters and camera extrinsic parameters. The camera intrinsic parameters include focal length, principal point coordinates, and distortion parameters. The camera extrinsic parameters include the rotation and translation relationship of the camera device 1 relative to the three-dimensional coordinate system.

[0072] S2: Based on the calibrated internal and external parameters, a virtual camera is constructed directly above the vehicle.

[0073] The processor 2 constructs a virtual camera directly above the vehicle based on the calibrated internal and external parameters. The construction parameters of the virtual camera include the spatial position of the virtual camera, the orientation of the virtual camera, the position of the virtual imaging plane, the virtual imaging resolution, and the spatial size of the virtual imaging plane.

[0074] S3: Performing a bird's-eye view projection on the image data acquired by the camera device 1 based on the virtual camera to obtain a plurality of bird's-eye view area images.

[0075] Multiple camera devices 1 respectively acquire image data, and the processor 2 performs a top-down projection on the image data acquired by each camera device 1 through a virtual camera. The processor 2 can obtain the top-down area images F1, F2, F3, ..., F corresponding to the n camera devices 1. n .

[0076] S4: Filter reference points of two adjacent overhead-view area images respectively to obtain a first reference point set and a second reference point set.

[0077] refer to Figure 3 For two overhead area images corresponding to any two adjacent camera devices 1, the two overhead area images are adjacent, and there is an overlapping area and a stitching line segment. In the embodiment of the present application, taking the overhead area image F1 and the overhead area image F2 as an example, assuming that the overhead area image F1 and the overhead area image F2 are adjacent, there is an overlapping area ABCD, and the stitching line segment is BC.

[0078] BC is the driving direction, that is, the y direction in the bird's-eye view. It should be understood that the overlapping area does not need to be a quadrilateral, and the splicing line segment can also have an arbitrary slope and starting position.

[0079] Specifically, step S4 includes the following steps S41 to S44:

[0080] S41: An arbitrary point P (u i , v j ), take the first color representation value of the point in the preset direction And any point Q(w a , x b ), take the second color representation value of the point in the preset direction

[0081] First color representation value The expression is:

[0082]

[0083] Second color representation value The expression is:

[0084]

[0085] u represents the horizontal pixel point on the stitching line segment BC, i represents the specific position of the horizontal pixel point on the stitching line segment BC, v represents the vertical pixel point on the stitching line segment BC, j represents the specific position of the vertical pixel point on the stitching line segment BC, and s represents the number of pixels on the stitching line segment BC. The first color representation value is the grayscale value, the first color represents the value is the grayscale average of s pixels on the horizontal splicing line segment BC.

[0086] w represents the horizontal pixel point on the splicing line segment B′C′, a represents the specific position of the horizontal pixel point on the splicing line segment B′C′, x represents the vertical pixel point on the splicing line segment B′C′, b represents the specific position of the vertical pixel point on the splicing line segment B′C′, and d represents the number of pixels on the splicing line segment B′C′. Second color representation value is the grayscale value, the second color represents the value is the grayscale average value of d pixels on the horizontal splicing line segment B′C′.

[0087] Among them, the above-mentioned preset direction is horizontal. Since the color representation value is on the splicing line segment, in order to further eliminate the interference of random point selection, the color representation value of a certain point is the average value of the pixels in the area where it is located. In the present invention, the area where a certain point is located takes a line segment from the current point to a certain direction. For the convenience of calculation, the horizontal direction of this line segment is consistent with the vertical coordinate of the pixel. Other directions can also be taken, and the execution steps are the same.

[0088] S42: In the top-view area image F1, N points are taken from the vehicle body position along the splicing line segment BC from near to far, and the first color representation value of each point is calculated in the same manner as in step S41. The first color representation value curve about the splicing line segment BC can be obtained In the top-view area image F2, N points are taken from the position of the vehicle body along the splicing line segment B′C′ from near to far, and the second color representation value of each point is calculated in the manner of step S41. The second color representation value curve about the splicing line segment B′C′ can be obtained: In the embodiment of the present application, the point taking method adopts a uniform and dense point taking method.

[0089] S43: First color representation value curve for splicing line segment BC By taking the first-order derivative, we can get the first first-order derivative point set about the first color representation value curve. The second color representation value curve for the splicing line segment B′C′ By taking the first-order derivative, we can get the second first-order derivative point set about the second color representation value curve.

[0090] Specifically, the stitching area of ​​each overlooking area image is compressed into a color representation value curve through the previous color representation value, and a derivative is formed at the location where the color changes sharply. We use this location as the reference point. For the overlooking area image F1 and the overlooking area image F2 with overlapping areas, the images in them are only stretched, but the reference points in the overlooking area image F1 also exist in the overlooking area image F2, and we will associate the reference points in the two images.

[0091] S44: According to the preset screening method, respectively select the first first-order derivative point set F set and the second first-order derivative point set G set Select m first-order derivative points from the first first-order derivative point set F set The selected m first-order derivative points are used as the first reference point set F′ set , from the second first-order derivative point set G set The selected m first-order derivative points are used as the second reference point set G′ set .

[0092] The above-mentioned preset screening method may adopt a sorting method or a threshold method. In this embodiment, a method of sorting from high to low is adopted.

[0093] S5: Perform texture matching on the first reference point set and the second reference point set to obtain a matching point set.

[0094] Specifically, step S5 includes the following steps S51 to S53:

[0095] S51: In the first reference point set F′ set Select a feature point P from f , take c pixels before and after, a total of 2c+1 pixels to form the first one-dimensional matching template, and generate the first template vector

[0096]

[0097] Among them, the feature point P f On the splicing line segment BC, Represents the color representation value on the bird's-eye view area image F1.

[0098] In the second reference point set G′ set Select a feature point P from l , take c pixels before and after, a total of 2c+1 points to form the second one-dimensional matching template, and generate the second template vector

[0099]

[0100] Among them, the feature point P l On the splicing line segment B′C′, Represents the color representation value on the bird's-eye view area image F2.

[0101] S52: for the second template vector Perform a scaling and translation transformation to obtain the second template vector Transformation of

[0102]

[0103] Among them, α represents the stretch coefficient and β represents the translation coefficient.

[0104] Although the first template vector and the second template vector is matched one by one, but the positions are not the same due to projection distortion. Therefore, we need to stretch and transform the multiple pixels corresponding to one of the template vectors. It should be understood that the first template vector can also be used to perform the scaling and translation transformation. Performs a scaling and translation transformation.

[0105] S53: The first template vector With the second template vector Transformation of Perform similarity matching, and its similarity coefficient is defined as:

[0106]

[0107] Among them, C l，f (α, β) represents the second template vector Transformation of With the first template vector For α and β, a parameter iteration optimization algorithm can be used to obtain the optimal value using a difference equation. The parameter iteration optimization algorithm belongs to a general iterative optimization technology and is not part of the discussion of the present invention, so it will not be described in detail here.

[0108] In the first template vector With the second template vector Transformation of After similarity matching is completed, we can get s pairs of matching point sets CM:

[0109] CM={(B,B′),(k s , k′ s ),...,(k1,k′1),(C,C′)}

[0110] S6: Use the matching point set to perform adaptive texture transformation and establish a mapping relationship of the matching point set.

[0111] Any two pairs of points in the matching point set CM are divided into source intervals and mapping intervals, and a mapping relationship between the source interval and the mapping interval is established. At this time, the texture pixels in the overlooking area image F1 can be aligned with the texture pixels in the overlooking area image F2, that is, the surrounding bird's-eye view splicing is realized, thereby eliminating the texture misalignment on both sides of the splicing seam of the surrounding bird's-eye view.

[0112] Now let’s take two pairs of points (k t , k′ t ) and (k t+1 , k′ t+1 ) as an example, it can be divided into the source interval [k t , k t+1 ] and the mapping interval [k′ t , k′ t+1 ], the mapping relationship between these two intervals refers to Figure 4 shown.

[0113] For the source interval [k t , k t+1 ], it is in the mapping interval [k′ t , k′ t+1 ] is k'. In the embodiment of the present application, for the convenience of description, the directions of the stitching line segment BC of the overhead area image F1 and the stitching line segment B'C' of the overhead area image F2 are both in the y direction of the bird's-eye view. Therefore, it is assumed that point k t, click k t+1 The coordinates corresponding to point k are (u, v t )、(u,v t+1 ) and (u,u), point k′ t , point k′ t+1 The coordinates corresponding to point k′ are (u, u′ t )、(u,v′ t+1 ) and (u, v′), the texture transformation becomes a mapping relationship from v to v′:

[0114]

[0115] By establishing the above-mentioned mapping relationship of adaptive texture transformation, the texture pixels of the overlooking area image F1 and the overlooking area image F2 can be aligned, thereby solving the problem of texture misalignment on both sides of the seam of the surround bird's-eye view.

[0116] It should be understood that, in the above-mentioned bird's-eye view stitching implementation method, except for the image data of the vehicle surrounding environment collected by the camera device 1, the remaining steps are completed by the processor 2. n After the texture pixels in the image are aligned, the stitching of the bird's-eye view can be realized and the texture misalignment problem on both sides of the seam of the bird's-eye view can be solved.

[0117] In an optional implementation manner of the present application, the method for implementing the stitching of the surround bird's-eye view further includes: the display screen 3 displays the stitched surround bird's-eye view for the convenience of the driver to view.

[0118] The embodiment of the present invention also provides a system for realizing stitching of a panoramic bird's-eye view.

[0119] refer to Figure 5 A surround bird's-eye view stitching realization system includes: a camera device 1 and a processor 2, wherein the processor 2 is in communication connection with the camera device 1. There are multiple camera devices 1, preferably four in the embodiment of the present application, and the four camera devices 1 can be installed in the front, left, right and rear of the vehicle respectively.

[0120] refer to Figure 6 A surround bird's-eye view stitching realization system also includes: a display 3, the display 3 is communicatively connected with the processor 2, the display screen 3 can be installed on the center console between the front main and co-driver seats of the vehicle, and the display 3 is used to show the surround bird's-eye view after the stitching is completed.

[0121] The various variations and specific examples in the methods provided in the above embodiments are applicable to a system for realizing a panoramic bird's-eye view stitching in the present embodiment. Through the above detailed description of a method for realizing a panoramic bird's-eye view stitching, those skilled in the art can clearly know the implementation method of a system for realizing a panoramic bird's-eye view stitching in the present embodiment. For the sake of brevity of the specification, it will not be described in detail here.

[0122] Reference Figure 7 The embodiment of the present invention further provides a vehicle, including a processor 2, a memory 4 and a communication bus 5, wherein the communication bus 5 is used to realize the communication connection between the processor and the memory, and realize the communication interaction with various systems in the vehicle through a communication interface 6, and the communication bus 5 is communicatively connected with the communication interface 6 to realize the interaction between various systems, and the processor 2 is used to execute the computer program stored in the memory 4 to implement the above-mentioned surround bird's-eye view splicing realization method.

[0123] The memory 4 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 4 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function, and instructions for implementing the method for splicing a bird's-eye view of the surround provided in the above embodiment, etc.; the data storage area may store data involved in the method for splicing a bird's-eye view of the surround provided in the above embodiment, etc.

[0124] The processor 2 may include one or more processing cores. The processor 2 executes various functions and processes data of the present invention by running or executing instructions, programs, code sets or instruction sets stored in the memory 4, calling the data stored in the memory 4. The processor 2 may be at least one of an application-specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller and a microprocessor. It is understandable that for different devices, the electronic device used to implement the above-mentioned processor 2 function may also be other, and the embodiment of the present invention is not specifically limited.

[0125] The embodiment of the present invention further provides a computer-readable storage medium, for example, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes. The computer-readable storage medium stores a computer program that can be loaded by the processor 2 and execute a method for implementing a surround bird's-eye view splicing of the above embodiment.

[0126] An embodiment of the present invention also provides a computer program product, which includes a computer program tangibly contained in a readable medium, wherein the computer program includes program code for executing any one of the methods for implementing the splicing of a panoramic bird's-eye view in the embodiments of the present invention. The computer program can be downloaded and installed over the network, and / or installed from a removable medium (such as a disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc.).

[0127] As described above, the above embodiments are only used to introduce the technical solution of the present invention in detail, but the description of the above embodiments is only used to help understand the method and core idea of ​​the present invention and should not be understood as limiting the present invention. Changes or substitutions that can be easily thought of by technicians in this technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for splicing a bird's-eye view of a panoramic view, characterized in that: The following steps are involved: S1: Establishing a three-dimensional coordinate system of the vehicle and calibrating the internal and external parameters of multiple camera devices (1); S2: Based on the calibrated internal and external parameters, a virtual camera is constructed directly above the vehicle; S3: performing overhead projection on the image data acquired by the camera device (1) based on the virtual camera to obtain a plurality of overhead area images; S4: respectively screening reference points of two adjacent overhead area images to obtain a first reference point set and a second reference point set; S5: Perform texture matching on the first reference point set and the second reference point set to obtain a matching point set; S6: Use the matching point set to perform adaptive texture transformation and establish a mapping relationship of the matching point set.

2. The method for splicing a bird's-eye view of a panoramic view according to claim 1, characterized in that: The step S3 comprises: A plurality of camera devices (1) respectively acquire image data, and a processor (2) performs a top-down projection on the image data acquired by each of the camera devices (1) through a virtual camera, so that the processor (2) can obtain top-down area images F1, F2, F3, ..., F n .

3. The method for splicing a bird's-eye view of a panoramic view according to claim 2, characterized in that: The step S4 comprises the following steps: S41: Obtain any point P(u i , v j ), and obtain any point P(u i , v j )'s first color representation value Get any point Q(w a , x b ), and obtain any point Q(w a , x b )'s second color representation value S42: N points are respectively selected from the vehicle body position along the splicing line segment from near to far in the two top-view area images, and based on the plurality of first color representation values Get the first color representation value curve about the splicing line segment Based on a plurality of second color representation values The second color representation value curve about the splicing line segment is obtained S43: First color representation value curve for the splicing line segment By taking the first-order derivative, we can get the first first-order derivative point set about the first color representation value curve. Second color representation value curve for the splicing line segment Calculate the first-order derivative and obtain the second first-order derivative point set about the second color representation value curve S44: According to the preset screening method, respectively select the first first-order derivative point set F set and the second first-order derivative point set G set Select m first-order derivative points from the first first-order derivative point set F set The selected m first-order derivative points are used as the first reference point set F′ set , from the second first-order derivative point set G set The selected m first-order derivative points are used as the second reference point set G′ set .

4. The method for splicing a bird's-eye view of a panoramic view according to claim 3, characterized in that: The step S5 comprises the following steps: S51: In the first reference point set F′ set Select a feature point P from f , take c pixels before and after, a total of 2c+1 pixels to form the first one-dimensional matching template, and generate the first template vector Among them, the feature point P f On the splicing line segment, Represents the color representation value of any overlooking area image; In the second reference point set G′ set Select a feature point P from z , take c pixels before and after, a total of 2c+1 points to form the second one-dimensional matching template, and generate the second template vector Among them, the feature point P l On the splicing line segment, Represents the color representation value on the image of the adjacent overlooking area; S52: for the second template vector Perform a scaling and translation transformation to obtain the second template vector Transformation of Among them, α represents the stretch coefficient and β represents the translation coefficient; S53: The first template vector With the second template vector Transformation of Perform similarity matching, and its similarity coefficient is defined as: Among them, C l，f (α, β) represents the second template vector Transformation of With the first template vector The similarity coefficient of In the first template vector With the second template vector Transformation of After similarity matching is completed, we can get s pairs of matching point sets CM: CM={(B,B′),(k s ,k′ s ),...,(k1,k′1),(C,C′}。 5. The method for splicing a bird's-eye view of a panoramic view according to claim 1, characterized in that: The step S6 comprises: By dividing any two pairs of points in the matching point set CM into source intervals and mapping intervals, and establishing a mapping relationship between the source interval and the mapping interval, the texture pixels in any overlooking area image and the adjacent overlooking area image can be aligned to achieve the stitching of the surround bird's-eye view.

6. The method for splicing a bird's-eye view of a panoramic view according to claims 1 to 5, characterized in that: Also includes: The display screen (3) shows the bird's-eye view of the completed splicing.

7. A system for realizing stitching of a bird's-eye view of a panoramic view, characterized in that: include: A processor (2) and a plurality of camera devices (1), wherein the processor (2) is respectively connected in communication with the plurality of camera devices (1) to implement the method for stitching a surround bird's-eye view as described in any one of claims 1 to 6.

8. The system for realizing the splicing of the bird's-eye view of the panoramic view according to claim 7, characterized in that: Also includes: A display screen (3), wherein the display screen (3) is communicatively connected to the processor (2).

9. A vehicle, characterized in that: The invention comprises a processor (2), a memory (4) and a communication bus (5), wherein the communication bus (5) is used to realize a communication connection between the processor (2) and the memory (4), and the processor (2) is used to execute a computer program stored in the memory (4) to implement a method for splicing a bird's-eye view according to any one of claims 1 to 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; the computer program can be executed by a processor (2) to implement a method for stitching a surround bird's-eye view as described in any one of claims 1 to 6.

Images