3D surround view seam elimination method and system
By adjusting the texture weight of the camera and optimizing the texture mapping of the stitching seam model, the problem of obvious stitching lines at the stitching seam in the vehicle surround view system was solved, improving the field of view and imaging quality of the 3D surround view system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- WUHAN KOTEI INFORMATICS
- Filing Date
- 2022-08-31
- Publication Date
- 2026-05-22
Smart Images
Figure CN115619654B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of vehicle auxiliary systems technology, and in particular to a method and system for eliminating 3D surround view stitching seams. Background Technology
[0002] Image stitching technology has become a hot research topic in the automotive electronics field in recent years and is applied to in-vehicle surround view display systems. In-vehicle surround view systems primarily utilize four cameras mounted on the front, rear, and two rearview mirrors to capture four images and stitch them together to form a panoramic image. Currently, most in-vehicle surround view systems on the market use 2D surround view image stitching, but 2D surround view imaging has drawbacks such as distortion and a small field of view.
[0003] Meanwhile, since each of the four cameras maps to a separate camera model, and the two camera models also overlap to map to a common field-of-view model—the seam model—a total of eight mapping models are generated from the four cameras. However, for computational convenience, the existing models at the seam are all superimposed with a 50% weight from the two cameras. This results in each mapped seam model having two relatively obvious seam lines, as shown in the attached figure. Figure 4 As shown.
[0004] To address the aforementioned issues, how can the obvious splicing lines in the texture mapping of the splicing seam model be resolved by adjusting the camera texture weights? Summary of the Invention
[0005] The present invention provides a method and system for eliminating 3D surround view stitching seams, which solves the problem of obvious stitching lines in the texture mapping of the stitching seam model by adjusting the texture weight of the camera.
[0006] Firstly, a method for eliminating seams in 3D surround view stitching is provided, including the following steps:
[0007] Obtain the seam model;
[0008] Adjust the camera texture weights based on the seam model;
[0009] The seam model is used to map the seam texture of two adjacent cameras based on the adjusted camera texture weights.
[0010] According to the first aspect, in a first possible implementation of the first aspect, the step of "adjusting camera texture weights according to the stitching seam model" specifically includes the following steps:
[0011] The camera texture weights are calculated based on the included angle of the seam model; or...
[0012] The camera texture weights are calculated based on the number of models in the splicing seam model.
[0013] According to the first possible implementation of the first aspect, in the second possible implementation of the first aspect, the step of "calculating camera texture weights based on the included angle of the stitching seam model" specifically includes the following steps:
[0014] Extend multiple line segments from the vertex of the seam model to the bottom edge of the seam model, and obtain the angle between each line segment and the right seam of the seam model.
[0015] Based on the included angle between the two seams of the seam model and the included angle between each line segment and the right seam of the seam model, obtain the texture weight of the left camera and the texture weight of the right camera of the seam.
[0016] In this case, the vertex pixels on a single line segment have the same camera texture weight.
[0017] According to the second possible implementation of the first aspect, in the third possible implementation of the first aspect, the step of "obtaining the texture weight of the left camera and the texture weight of the right camera based on the included angle between the two seams of the seam model and the included angle between each line segment and the right seam of the seam model" specifically includes the following steps:
[0018] Based on the included angle α between the two seams of the splicing seam model, and the included angle β between each line segment and the right seam of the splicing seam model;
[0019] The texture weight of the camera on the left side of the seam is obtained as A:
[0020] A = β / α (1);
[0021] The texture weight of the camera on the right side of the seam is obtained as B:
[0022] B = 1 - A (2).
[0023] According to the third possible implementation of the first aspect, in the fourth possible implementation of the first aspect, the step of "calculating camera texture weights based on the number of models for the seam model" specifically includes the following steps:
[0024] Multiple line segments extend from the vertex of the splice seam model to the bottom edge of the splice seam model. The multiple line segments divide the splice seam model into multiple unit splice seam models. The position of each unit splice seam model is assigned a position number that gradually increases from left to right.
[0025] Based on the position number of each unit splice seam model, obtain the texture weight of the camera on the left side of the splice seam and the texture weight of the camera on the right side of the splice seam;
[0026] In this model, the vertex pixels on a single unit seam have the same camera texture weight.
[0027] According to the fourth possible implementation of the first aspect, in the fifth possible implementation of the first aspect, the step of "obtaining the texture weight of the left camera and the texture weight of the right camera of the splicing seam according to the position number of each unit splicing seam model" specifically includes the following steps:
[0028] The location number of the unit splice seam model is k;
[0029] The texture weight of the camera on the left side of the seam is obtained as A:
[0030] Equation (3);
[0031] The texture weight of the camera on the right side of the seam is obtained as B:
[0032] B = 1 - A (2);
[0033] Where k ranges from 1 to n.
[0034] According to the fifth possible implementation of the first aspect, in the sixth possible implementation of the first aspect, the step of "mapping the stitching seam model to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weights" specifically includes the following steps:
[0035] Obtain the vertex pixels of the left camera and the vertex pixels of the right camera;
[0036] The vertex pixels of the splicing seam model are obtained based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera at the splicing seam, and the texture weight of the right camera at the splicing seam.
[0037] According to the sixth possible implementation of the first aspect, in the seventh possible implementation of the first aspect, the step of "obtaining the vertex pixels of the splicing seam model based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera of the splicing seam, and the texture weight of the right camera of the splicing seam" specifically includes the following steps:
[0038] Based on the vertex pixel a of the left camera, the vertex pixel b of the right camera, the texture weight A of the left camera at the seam, and the texture weight B of the right camera at the seam;
[0039] Obtain the vertex pixel value S of the seam model:
[0040] Equation (4).
[0041] Secondly, a 3D surround view stitching seam elimination method system is provided, including:
[0042] The seam model module is used to obtain the seam model;
[0043] The camera texture weight module is communicatively connected to the stitching seam model module and is used to adjust the camera texture weight according to the stitching seam model.
[0044] The mapping module, which is communicatively connected to the camera texture weight module, is used to map the stitching seam model to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weight.
[0045] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the 3D surround view stitching seam elimination method as described in any of the preceding claims.
[0046] Compared with the prior art, the advantages of the present invention are as follows: First, a seam model is obtained; then, the camera texture weights are adjusted according to the seam model; then, the seam model is mapped to the seam textures of two adjacent cameras based on the adjusted camera texture weights; thus, the obvious seam line problem of the seam model texture mapping is solved by adjusting the camera texture weights. Attached Figure Description
[0047] Figure 1 This is a flowchart illustrating an embodiment of a 3D surround view splicing seam elimination method of the present invention;
[0048] Figure 2 This is a schematic diagram of the camera texture weights on the splicing seam model of the present invention;
[0049] Figure 3 This is another schematic diagram of the camera texture weight on the splicing seam model of the present invention;
[0050] Figure 4 It is a 3D surround view model with obvious seams.
[0051] Figure 5 This invention is a 3D surround view model without obvious seams;
[0052] Figure 6 This is a schematic diagram of the structure of a 3D surround view splicing seam elimination system according to the present invention. Attached image description:
[0054] 100. 3D surround view stitching seam elimination method system; 110. Stitching seam model module; 120. Camera texture weight module; 130. Mapping module. Detailed Implementation
[0055] Referring now to specific embodiments of the invention, examples of which are illustrated in the accompanying drawings. Although the invention will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the invention to the described embodiments. Rather, it is intended to cover variations, modifications, and equivalents included within the spirit and scope of the invention as defined by the appended claims. It should be noted that the method steps described herein can be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of both.
[0056] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0057] Note: The examples described below are merely specific examples and are not intended to limit the embodiments of the present invention to the specific steps, values, conditions, data, order, etc. Those skilled in the art can utilize the concept of the present invention to construct more embodiments not mentioned herein by reading this specification.
[0058] See Figure 1 As shown, this embodiment of the invention provides a method for eliminating 3D surround view stitching seams, including the following steps:
[0059] S100, Obtain the seam model;
[0060] S200, Adjust the camera texture weights according to the splicing seam model;
[0061] S300, the stitching seam model is mapped to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weights.
[0062] Specifically, in this embodiment, since each of the four cameras maps out a camera model, and the two camera models also overlap to map a common field of view model - the seam model, a total of eight mapping models are obtained from the four cameras. However, for the sake of calculation convenience, the existing models at the seam are all superimposed with a weight of 50% for the two cameras, which results in each mapped seam model having two relatively obvious seam lines.
[0063] Therefore, this invention first obtains a seam model (the height information of the seam model can be ignored, and it can be viewed as a triangular shape from above); then adjusts the camera texture weights according to the seam model; then maps the seam model to the seam texture of two adjacent cameras based on the adjusted camera texture weights; thus, the obvious seam line problem of texture mapping of the seam model is solved by adjusting the camera texture weights.
[0064] Preferably, in another embodiment of this application, the present invention provides two methods for adjusting the camera, and the step "S200, adjusting the camera texture weight according to the stitching seam model" specifically includes the following steps:
[0065] S210, calculate the camera texture weights based on the included angle of the stitching seam model; or...
[0066] S220, calculate the camera texture weight based on the number of models for the splicing seam model.
[0067] See also Figure 2 As shown, preferably, in another embodiment of this application, the step "S210, calculating the camera texture weight based on the included angle of the splicing seam model" specifically includes the following steps:
[0068] S211, extend multiple line segments from the vertex of the splice seam model to the bottom edge of the splice seam model, and obtain the included angle between each line segment and the right splice seam of the splice seam model;
[0069] S212, based on the included angle between the two splicing seams of the splicing seam model and the included angle between each line segment and the right splicing seam of the splicing seam model, obtain the texture weight of the left camera and the texture weight of the right camera of the splicing seam.
[0070] In this case, the vertex pixels on a single line segment have the same camera texture weight.
[0071] Figure 2 In the stitching seam model, each vertex pixel is the intersection of the horizontal and vertical lines in the image, and the vertex pixels on a single line segment have the same camera texture weight.
[0072] Preferably, in another embodiment of this application, the step "S212, obtaining the texture weight of the left camera and the texture weight of the right camera based on the included angle between the two seams of the seam model and the included angle between each line segment and the right seam of the seam model" specifically includes the following steps:
[0073] Based on the included angle α between the two seams of the splicing seam model, and the included angle β between each line segment and the right seam of the splicing seam model;
[0074] The texture weight of the camera on the left side of the seam is obtained as A:
[0075] A = β / α (1);
[0076] The texture weight of the camera on the right side of the seam is obtained as B:
[0077] B = 1 - A (2).
[0078] See Figure 3 As shown, preferably, in another embodiment of this application, the step "S220, calculating camera texture weights based on the number of models for the seam model" specifically includes the following steps:
[0079] S221, extend multiple line segments from the vertex of the splicing seam model to the bottom edge of the splicing seam model. The multiple line segments divide the splicing seam model into multiple unit splicing seam models. The position of each unit splicing seam model is assigned a position number that gradually increases from left to right.
[0080] S222, based on the position number of each unit splicing seam model, obtain the texture weight of the camera on the left side of the splicing seam and the texture weight of the camera on the right side of the splicing seam;
[0081] In this model, the vertex pixels on a single unit seam have the same camera texture weight.
[0082] Preferably, in another embodiment of this application, the step "S222, obtaining the texture weight of the camera on the left side of the splicing seam and the texture weight of the camera on the right side of the splicing seam according to the position number of each unit splicing seam model" specifically includes the following steps:
[0083] The location number of the unit splice seam model is k;
[0084] The texture weight of the camera on the left side of the seam is obtained as A:
[0085] Equation (3);
[0086] The texture weight of the camera on the right side of the seam is obtained as B:
[0087] B = 1 - A (2);
[0088] Where k ranges from 1 to n.
[0089] Preferably, in another embodiment of this application, the step "S300, mapping the seam model to the seam texture of two adjacent cameras based on the adjusted camera texture weights" specifically includes the following steps:
[0090] S310, obtain the vertex pixels of the left camera and the vertex pixels of the right camera;
[0091] S320: Obtain the vertex pixels of the splicing seam model based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera at the splicing seam, and the texture weight of the right camera at the splicing seam.
[0092] Preferably, in another embodiment of this application, the step "S320, obtaining the vertex pixels of the splicing seam model based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera at the splicing seam, and the texture weight of the right camera at the splicing seam" specifically includes the following steps:
[0093] Based on the vertex pixel a of the left camera, the vertex pixel b of the right camera, the texture weight A of the left camera at the seam, and the texture weight B of the right camera at the seam;
[0094] Obtain the vertex pixel value S of the seam model:
[0095] Equation (4).
[0096] Specifically, in this embodiment, the corresponding pixel value can be calculated by traversing all camera vertex pixels according to equation (4). The mapping result according to the above formula can be found in [reference]. Figure 5 As shown, no obvious seams are visible, and this method can effectively eliminate seams in 3D surround view models.
[0097] Therefore, each time only the weight of each vertex of the camera is saved during initialization, when 3D surround view stitching is needed again, the corresponding calculation can be performed by using the saved weight table lookup algorithm. Therefore, the processing speed is very fast and the 3D surround view is very smooth.
[0098] See also Figure 6 As shown, this embodiment of the invention also provides a 3D surround view stitching seam elimination method system 100, comprising:
[0099] The seam model module 110 is used to acquire the seam model;
[0100] The camera texture weight module 120 is communicatively connected to the splicing seam model module 110 and is used to adjust the camera texture weight according to the splicing seam model.
[0101] The mapping module 130 is communicatively connected to the camera texture weight module 120 and is used to map the stitching seam model to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weight.
[0102] This invention first obtains a seam model; then adjusts the camera texture weights based on the seam model; and finally maps the seam textures of two adjacent cameras to the seam model based on the adjusted camera texture weights. Therefore, the obvious seam line problem in the texture mapping of the seam model is solved by adjusting the camera texture weights. Thus, each time only initialization is needed, the weight of each vertex of the camera is saved. When 3D surround view stitching is needed again, only the saved weight table and lookup table algorithm are needed for corresponding calculations, resulting in fast processing speed and high 3D surround view smoothness.
[0103] Specifically, this embodiment corresponds one-to-one with the above method embodiments. The functions of each module have been described in detail in the corresponding method embodiments, so they will not be repeated here.
[0104] Based on the same inventive concept, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements all or part of the method steps of the above method.
[0105] The present invention can implement all or part of the processes in the above methods, or it can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.
[0106] Based on the same inventive concept, embodiments of this application also provide an electronic device, including a memory and a processor. The memory stores a computer program that runs on the processor. When the processor executes the computer program, it implements all or part of the method steps described above.
[0107] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the computer device, connecting all parts of the computer device through various interfaces and lines.
[0108] Memory can be used to store computer programs and / or modules. The processor performs various functions of the computer device by running or executing the computer programs and / or modules stored in the memory, and by accessing data stored in the memory. Memory can primarily include a program storage area and a data storage area. The program storage area can store the operating system and at least one application program required for a function (e.g., sound playback, image playback, etc.); the data storage area can store data created based on the use of the mobile phone (e.g., audio data, video data, etc.). Furthermore, memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, RAM, plug-in hard disks, SmartMedia Cards (SMC), Secure Digital (SD) cards, Flash Cards, at least one disk storage device, flash memory device, or other volatile solid-state storage devices.
[0109] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, servers, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.
[0110] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), servers, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0111] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0112] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0113] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.
Claims
1. A method for eliminating seams in 3D surround view stitching, characterized in that, Includes the following steps: Obtain the seam model; Adjust the camera texture weights based on the seam model; The stitching seam model is mapped to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weights; The step of adjusting the camera texture weights based on the seam model specifically includes the following steps: Multiple line segments extend from the vertex of the splice seam model to the bottom edge of the splice seam model. The multiple line segments divide the splice seam model into multiple unit splice seam models. The position of each unit splice seam model is assigned a position number that gradually increases from left to right. Based on the position number of each unit splice seam model, obtain the texture weight of the camera on the left side of the splice seam and the texture weight of the camera on the right side of the splice seam; In this model, the vertex pixels on a single unit stitching seam have the same camera texture weight; The step of obtaining the texture weights of the camera on the left and right sides of the splicing seam based on the position number of each unit splicing seam model specifically includes the following steps: The location number of the unit splice seam model is k; The texture weight of the camera on the left side of the seam is obtained as A: ； The texture weight of the camera on the right side of the seam is obtained as B: B = 1 - A; Where k ranges from 1 to n.
2. The 3D surround view stitching seam elimination method as described in claim 1, characterized in that, The calculation of camera texture weights based on the included angle for the seam model specifically includes the following steps: Extend multiple line segments from the vertex of the seam model to the bottom edge of the seam model, and obtain the angle between each line segment and the right seam of the seam model. Based on the included angle between the two seams of the seam model and the included angle between each line segment and the right seam of the seam model, obtain the texture weight of the left camera and the texture weight of the right camera of the seam. In this case, the vertex pixels on a single line segment have the same camera texture weight.
3. The 3D surround view stitching seam elimination method as described in claim 2, characterized in that, The step of obtaining the left-side camera texture weight and the right-side camera texture weight of the splicing seam based on the included angle between the two splicing seams of the splicing seam model and the included angle between each line segment and the right-side splicing seam of the splicing seam model specifically includes the following steps: Based on the included angle α between the two seams of the splicing seam model, and the included angle β between each line segment and the right seam of the splicing seam model; The texture weight of the camera on the left side of the seam is obtained as A: A = β / α； The texture weight of the camera on the right side of the seam is obtained as B: B = 1 - A.
4. The 3D surround view stitching seam elimination method as described in claim 2, characterized in that, The step of mapping the seam model to the seam texture of two adjacent cameras based on the adjusted camera texture weights specifically includes the following steps: Obtain the vertex pixels of the left camera and the vertex pixels of the right camera; The vertex pixels of the splicing seam model are obtained based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera at the splicing seam, and the texture weight of the right camera at the splicing seam.
5. The 3D surround view stitching seam elimination method as described in claim 4, characterized in that, The step of obtaining the vertex pixels of the splicing seam model based on the vertex pixels of the left camera, the vertex pixels of the right camera, the texture weight of the left camera at the splicing seam, and the texture weight of the right camera at the splicing seam specifically includes the following steps: Based on the vertex pixel a of the left camera, the vertex pixel b of the right camera, the texture weight A of the left camera at the seam, and the texture weight B of the right camera at the seam; Obtain the vertex pixel value S of the seam model: 。 6. A 3D surround view seam elimination system, characterized in that, include: The seam model module is used to obtain the seam model; The camera texture weight module is communicatively connected to the stitching seam model module and is used to adjust the camera texture weight according to the stitching seam model. The mapping module is communicatively connected to the camera texture weight module and is used to map the stitching seam model to the stitching seam texture of two adjacent cameras based on the adjusted camera texture weight. The step of adjusting the camera texture weights based on the seam model specifically includes the following steps: Multiple line segments extend from the vertex of the splice seam model to the bottom edge of the splice seam model. The multiple line segments divide the splice seam model into multiple unit splice seam models. The position of each unit splice seam model is assigned a position number that gradually increases from left to right. Based on the position number of each unit splice seam model, obtain the texture weight of the camera on the left side of the splice seam and the texture weight of the camera on the right side of the splice seam; In this model, the vertex pixels on a single unit stitching seam have the same camera texture weight; The step of obtaining the texture weights of the camera on the left and right sides of the splicing seam based on the position number of each unit splicing seam model specifically includes the following steps: The location number of the unit splice seam model is k; The texture weight of the camera on the left side of the seam is obtained as A: ； The texture weight of the camera on the right side of the seam is obtained as B: B = 1 - A; Where k ranges from 1 to n.
7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the 3D surround view seam elimination method as described in any one of claims 1 to 5.