Panoramic image rapid splicing method based on star flash wireless communication technology

Abstract

The invention is suitable for the technical field of automobile auxiliary driving, and provides a quick panoramic image splicing method based on a star flash wireless communication technology, which comprises the following steps of: S1, calibrating internal parameters and distortion coefficients of a camera by adopting a Zhang Zhengyou calibration method, calibrating external parameters by combining perspective transformation, and mapping each view angle image to a unified overlook coordinate system; s2, acquiring an image through a camera supporting a star flash protocol, and transmitting the image to a vehicle body domain controller by using a star flash technology; and S3, after distortion removal and perspective transformation are performed on the received image, a local pyramid fusion strategy is adopted: binary mask marking and multi-band fusion are performed on an overlapping region, original data are reserved in a non-overlapping region, and a panoramic image is finally output. Through combination of high-precision calibration, local pyramid fusion and a star flash communication technology, high-quality and low-delay splicing of vehicle-mounted panoramic images is realized, and the stability and practicability of the system in a complex environment are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of automobile assisted driving, and in particular relates to a method for quickly stitching panoramic images based on star flash wireless communication technology. Background Art

[0002] With the continuous development of intelligent vehicles and autonomous driving technology, 360-degree surround view systems have become a key feature of smart cars. The core goal of this system is to achieve full coverage of the vehicle's surroundings by capturing real-time images through multiple cameras installed around the vehicle body. This helps the driver better perceive the surrounding conditions, providing visual assistance without blind spots during reversing, parking, and driving, effectively reducing collision risks and improving driving safety.

[0003] A complete 360-degree surround view system usually consists of the following key components: Multi-camera configuration: Wide-angle (fisheye) cameras are generally installed in the front, rear, left, and right directions of the vehicle to ensure that the entire field of view around the vehicle is covered.

[0004] Image acquisition and transmission: Each camera collects images in real time and transmits them to the central processor via the vehicle network.

[0005] Image preprocessing: including dedistortion, color correction, image enhancement and other operations to improve image quality.

[0006] Image stitching and fusion: Through perspective transformation and multi-band fusion algorithms, images from different cameras are seamlessly stitched into a complete panoramic image.

[0007] User display and interaction: The final panoramic image will be presented in real time on the vehicle display, providing the driver with an intuitive and clear surround view.

[0008] However, the following technical challenges still exist in current mainstream 360-degree surround view solutions: Poor image consistency: Due to the different viewing angles, imaging characteristics, and lighting conditions of each camera, directly stitched images often have obvious seams and inconsistent colors and brightness, affecting the overall continuity and visual experience of the panoramic image.

[0009] Insufficient fusion strategy: When processing overlapping areas, some image fusion methods often fail to balance the smooth transition of low-frequency information and the retention of high-frequency details, resulting in blurring or abrupt transitions in the stitching area, causing loss of details or color aberration.

[0010] Inadequate real-time performance: In practical scenarios, surround view systems require high real-time image processing. Traditional signal transmission and processing methods often experience delays or instability when faced with complex and dynamic scenes, making it difficult to meet the demand for accurate visual output in highly dynamic environments.

[0011] Therefore, in view of the above situation, there is an urgent need to develop a fast panoramic image stitching method based on star flash wireless communication technology to overcome the shortcomings in current practical applications. Summary of the Invention

[0012] The purpose of the present invention is to provide a method for rapid stitching of panoramic images based on star flash wireless communication technology, aiming to solve the problems mentioned in the above background technology.

[0013] The present invention is implemented as follows: a method for rapidly stitching panoramic images based on star flash wireless communication technology, comprising the following steps: S1. Use Zhang Zhengyou calibration method to calibrate the intrinsic parameters and distortion coefficients of the camera, and combine it with the perspective transformation calibration method to obtain the extrinsic parameters so that the images of each perspective are mapped to a unified top-down coordinate system; S2. Collect image data through a camera that supports the StarFlash communication protocol and transmit the image data to the vehicle body domain controller based on the StarFlash communication technology; S3. After dedistortion and perspective transformation of the transmitted image, the local pyramid fusion strategy is used to fuse the image: the overlapping areas of the images are marked with binary masks, the marked overlapping areas are fused with multi-bands, the original image data is reused for the non-overlapping areas, and the merged panoramic image is finally output.

[0014] A further technical solution is that in step S1, when the Zhang Zhengyou calibration method is used, multiple sets of calibration images containing known geometric patterns are taken, the feature points of the geometric patterns in the images are detected, the homography matrix is ​​calculated, and the intrinsic parameters and distortion coefficients of the camera are solved based on the homography matrix, where the intrinsic parameters include focal length and principal point coordinates.

[0015] A further technical solution, in step S1, when obtaining external parameters in combination with the perspective transformation calibration method, includes: A unified vehicle bird's-eye view coordinate system is established, the dedistorted image is mapped to the bird's-eye view coordinate system, and a projection matrix from the camera coordinates to the bird's-eye view coordinate system is calculated. The projection matrix is ​​determined by matching a preset source point in the image with a target point in the bird's-eye view coordinate system.

[0016] According to a further technical solution, in step S2, when transmitting image data based on the Star Flash communication technology, the camera is equipped with an SLB module; Among them, the camera module collects image data through the SLB module T node, the body domain controller receives image data through the SLB module G node, and the image data is transmitted through the SLB communication mechanism.

[0017] According to a further technical solution, in step S3, when performing dedistortion processing on the image, the geometric shape of the image is restored based on the intrinsic parameters and distortion coefficients calibrated in step S1, and the nonlinear distortion of the fisheye lens is eliminated.

[0018] A further technical solution, in step S3, before performing local pyramid fusion on the image, further includes: The image is spatially divided into a central area and a stitching area. The central area is directly mapped to the panoramic image through geometric projection, and the stitching area is the overlapping area of ​​adjacent camera images.

[0019] In a further technical solution, in step S3, performing multi-band fusion on the overlapping area includes: S31. Construct a Gaussian pyramid and a Laplacian pyramid for the images in the overlapping area to extract image details at different frequency levels; S32, performing weighted averaging on the image details at each frequency level according to a preset weight coefficient; S33. Combining the weighted results of each frequency level to obtain a fused overlapping area image.

[0020] According to a further technical solution, in step S31 , a Gaussian pyramid is generated by gradually downsampling an image and applying Gaussian filtering, and a Laplacian pyramid is generated by calculating the difference between a current layer Gaussian image and an upper layer Gaussian image after upsampling.

[0021] According to a further technical solution, the camera includes cameras distributed in four directions: front, left, right, and rear of the vehicle, and the image data is a four-channel surround view image.

[0022] The present invention provides a method for rapidly stitching panoramic images based on star flash wireless communication technology, which has the following beneficial effects: By adopting the Zhang Zhengyou calibration method and perspective transformation calibration technology, high-precision calibration and unified coordinate mapping of the camera's internal and external parameters are achieved, significantly improving the geometric accuracy of image registration and the stability of the system in complex environments; at the same time, a local pyramid fusion strategy is introduced to dynamically fuse only the overlapping areas of the images, effectively solving problems such as edge blur, chromatic aberration, and stitching lines, and significantly reducing computing resource consumption; combined with the high bandwidth and low latency advantages of Star Flash communication technology, image transmission efficiency is guaranteed, and the average processing delay of the system is controlled within 35ms, meeting the strict real-time requirements of scenarios such as intelligent driving and automatic parking.

[0023] This solution is not only suitable for standard four-way surround view systems, but also has good scalability and can be applied to panoramic vision systems that integrate multiple cameras and heterogeneous sensors, providing efficient, stable and practical technical support for the development of intelligent vehicle perception technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is the overall block diagram of the rapid stitching of panoramic images based on Star Flash wireless communication technology; Figure 2 This is a schematic diagram of the actual overhead coordinate system constructed manually; Figure 3 Schematic diagram of dedistorted image; Figure 4 Schematic diagram of boundary fusion preprocessing. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] The specific implementation of the present invention is described in detail below with reference to specific embodiments.

[0027] In a multi-camera system, the calibration accuracy of each camera directly impacts image registration and stitching quality. Traditional calibration methods suffer from instability in complex lighting or dynamic environments. During image fusion, direct stitching is prone to artifacts such as stitching lines, blurred edges, and color inconsistencies. Traditional linear fusion strategies struggle to simultaneously achieve a smooth transition between low-frequency information and the effective preservation of high-frequency details.

[0028] To address the problems of image distortion and stitching defects in 360-degree panoramic stitching, this paper proposes a complete solution covering key links such as camera calibration, perspective transformation and image fusion.

[0029] like Figure 1 As shown, an embodiment of the present invention provides a method for quickly stitching panoramic images based on star flash wireless communication technology, including the following steps: S1. Use Zhang Zhengyou calibration method to accurately calibrate the camera's intrinsic parameters and distortion coefficients, and combine it with the perspective transformation calibration method to complete the acquisition of extrinsic parameters to ensure that images from each perspective can be accurately mapped to a unified overhead coordinate system.

[0030] During the calibration phase, the Zhang Zhengyou calibration method was used to accurately calibrate the intrinsic parameters and distortion coefficients of the Star Flash camera, and the extrinsic parameters were obtained by combining the perspective transformation calibration method. This method ensures the accurate mapping of the image from the original perspective to the top view, significantly improving the geometric accuracy and overall consistency of the panoramic stitching.

[0031] It should be noted that the Zhang Zhengyou calibration method is a camera calibration method widely used in the field of computer vision. It mainly calculates the internal parameters and distortion coefficients of the camera by taking a series of pictures of known geometric patterns (usually a checkerboard).

[0032] The basic steps include: using a planar chessboard with known dimensions as a calibration plate; taking multiple images containing the chessboard from different angles and positions; automatically detecting the positions of the chessboard corners in each image; using the correspondence between the corner coordinates to calculate the homography matrix at each viewing angle; based on the homography matrix, using a nonlinear optimization algorithm to solve the camera's intrinsic parameters (such as focal length, principal point coordinates, etc.), extrinsic parameters (such as rotation and translation vectors), and radial and tangential distortion coefficients.

[0033] The main advantages are: no complex equipment is required, only a printed chessboard is needed for calibration; it can accurately estimate the internal parameters and distortion coefficients of the camera, providing high-quality data support for subsequent image processing; it is suitable for all types of lenses, whether it is a normal lens, a wide-angle lens, or even a fisheye lens, and can be effectively calibrated through appropriate adjustments.

[0034] Zhang Zhengyou's calibration method greatly simplifies the camera calibration process and improves the accuracy and reliability of the calibration results. As a result, it has been widely used in fields such as robotic navigation, augmented reality, and 3D reconstruction. In particular, it helps achieve precise camera calibration in automotive surround view systems, ensuring the quality and accuracy of panoramic image stitching.

[0035] S2: A camera that supports the Star Flash communication protocol collects image data and efficiently transmits the video signal to the vehicle body domain controller through the SLB (Scalable Low-latency Broadcast) communication mechanism.

[0036] S3. In the image fusion stage, a local pyramid fusion strategy is introduced. By constructing a binary mask image to mark the overlapping area to be fused, a multi-band fusion algorithm is applied within this area to achieve an organic combination of high-frequency detail preservation and low-frequency information smoothing, ultimately outputting a high-quality panoramic stitching image.

[0037] To address the high computational complexity and memory usage commonly encountered in existing 360-degree surround view systems, this paper proposes a local pyramid fusion method. Unlike traditional multi-band fusion techniques, which construct Gaussian and Laplacian pyramids for the entire image and then perform weighted fusion layer by layer, this method dynamically generates mask images to accurately identify overlapping regions requiring fusion. These regions typically contain high-frequency details such as edge textures that vary due to perspective differences.

[0038] For the marked key areas, a traditional multi-band fusion strategy is used to extract and fuse high-frequency information. For non-overlapping areas or areas dominated by low-frequency information, the original image data is directly reused to avoid unnecessary complex calculations. This approach effectively reduces overall computational workload and memory overhead while preserving the clarity and visual consistency of key details.

[0039] Through this strategy, the present invention not only achieves high-quality image fusion but also significantly reduces system resource consumption. This reduces the processing burden on the vehicle's hardware and improves the system's real-time responsiveness. Furthermore, it enhances the overall visual experience, making the output 360-degree surround view more natural and coherent.

[0040] This local pyramid fusion method provides an efficient and practical image stitching solution for vehicle-mounted panoramic vision systems, which can better meet the multiple requirements of current smart cars for image quality, system stability and real-time processing.

[0041] In summary, the image stitching method proposed in this invention effectively takes into account the system's comprehensive requirements for real-time performance, stability, and image detail representation capabilities, and provides reliable technical support for application scenarios such as intelligent assisted driving and automatic parking.

[0042] also, Figure 1 The overall process and system architecture of rapid in-vehicle panoramic image stitching based on Star Flash (wireless communication) technology are demonstrated. The details are as follows: Hardware components: Includes four camera modules for the front, left, right, and rear sides. Each camera module is equipped with an SLB module (T node); the vehicle body domain controller is equipped with an SLB module (G node).

[0043] Communication method: Data transmission is achieved through Star Flash wireless communication technology, in which the SLB module (T node) of the camera module serves as the data transmitter, and the SLB module (G node) of the vehicle body domain controller serves as the data receiver. The SLB communication mechanism is used to complete efficient interaction of image data.

[0044] Image stitching process: a) Start (start the panoramic image stitching system); b) Reading four-channel surround view images: Camera modules in four directions collect image data from their respective perspectives and transmit it to the vehicle body domain controller via the SLB module (T-node); c) Preprocessing: Dedistort the four-channel image based on the calibration parameters to eliminate geometric distortions such as those caused by fisheye lenses. A perspective transformation is performed based on the calibration parameters, and the dedistorted image is mapped to a unified bird's-eye coordinate system. d) Fusion Processing: Boundary fusion preprocessing is performed to prepare for subsequent processing of overlapping areas. Binary masks are used to mark overlapping areas between adjacent camera images, accurately locating the areas to be fused. Multi-band fusion is performed on the marked overlapping areas to ensure a smooth transition and no noticeable stitching lines.

[0045] e) Output and end: The fused images are merged into a complete panoramic image and output, and the process ends.

[0046] The entire image stitching process closely integrates Star Flash communication technology and image processing algorithms. Through the collaboration of SLB modules (T-node and G-node), low-latency data transmission is achieved, providing a foundation for fast stitching. The process covers the entire chain from image acquisition and preprocessing to fused output, highlighting the technical logic of "high-precision calibration → dedistortion and perspective transformation → local fusion", ultimately achieving the efficient generation of high-quality panoramic images.

[0047] In the specific implementation, during the calibration stage, a camera at a fixed position first takes multiple sets of calibration images, and the Zhang Zhengyou calibration method is used to accurately calibrate the camera's internal parameters (such as focal length, principal point coordinates, etc.) and lens distortion coefficients.

[0048] Then, the perspective transformation calibration method is used to obtain the external parameters of the camera. Specifically, a unified vehicle-viewing coordinate system is established, such as Figure 2 As shown in Figure 1. The rectified image from each camera is mapped to the corresponding position in this coordinate system, and the projection matrix from camera coordinates to ground coordinates is calculated. The red dots (around the perimeter) serve as the source points in the perspective transformation, and the blue area (at the four corners) represents the area where the calibration plate is located.

[0049] Since the fisheye lens has a large nonlinear distortion, the straight lines in the image will be obviously curved. Therefore, before performing the perspective transformation, the original image must be dedistorted. The geometric shape of the image is restored using the calibrated internal parameters and distortion parameters, and the output is as follows: Figure 3The dedistorted image is shown. Based on this, the user can manually select four key points through a graphical interface. These points correspond to the source points in the bird's-eye view coordinate system. Each camera also presets a set of target points, which define the desired location in the bird's-eye view, typically forming a rectangular area. By matching the source and target points, a perspective transformation matrix is ​​calculated. This matrix maps the current image space to the bird's-eye view space, thereby converting the image from the camera's perspective to the bird's-eye view.

[0050] For the stitching method, the system collects image data from four star-flash cameras located in front of the vehicle: left, right, and rear. Each camera image is calibrated and compensated to generate a corresponding dedistorted image. This step ensures optimal visual quality in the central area of ​​the image and serves as the foundation for subsequent fusion processing.

[0051] Before entering the image fusion process, the image is first divided into a central area and a stitching area. The central area has the best de-distortion effect and can be directly mapped to the final panoramic image through geometric projection, preserving detailed information, such as Figure 4 However, there are overlapping areas between adjacent cameras. Due to differences in imaging angles and optical properties, direct stitching is prone to problems such as edge discontinuity and brightness and color difference.

[0052] To address these issues, the present invention employs a local pyramid fusion method. This method dynamically generates binary masks, processing only the overlapping regions that require fusion and skipping redundant calculations in non-overlapping regions, significantly reducing memory usage and computational complexity. Specifically, image details are extracted and reconstructed at different frequency levels, and then weighted averages are performed on the images at each scale based on pre-calculated weight coefficients. Ultimately, the results at each scale are combined into a seamless output image, effectively avoiding the issues of color aberration, uneven brightness, and blurred edges that can arise from direct stitching. This method also significantly reduces the memory requirements and computational complexity of the image stitching process, making it more suitable for the real-time processing needs of on-board equipment.

[0053] The local pyramid fusion method is an efficient fusion strategy based on an image pyramid structure. It aims to eliminate seams in stitched images (by fusing only overlapping regions of the images, generating a binary mask to identify the regions to be fused, and constructing a Gaussian pyramid for the overlapping regions), while balancing the preservation of high-frequency details with smooth transitions in low-frequency regions. Its core process is as follows: First, we need to detect the overlapping areas of the diagonal area and input the image of the diagonal area to be stitched. and , perform geometric alignment on the two images to ensure that the overlapping areas are aligned. The coordinate range of the overlapping area is calculated by the homography matrix, the homography matrix is a The projection transformation matrix is ​​used to describe the mapping relationship between the source image plane and the target image plane. Its mathematical expression is: (1) in, is the homogeneous coordinate of the source image, is the homogeneous coordinate of the target image; it is solved by combining the least squares method with the image matching points . Through the homography matrix Projection to The coordinate system and generate the corresponding binary mask , and mark the overlapping pixels. Only the overlapping areas are used to construct the mask and image pyramid, while the non-overlapping areas are directly taken from the original image pixels to avoid redundant calculations.

[0054] Image pyramids include Gaussian pyramids and Laplacian pyramids. The Gaussian pyramid is a core tool for multi-scale representation in image processing. By gradually downsampling the image and applying Gaussian filtering, a series of images with decreasing resolution are generated. The steps to build a Gaussian pyramid are as follows: First, the image is processed using a Gaussian filter to suppress high-frequency noise and details in the image, avoiding aliasing during subsequent sampling. The original image is convolved with a Gaussian kernel to blur the details. The weight distribution of the Gaussian kernel conforms to the normal distribution (two-dimensional normal distribution function), and the mathematical expression is: (2) Among them, is the standard deviation, which determines the blurriness of the Gaussian kernel; is the coordinate of the pixel within the kernel relative to the kernel center. Downsampling is used to reduce the image size, generating a low-resolution version that forms the next level of the pyramid. Interlaced sampling is performed on the blurred image to reduce the size to 1 / 4 of the original image. The mathematical expression is: (3) in, For the layer images, For the Repeat the above operation to generate multiple levels of low-resolution images to form a complete pyramid structure (i.e. The coordinates in the layer image are pixels to determine the The coordinates in the layer image are , thereby completing image downsampling and building a hierarchical structure of the Gaussian pyramid). When the image size is smaller than the preset threshold, or reaches the specified number of layers Stop when The dimensions of the layer image are: (4) in, and They represent the first The width and height of the layer image. As the image size increases (the number of layers increases), the image undergoes multiple downsampling, and the width and height will continue to decrease. and They represent the width and height of the 0th layer of the Gaussian pyramid (i.e., the original input image), which are the basic sizes for constructing the entire pyramid. The sizes of subsequent layers are scaled based on the original image size.

[0055] By downsampling the high-level image of the Gaussian pyramid Zoom in and low-level The same size is used and the image size is enlarged by 2 times using the interpolation method. Gaussian blur slightly blurs the enlarged image to eliminate the jagged edges caused by interpolation (the first Layer Image Result after upsampling operation): (5) in, represents convolution, is the Gaussian kernel, Indicates interpolation and amplification. Then extract the current layer The high-frequency information between the upsampled high-level image is calculated using the difference formula: (6) in, It is Layer Laplacian image. Then from the bottom layer of the Gaussian pyramid Start by calculating the difference layer by layer until the second highest layer The highest level of the Laplace pyramid is the top level of the pyramid .

[0056] After that, the overlapping area is extracted. Only the overlapping area marked by the mask is processed, and the non-overlapping area is skipped directly. The following is the overlapping area extraction formula: (7) Extract overlapping areas and After that, the two images are geometrically aligned to ensure that the overlapping areas are registered. Then the weight mask is constructed and the fusion weight image is read. Then the pyramid is constructed for the image and mask. and Constructing a Laplacian pyramid and , construct a Gaussian pyramid for the weight mask For each layer , calculate the fused Laplace coefficient: (8) Low-frequency layer, mid-high layer Larger, wider weight mask transition, smoothing lighting and color differences, high-frequency layers and low-frequency layers Smaller, narrower weight mask transition, retaining edge and texture details, and finally reconstructing the image by inverse Laplace transform, from the top layer Start by upsampling and stacking layer by layer to reconstruct the final image: (9) Finally, the overlapping area image and the non-overlapping area image are merged to obtain the complete image.

[0057] To verify the effectiveness of this invention, the following experiments were conducted: On an i7 9700K processor, the average computational latency reached 35ms, meeting the daily demands of in-vehicle displays. The experimental results demonstrate that this solution significantly improves processing efficiency while maintaining visual quality, meeting the real-time demands of vehicles.

[0058] The above embodiments of the present invention provide a method for rapid panoramic image stitching based on Starflash wireless communication technology, which provides a complete solution to the key technical problems of multi-camera systems in terms of calibration accuracy, image fusion quality, and real-time performance. The method has the following significant advantages: 1) We use the Zhang Zhengyou calibration method to accurately calibrate the camera's intrinsic parameters and distortion coefficients, and combine it with a perspective transformation calibration method to obtain extrinsic parameters, ensuring that each camera image is accurately mapped to a unified bird's-eye view coordinate system. This method maintains excellent stability and adaptability in complex lighting and dynamic environments, effectively improving the geometric accuracy of image registration.

[0059] 2) A local pyramid fusion strategy is introduced to dynamically generate binary masks to extract and fuse high-frequency information only in overlapping areas. This avoids the problems of blurred edges, obvious color differences, and prominent stitching lines that exist in traditional image fusion. It achieves an organic unity of smooth transition of low-frequency information and preservation of high-frequency details, making the output image more natural and coherent.

[0060] 3) Compared with the traditional global pyramid fusion method, the local pyramid fusion strategy proposed in this paper only processes the key overlapping areas and skips the redundant calculations in the non-overlapping areas, which significantly reduces the memory usage and computational complexity and is more suitable for the real-time processing requirements of vehicle embedded systems.

[0061] 4) Leveraging the high-bandwidth, low-latency characteristics of Star Flash wireless communication technology, efficient transmission of image data from the camera to the vehicle domain controller is achieved. Combined with a lightweight image processing process, the overall system's average processing delay is controlled within 35ms, meeting the stringent requirements for real-time image display and feedback in scenarios such as intelligent driving and automatic parking.

[0062] 5) This invention is not only applicable to standard four-way surround view systems, but can also be flexibly expanded to panoramic vision systems that integrate multiple cameras and heterogeneous sensors, providing a solid technical foundation for the development of future intelligent vehicle perception systems.

[0063] In summary, the present invention has achieved significant improvements in image quality, system performance, resource utilization, etc., and can provide high-quality, low-latency, and highly stable panoramic vision support for applications such as intelligent assisted driving, automatic parking, and vehicle safety monitoring. It has broad application prospects and practical value.

[0064] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0065] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for rapid panoramic image stitching based on star flash wireless communication technology, characterized in that: The following steps are involved: S1. Use Zhang Zhengyou calibration method to calibrate the intrinsic parameters and distortion coefficients of the camera, and combine it with the perspective transformation calibration method to obtain the extrinsic parameters so that the images of each perspective are mapped to a unified top-down coordinate system; S2. Collect image data through a camera that supports the StarFlash communication protocol and transmit the image data to the vehicle body domain controller based on the StarFlash communication technology; S3. After dedistortion and perspective transformation of the transmitted image, the local pyramid fusion strategy is used to fuse the image: the overlapping areas of the images are marked with binary masks, the marked overlapping areas are fused with multi-bands, the original image data is reused for the non-overlapping areas, and the merged panoramic image is finally output.

2. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 1, characterized in that: In step S1, when using the Zhang Zhengyou calibration method, multiple sets of calibration images containing known geometric patterns are taken, the feature points of the geometric patterns in the images are detected, the homography matrix is ​​calculated, and the intrinsic parameters and distortion coefficients of the camera are solved based on the homography matrix. The intrinsic parameters include focal length and principal point coordinates.

3. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 2, characterized in that: In step S1, when obtaining external parameters by combining the perspective transformation calibration method, it includes: A unified vehicle bird's-eye view coordinate system is established, the dedistorted image is mapped to the bird's-eye view coordinate system, and a projection matrix from the camera coordinates to the bird's-eye view coordinate system is calculated. The projection matrix is ​​determined by matching a preset source point in the image with a target point in the bird's-eye view coordinate system.

4. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 1, characterized in that: In step S2, when transmitting image data based on the Star Flash communication technology, the camera is equipped with an SLB module; Among them, the camera module collects image data through the SLB module T node, the body domain controller receives image data through the SLB module G node, and the image data is transmitted through the SLB communication mechanism.

5. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 1, characterized in that: In step S3, when dedistorting the image, the geometric shape of the image is restored based on the intrinsic parameters and distortion coefficients calibrated in step S1, and the nonlinear distortion of the fisheye lens is eliminated.

6. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 5, characterized in that: In step S3, before performing local pyramid fusion on the image, the following steps are also included: Divide the image into spatial regions, namely the central region and the stitching region; The central area is directly mapped to the panoramic image through geometric projection, and the stitching area is the overlapping area of ​​adjacent camera images.

7. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 6, characterized in that: In step S3, performing multi-band fusion on the overlapping area includes: S31. Construct a Gaussian pyramid and a Laplacian pyramid for the images in the overlapping area to extract image details at different frequency levels; S32, performing weighted averaging on the image details at each frequency level according to a preset weight coefficient; S33. Combining the weighted results of each frequency level to obtain a fused overlapping area image.

8. The method for rapid panoramic image stitching based on star flash wireless communication technology according to claim 7, characterized in that: In step S31, a Gaussian pyramid is generated by gradually downsampling the image and applying Gaussian filtering; The Laplacian pyramid is generated by calculating the difference between the current layer Gaussian image and the previous layer Gaussian image after upsampling.

9. The method for rapid panoramic image stitching based on star flash wireless communication technology according to any one of claims 1 to 8, characterized in that: The cameras include cameras distributed in the front, left, right and rear directions of the vehicle; The image data is a four-channel surround view image.

Images