Method and apparatus for operating a holographic electronic rearview mirror system

Abstract

The application provides a working method and device of a holographic electronic rearview mirror. The method comprises: receiving a rearview picture and a side view picture; extracting matching feature points in the overlapping area of the rearview picture and the side view picture to determine the mapping relationship between the rearview picture and the side view picture according to the matching feature points; splicing the rearview picture and the side view picture according to the mapping relationship by using a preset algorithm to obtain a holographic image; and sending the holographic image to a display to make the display display the holographic image. The application realizes the fusion of the rearview picture and the side view picture collected by the camera to generate a holographic image, so as to create a more comprehensive driving field of view. This is helpful to improve the perception ability of the driver, reduce the blind area, and thus improve the driving safety.

Description

Technical Field

[0001] This application relates to the field of electronic rearview mirror technology, and in particular to a working method and apparatus for a holographic electronic rearview mirror system. Background Technology

[0002] An electronic rearview mirror is a car accessory that uses new technology to replace traditional optical rearview mirrors. It provides a clearer, more accurate, and more comprehensive field of vision, and is waterproof, anti-fog, and anti-glare, effectively reducing blind spots and improving driving safety. Furthermore, electronic rearview mirrors can integrate more intelligent functions, such as recording driving data, taking photos, and recording videos, providing drivers with greater convenience.

[0003] However, current electronic rearview mirrors use two small screens on the left and right to display the rear view, which cannot provide a comprehensive field of vision. In this situation, drivers may still miss some important traffic information due to blind spots, thus increasing the risk of driving accidents. Summary of the Invention

[0004] In view of this, the present application provides a method and apparatus for operating a holographic electronic rearview mirror system to solve the problem that existing electronic rearview mirrors cannot provide a comprehensive field of vision, resulting in blind spots for the driver and thus increasing driving risks.

[0005] A first aspect of this application provides a method for operating a holographic electronic rearview mirror system, comprising:

[0006] Receives rear-view and side-view images;

[0007] In the overlapping area of ​​the rear view and the side view, matching feature points are extracted to determine the mapping relationship between the rear view and the side view based on the matching feature points.

[0008] Based on the mapping relationship, the rear view and the side view are stitched together using a preset algorithm to obtain a holographic image;

[0009] The holographic image is sent to the display so that the display shows the holographic image.

[0010] A second aspect of this application provides a working device for a holographic electronic rearview mirror system, comprising:

[0011] The image receiving module is configured to receive rear-view and side-view images;

[0012] The mapping relationship determination module is configured to extract matching feature points in the overlapping area of ​​the rear view and the side view, so as to determine the mapping relationship between the rear view and the side view based on the matching feature points.

[0013] The holographic image determination module is configured to stitch the rear view and the side view images together using a preset algorithm based on the mapping relationship to obtain a holographic image;

[0014] The display module is configured to send holographic images to the display so that the display shows the holographic images.

[0015] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0016] A fourth aspect of this application provides a readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0017] The beneficial effects of this application embodiment compared with the prior art are as follows: By receiving rear-view and side-view images; extracting matching feature points in the overlapping area of ​​the rear-view and side-view images to determine the mapping relationship between them; stitching the rear-view and side-view images together using a preset algorithm based on the mapping relationship to obtain a holographic image; and sending the holographic image to a display for display. This application embodiment achieves the fusion of rear-view and side-view images captured by a camera to generate a holographic image, creating a more comprehensive driving field of vision. This helps improve the driver's perception ability, reduce blind spots, and thus improve driving safety. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram illustrating an application scenario of this application embodiment;

[0020] Figure 2 This is a flowchart illustrating the working method of a holographic electronic rearview mirror system provided in an embodiment of this application;

[0021] Figure 3 This is a schematic diagram of the working device of a holographic electronic rearview mirror system provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] The working method and apparatus of a holographic electronic rearview mirror system according to an embodiment of this application will be described in detail below with reference to the accompanying drawings.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this application. The application scenario may include cameras 1 and 2, a display 3, and a controller (not shown in the figure). Camera 1 is a rear-view camera used to capture the rear view of the target vehicle; camera 2 is a side-view camera used to capture the side view of the target vehicle; and display 3 displays holographic images, providing the user with a comprehensive field of vision, avoiding blind spots, and reducing driving risks.

[0026] The controller receives the rear-view and side-view images captured by cameras 1 and 2, and uses a preset algorithm to stitch the rear-view and side-view images together to obtain a holographic image. The holographic image is then sent to display 3 for display.

[0027] It should be noted that the specific types, quantities, and combinations of cameras 1 and 2 and display 3 can be adjusted according to the actual needs of the application scenario, and this application embodiment does not impose any restrictions on this.

[0028] Figure 2 This is a flowchart illustrating the working method of a holographic electronic rearview mirror system provided in an embodiment of this application. Figure 2 The working method of the holographic electronic rearview mirror system is applied to the controller. For example... Figure 2 As shown, the working method of this holographic electronic rearview mirror system includes:

[0029] S201, receives rear-view and side-view images;

[0030] S202, in the overlapping area of ​​the rear view and the side view, extract matching feature points to determine the mapping relationship between the rear view and the side view based on the matching feature points;

[0031] S203, based on the mapping relationship, uses a preset algorithm to stitch together the rear view and the side view to obtain a holographic image;

[0032] S204, The holographic image is sent to the display so that the display shows the holographic image.

[0033] Specifically, electronic rearview mirrors use digital technology to replace traditional optical rearview mirrors. They use cameras to capture real-time images of the vehicle's surroundings and display these images on an in-car screen for the driver to view. This technology provides clearer and more accurate images while integrating various intelligent functions, such as recording driving data, taking photos, and recording videos, adding more convenience and functionality to the driving experience.

[0034] While electronic rearview mirrors offer many advantages in enhancing the driving experience, they still have some shortcomings. Most electronic rearview mirror systems use two small screens on the left and right to display the rear view, failing to provide the same wide field of vision as traditional rearview mirrors. Drivers may still miss important traffic information, such as vehicles approaching from the side or objects near the rear of the vehicle, due to blind spots. Furthermore, using multiple small screens can cause information to overlap or become scattered across different screens, making it difficult for drivers to concentrate and integrate this information.

[0035] To address the shortcomings of current electronic rearview mirrors, developing a holographic electronic rearview mirror system is particularly necessary. The core advantage of a holographic electronic rearview mirror system lies in its integration of rear-view and side-view images, presenting these images to the driver through holographic technology. This technology eliminates blind spots, providing the driver with a more comprehensive and three-dimensional field of vision. Simultaneously, holographic technology helps seamlessly integrate rear-view and side-view information, avoiding information overlap and dispersion, thus helping the driver more easily obtain the necessary information. In conclusion, the development of a holographic electronic rearview mirror system can effectively address the problems existing in current electronic rearview mirrors, providing drivers with a safer and more convenient driving experience.

[0036] Furthermore, the rearview image refers to the real-time view of the area behind the vehicle captured by a rearview camera. Similar to a traditional rearview mirror, the rearview image provides a clearer and wider field of view, helping drivers more accurately assess traffic conditions behind them, such as following vehicles and oncoming traffic. With the rearview image, drivers can understand the situation behind the vehicle without turning their heads, increasing driving safety. The sideview image refers to the real-time view of both sides of the vehicle captured by a sideview camera. The sideview image is crucial for lane changes, pedestrian approach, and other situations, as this information helps drivers make safe driving decisions.

[0037] Receiving rear-view and side-view images is essential for achieving a comprehensive field of vision and effective information integration within the holographic electronic rearview mirror system. The rear-view and side-view images originate from the rearview camera and side-view camera respectively. Combining these two image sources overcomes the limitations of traditional electronic rearview mirrors, providing drivers with more accurate and comprehensive environmental information, thereby improving driving safety and convenience.

[0038] Receiving rear-view and side-view images, integrating and processing these two images before presenting them to the driver helps reduce information fragmentation, improve readability and comprehension, thereby enhancing the driver's perception of road conditions and improving driving safety.

[0039] The overlapping area refers to the region in a holographic electronic rearview mirror system where the rear-view and side-view images appear together and partially overlap. This area is within the vehicle's visual field and is typically where the fields of view of the rear-view and side-view cameras intersect, encompassing parts of the environment behind and to the sides of the vehicle.

[0040] In a holographic electronic rearview mirror system, a series of processes are required in the overlapping area to achieve the fusion of the rear-view and side-view images. The overlapping area is the region where the rear-view and side-view images appear together, and the information from both can be fused to provide a more complete visual effect.

[0041] The system analyzes the rear-view and side-view images to find points with significant common features between the two; these points are called feature points. These feature points may be corners of objects, textured areas, etc., and can be accurately matched in the two images.

[0042] By matching feature points, the system can establish a mapping relationship between the rear-view and side-view images. This means the system knows how to associate a specific location in the rear-view image with a corresponding location in the side-view image. A mapping relationship, in image processing, refers to the relationship between a location in one image and a corresponding location in another. In a holographic electronic rearview mirror system, this mapping relationship is used to match and correspond image elements in the rear-view image with their corresponding image elements in the side-view image, thereby achieving information fusion and stitching between the two.

[0043] Specifically, with the help of mapping relationships, the system can determine which specific region in the side view corresponds to a particular area (such as a pixel) in the rear view. This correspondence is typically established based on image processing techniques such as feature point matching and geometric transformations.

[0044] By establishing mapping relationships, the system can fuse information from two perspectives into a single image, ensuring the continuity and accuracy of the image. For example, in the overlapping area of ​​the rear view and the side view, the mapping relationship can align an object in the rear view with the corresponding object in the side view, thus forming a coherent holographic image.

[0045] Mapping relationships play a crucial role in holographic electronic rearview mirror systems, seamlessly integrating information from different perspectives to provide drivers with a more complete and immersive visual experience. By effectively establishing and applying mapping relationships, the system can overcome the differences between rear-view and side-view images, creating more practical and accurate driving assistance tools.

[0046] Based on the established mapping relationship, the system can determine how each image element in the rear view corresponds to an image element in the side view. This means the system knows how to align similar areas in the two views. Using a preset algorithm, the system transforms and adjusts the rear and side views to seamlessly connect them in overlapping areas. This process may involve image rotation, scaling, translation, and other transformations to ensure a natural visual transition between the two views.

[0047] During the stitching process, the system also considers how to blend and transition the images to ensure a smooth transition within overlapping areas without obvious boundaries. This can be achieved through techniques such as gradation and blending, making the presentation of holographic images more continuous.

[0048] By applying a preset algorithm, the rear-view and side-view images are stitched, merged, and transitioned within the overlapping area to generate a holographic image. This holographic image will include information about the rear and sides of the vehicle, providing the driver with a more comprehensive and three-dimensional visual experience.

[0049] Through this process, the holographic electronic rearview mirror system effectively integrates information from two different perspectives, creating a more comprehensive and practical driving assistance tool. This holographic imaging not only helps drivers obtain more accurate environmental information but also enhances driving safety and the driving experience.

[0050] In a holographic electronic rearview mirror system, the stitched and fused holographic image is transmitted to a display inside the vehicle. This transmission process can be achieved through wired or wireless connections, ensuring that the holographic image is delivered to the display in a timely and stable manner.

[0051] Once the holographic image is transmitted to the display, the system uses the display's technology and performance to project the holographic image onto its surface. The display can be a central control panel inside the vehicle or a dedicated screen for displaying holographic images, depending on the system design.

[0052] Through the display, the driver can see a three-dimensional, comprehensive holographic image that integrates information from the rear and sides of the vehicle. This presentation method eliminates blind spots and information dispersion problems that may exist in traditional rearview and side mirrors, providing the driver with more accurate and comprehensive visual feedback.

[0053] According to the technical solution provided in this application, a rear-view image and a side-view image are received; matching feature points are extracted in the overlapping area of ​​the rear-view and side-view images to determine the mapping relationship between the rear-view and side-view images; based on the mapping relationship, the rear-view and side-view images are stitched together using a preset algorithm to obtain a holographic image; the holographic image is then sent to a display for display. This application embodiment achieves the fusion of rear-view and side-view images captured by a camera to generate a holographic image, creating a more comprehensive driving field of vision. This helps improve the driver's perception ability, reduce blind spots, and thus improve driving safety.

[0054] In some embodiments, the method further includes: determining the installation position coordinates of each camera; determining the field of view coordinates of each camera based on the installation position coordinates and the appearance features of the target vehicle; establishing a field of view spatial model corresponding to each camera based on the field of view coordinates; determining the overlapping field of view coordinates between the rear-view field of view spatial model corresponding to the rear-view camera and the side-view field of view spatial model corresponding to the side-view camera based on the geometric relationship between the field of view spatial models; and determining the overlapping area based on the overlapping field of view coordinates.

[0055] Specifically, since rear-view and side-view images come from different perspectives, both may contain important information about the environment behind and to the sides. Identifying the overlapping area ensures that a portion of the two images is shared, thus enabling them to be merged.

[0056] The overlapping region provides crucial information for establishing the mapping relationship between the rear-view and side-view images. By identifying this region, the system can align corresponding parts of the two images, laying the foundation for subsequent image stitching and processing. Within this region, the system executes an image stitching algorithm to merge the overlapping parts of the two images, eliminating potential boundary issues and creating a more coherent and smooth holographic image.

[0057] To establish the mapping relationship between the rear-view and side-view images, it is first necessary to accurately determine the installation coordinates of each camera (rear-view camera and side-view camera). These coordinates indicate the specific location of the cameras on the vehicle.

[0058] The target vehicle's exterior features can include its size, shape, and body curvature. These features are crucial for calculating the camera's field of view, as the camera's field of view can be obstructed and limited by the vehicle's exterior. By combining the camera's installation location with the target vehicle's exterior features, the system can utilize geometric and 3D computational methods to calculate the field of view of each camera.

[0059] Field of view coordinates refer to the spatial coordinates of the image area that a camera can capture. In a holographic electronic rearview mirror system, field of view coordinates describe the range of the external environment that each camera can see from its installation position. This range can be a region in three-dimensional space, typically represented by a rectangular or circular planar area.

[0060] Furthermore, after determining the field-of-view coordinates of each camera, the system uses these coordinates to build a field-of-view spatial model. These models are geometric shapes in three-dimensional space that represent the field of view of each camera.

[0061] Because of the geometric relationships between the field-of-view models of the rear-view and side-view cameras, which may involve the relative positions, orientations, and angles of the cameras, the system can determine the overlapping field-of-view coordinates between the field-of-view models of the rear-view and side-view cameras by analyzing these geometric relationships.

[0062] By analyzing the geometric relationships, the system can calculate the area where the rear-view and side-view images overlap in three-dimensional space. The coordinates of this overlapping area are called the overlapping field-of-view coordinates, which describe the intersection of the rear-view and side-view fields of view in space.

[0063] By calculating the coordinates of the overlapping fields of view, the system can accurately determine the overlapping area between the rear-view and side-view images. This area is crucial for subsequent image stitching and processing, as it contains information from different perspectives and provides a foundation for generating holographic images.

[0064] By establishing a field-of-view spatial model, determining geometric relationships, and calculating the coordinates of overlapping fields of view, the system can locate the overlapping areas of the rear-view and side-view images. This process provides crucial input for subsequent image processing and information fusion, ensuring that the holographic electronic rearview mirror system can generate accurate and comprehensive driver assistance images, thereby enhancing driver safety and the driving experience.

[0065] In some embodiments, extracting matching feature points includes: determining preset reference marker points in the overlapping area; obtaining the pixel coordinates of the reference marker points in the rear view and side view using a camera calibration algorithm; and determining matching feature points based on the pixel coordinates.

[0066] Specifically, extracting matching feature points helps the system establish a mapping relationship between the rear-view and side-view images. These feature points, acting as a bridge between the two images, help determine the correspondence between them, thereby enabling image stitching and fusion.

[0067] By matching feature points, the system can find common feature points in two images, where the same object or area is seen from different perspectives. Merging these feature points creates a more accurate and comprehensive driver assistance image.

[0068] In summary, extracting matching feature points is a crucial step in establishing a connection between the rear-view and side-view images, enabling information fusion and image stitching. Through this process, the holographic electronic rearview mirror system can generate more accurate and comprehensive driver assistance images, providing drivers with greater driving safety and convenience.

[0069] Furthermore, in the holographic electronic rearview mirror system, the preset reference markers in the overlapping area, the camera calibration algorithm, and the matching feature points work together to ensure that the rear view and the side view can be accurately fused and mapped.

[0070] The preset reference markers are key points located within the overlapping area that can be accurately identified in both the rear-view and side-view images. These are typically selected from objects that have distinct features in both images, such as protruding parts of a vehicle or road markings. These markers form the basis for determining the mapping relationship, and their selection must ensure good recognizability in both images.

[0071] Camera calibration algorithms are mathematical model-based processes that calibrate the camera's intrinsic and extrinsic parameters by collecting pixel coordinates of objects at known locations and their real-world coordinates. A calibrated camera can map pixel coordinates to actual physical coordinates, thus establishing a correspondence between the image and the real world. Calibration algorithms can consider various factors, including lens distortion and perspective transformation, to ensure accurate mapping.

[0072] After camera calibration, the pixel coordinates of preset reference markers in the rear and side views can be obtained. These pixel coordinates can be converted into actual physical coordinates using the camera's calibration parameters. Then, by finding corresponding markers in the two views, matching feature points can be determined. These feature points have the same physical location in both views, forming the basis for establishing a mapping relationship.

[0073] In summary, the pre-set reference markers within the overlapping area, the camera calibration algorithm, and the matching feature points work together to ensure the accuracy of information fusion and mapping between the rear-view and side-view images. This process is a key step in enabling the holographic electronic rearview mirror system to provide accurate and comprehensive driver assistance images, creating a safer and more convenient driving experience for the driver.

[0074] In some embodiments, determining the mapping relationship between the rear view and the side view based on matching feature points includes: constructing a system of linear equations between the rear view and the side view based on the pixel coordinates of the matching feature points; solving the system of linear equations using a preset linear transformation algorithm to obtain a transformation matrix based on pixel coordinates; and determining the mapping relationship based on the transformation matrix.

[0075] Specifically, by matching feature points, the pixel coordinates of these matching feature points in the rear-view and side-view images can be obtained. These matching feature points have the same physical location in both images. To establish a mapping relationship, a system of linear equations can be constructed using the pixel coordinates of these matching feature points. Each equation represents the correspondence between a feature point in the rear-view and side-view images.

[0076] Using a pre-defined linear transformation algorithm, the constructed system of linear equations can be solved to obtain a transformation matrix based on pixel coordinates. This transformation matrix describes the mapping relationship between two frames, mapping pixel coordinates in one frame to corresponding positions in another frame.

[0077] Furthermore, by solving the system of linear equations to obtain the transformation matrix, the mapping relationship between the rear-view and side-view images can be established. This mapping relationship can describe the translation, rotation, scaling, and other transformation relationships between the two images, thereby enabling the fusion of the rear-view and side-view images into a single holographic image.

[0078] By using linear equations and linear transformation algorithms, the transformation matrix can be mathematically solved accurately. This matrix describes the transformation pattern between two images. This makes subsequent image stitching and fusion more accurate and reliable, providing drivers with more complete and realistic driving assistance images. The entire process acts as a bridge in the holographic electronic rearview mirror system, seamlessly integrating information from different images and improving the visibility and safety of the driving experience.

[0079] In some embodiments, according to the mapping relationship, stitching the rear-view image and the side-view image together using a preset algorithm to obtain a holographic image includes: determining the rotation angle and translation distance of the side-view image relative to the rear-view image based on a transformation matrix; determining the relative position information of the side-view image and the rear-view image based on the rotation angle and translation distance; stitching the overlapping areas together using image fusion technology based on the relative position information to generate an overlapping area image; determining the non-overlapping area images corresponding to the non-overlapping areas of the rear-view image and the side-view image based on the relative position information; and determining the holographic image based on the overlapping area image and the non-overlapping area image.

[0080] Specifically, in a holographic electronic rearview mirror system, the rear view and side view are stitched together according to the mapping relationship to create a whole, seamless holographic image, providing the driver with clearer and more comprehensive information about the vehicle's surroundings.

[0081] Based on the obtained transformation matrix, rotation angle and translation distance information can be extracted. This information describes the position and orientation transformation of the side view relative to the rear view. Using these parameters, the position of the side view within the rear view can be accurately determined.

[0082] Based on the rotation angle and translation distance, the relative position information of the side view and the rear view can be determined. This includes the offset, rotation, and scaling information of the side view in the rear view, thereby ensuring that the two views can be accurately aligned and blended.

[0083] Furthermore, within the overlapping area, the side-view and rear-view images are stitched together using image fusion technology based on relative positional information. Image fusion technology can employ various methods, such as pixel-level fusion and hybrid fusion, to ensure a smooth transition between the two images within the overlapping area and eliminate discontinuities.

[0084] Based on relative position information, image fusion technology processes pixels within overlapping areas to create a gradual transition between the two images. This may involve pixel blending, transparency adjustments, and color calibration. Through these operations, the system can generate an overlapping image that visually presents a smooth transition between the two images.

[0085] Image fusion technology needs to excel in both real-time performance and smoothness to ensure that drivers experience no discontinuities or jumps while viewing the image. This requires the system to process and synthesize images quickly, ensuring that drivers receive a coherent and stable holographic image.

[0086] After identifying the overlapping image areas, the non-overlapping areas of the rear-view and side-view images can be determined based on their relative position information; these are the parts that do not appear simultaneously in both images. Once the non-overlapping areas are identified, they can be mapped to the same image coordinate system based on their relative position information. This involves mapping pixel positions from one image to the other using parameters such as rotation and translation to ensure accurate correspondence between the non-overlapping areas.

[0087] By combining overlapping and non-overlapping area images, a complete holographic image can be generated. The overlapping and non-overlapping area images are blended together to present a coherent and seamless picture, showing panoramic information about the rear and sides of the vehicle.

[0088] Through the above steps, the holographic electronic rearview mirror system can accurately fuse the rear-view and side-view images to generate a realistic and accurate holographic image. This process not only provides richer driving assistance information but also enhances the driver's visibility and safety, significantly improving the driving experience.

[0089] In some embodiments, sending a holographic image to a display includes: determining a preset coordinate system; mapping the holographic image to the preset coordinate system to generate a mapped image; and sending the mapped image to the display.

[0090] Specifically, the process of sending holographic images to the display needs to take into account the driver's individual needs and habits. Different drivers may prefer different rearview mirror viewing angles and field of view. Therefore, holographic electronic rearview mirror systems can be designed to be adjustable to suit different driver preferences.

[0091] Drivers may adjust the viewing angle and field of view of their rearview camera to better suit their personal preferences, driving style, and visual habits. For example, some drivers may focus more on the sides and rear of the vehicle and therefore prefer to focus the rearview camera on the side angle. Others may focus more on the center and rear of the vehicle and prefer to focus the rear angle. Adjusting the viewing angle and field of view allows drivers to better obtain the information they need in different driving situations.

[0092] The system can personalize and preset a coordinate system to meet the driver's individual needs. This personalized preset coordinate system is a virtual coordinate system that can locate specific areas in the image based on the driver's adjustments. This coordinate system can be viewed as a "user-customized viewing angle," which affects the projection of the holographic image onto the display. When the driver adjusts the viewing angle and field of view of the rearview image, they are actually modifying the parameters of the personalized preset coordinate system.

[0093] Furthermore, drivers may have personalized needs and habits in their daily driving, such as their level of attention to rear information, driving style, and preferred observation angles. Therefore, a holographic electronic rearview mirror system allows drivers to define a personalized preset coordinate system—a virtual observation frame used to guide the presentation of the holographic image. By setting the preset coordinate system, drivers can determine the areas and perspectives they are interested in.

[0094] Once the preset coordinate system is established, the system maps the holographic image onto the specified coordinate system based on these parameters. This involves transforming the holographic image by rotation, scaling, and translation to suit the driver's desired viewing angle. In this way, the system ensures that the holographic image displayed on the monitor matches the driver's individual needs.

[0095] The mapped image is generated in a preset coordinate system based on the driver's settings. This image retains the information of the holographic image but has been adjusted according to the driver's personal habits and expectations. This process ensures that the holographic image is presented on the display at the area and angle most relevant to the driver.

[0096] The generated mapped image is transmitted to a display inside the vehicle for the driver to view. Based on the driver's personalized settings, the display shows the areas the driver is most interested in, providing a customized driving perspective. This allows the driver to more easily access the information they need while driving, thereby enhancing driving visibility and safety.

[0097] In summary, by combining the generation and display of holographic images with the driver's personalized needs and habits, the holographic electronic rearview mirror system offers a more flexible and customized driving experience. This design allows drivers to adjust the rearview image according to their preferences and obtain optimal visual support in different driving scenarios, thereby improving driving convenience and comfort.

[0098] In some embodiments, when the camera is a wide-angle camera, the method further includes: acquiring a captured image from the camera; determining the distortion parameters corresponding to the camera; and using a distortion correction algorithm to eliminate the distortion of the captured image based on the distortion parameters, so as to generate a rear view and / or a side view.

[0099] Specifically, wide-angle cameras have a larger field of view, allowing them to capture more environmental information, especially behind and to the sides of vehicles. This is important for detecting and avoiding potential hazards, such as approaching vehicles, pedestrians, or other obstacles.

[0100] Wide-angle cameras are often used to provide a wider field of view, but their distortion issues can cause images to look unnatural, affecting the driver's perception and experience. The characteristics of wide-angle cameras mean they can introduce radial and tangential distortion at the edges of the image. This means that straight lines in the image may appear bent, stretched, or twisted, resulting in image distortion. If these distortions are not addressed, the generated holographic images may be inaccurate in shape and proportion, affecting the driver's judgment of the rear and side environments.

[0101] Rear and side views are crucial for driving safety. Images captured by wide-angle cameras, if not corrected for distortion, can lead to incorrect judgments by the driver, especially in edge areas. Accurate perception of other vehicles, obstacles, and traffic conditions is essential while driving; therefore, it is necessary to ensure that the images captured by cameras are accurate.

[0102] Furthermore, the captured image refers to the real-time image captured by the camera; it is the data presented by the camera lens after sensing the environment through its optical sensors. Wide-angle cameras, due to their lens characteristics, introduce specific types of distortion, such as radial and tangential distortion. The system needs to calibrate the camera, determining its distortion parameters through experimental or computational methods. These parameters describe the type, degree, and location of the distortion introduced by the camera.

[0103] Based on the camera's distortion parameters, the system applies a distortion correction algorithm to process the captured image. This algorithm is a mathematical model that eliminates distortion by adjusting the pixels in the image. For radial distortion, the algorithm transforms the pixels according to a specific pattern, making straight lines appear straighter after correction. For tangential distortion, the algorithm adjusts the pixel positions to make the image smoother after correction.

[0104] After processing by the distortion correction algorithm, distortion in the captured images is eliminated, resulting in more accurate and realistic rear-view and / or side-view images. These rear-view and / or side-view images will better reflect the actual environment, providing more reliable input for subsequent stitching, mapping, and display steps.

[0105] This process effectively corrects the images captured by the wide-angle camera, reducing or eliminating distortion. This provides accurate input for the holographic electronic rearview mirror system, helping to generate high-quality, reliable holographic images. The key to the distortion correction process is ensuring the accuracy and realism of the image, thereby improving the driver's perception and understanding of the vehicle's surroundings.

[0106] All of the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this application, and will not be described in detail here.

[0107] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0108] Figure 3 This is a schematic diagram of the working device of a holographic electronic rearview mirror system provided in an embodiment of this application. Figure 3 As shown, the working device of the holographic electronic rearview mirror system includes:

[0109] The image receiving module 301 is configured to receive rear view and side view images;

[0110] The mapping relationship determination module 302 is configured to extract matching feature points in the overlapping area of ​​the rear view and the side view, so as to determine the mapping relationship between the rear view and the side view based on the matching feature points.

[0111] The holographic image determination module 303 is configured to stitch the rear view and the side view images together using a preset algorithm based on the mapping relationship to obtain a holographic image;

[0112] Display module 304 is configured to send a holographic image to a display so that the display shows the holographic image.

[0113] In some embodiments, Figure 3 The mapping relationship determination module 302 determines preset reference marker points in the overlapping area; obtains the pixel coordinates of the reference marker points in the rear view and side view through the camera calibration algorithm; and determines matching feature points based on the pixel coordinates.

[0114] In some embodiments, Figure 3 The mapping relationship determination module 302 constructs a set of linear equations between the rear view and the side view based on the pixel coordinates of the matching feature points; solves the set of linear equations using a preset linear transformation algorithm to obtain a transformation matrix based on pixel coordinates; and determines the mapping relationship based on the transformation matrix.

[0115] In some embodiments, Figure 3 The holographic image determination module 303 determines the rotation angle and translation distance of the side view image relative to the rear view image based on the transformation matrix; determines the relative position information of the side view image and the rear view image based on the rotation angle and translation distance; stitches the overlapping areas together using image fusion technology based on the relative position information to generate an overlapping area image; determines the non-overlapping area images corresponding to the non-overlapping areas of the rear view image and the side view image based on the relative position information; and determines the holographic image based on the overlapping area image and the non-overlapping area image.

[0116] In some embodiments, Figure 3The central display module 304 determines a preset coordinate system; maps the holographic image to the preset coordinate system to generate a mapped image; and sends the mapped image to the display.

[0117] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0118] Figure 4 This is a schematic diagram of the electronic device 4 provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0119] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or different components.

[0120] The processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0121] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM of the electronic device 4. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc., equipped on the electronic device 4. The memory 402 can also include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device.

[0122] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0123] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable storage medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0124] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for operating a holographic electronic rearview mirror system, characterized in that, The holographic electronic rearview mirror system includes a camera, a display, and a controller: the camera includes a rearview camera and a sideview camera; the rearview camera is used to capture the rear view of the target vehicle; The side-view camera is used to capture side-view images of the target vehicle; The method is applied to a controller, and the method includes: Receive the rear view and the side view; In the overlapping area of ​​the rear view and the side view, matching feature points are extracted to determine the mapping relationship between the rear view and the side view based on the matching feature points; Based on the mapping relationship, the rear view and the side view are stitched together using a preset algorithm to obtain a holographic image; Sending the holographic image to the display so that the display shows the holographic image includes: mapping the holographic image to a preset coordinate system to generate a mapped image, wherein the preset coordinate system can locate a specific area in the image according to the driver's adjustment; and sending the mapped image to the display.

2. The holographic electronic rearview mirror according to claim 1, characterized in that, Also includes: Determine the installation coordinates of each of the cameras; Based on the installation location coordinates and the appearance features of the target vehicle, determine the field of view coordinates of each camera; Establish a field-of-view space model for each of the cameras based on the field-of-view coordinates; Based on the geometric relationship between the various field-of-view spatial models, determine the overlapping field-of-view coordinates between the rear-view field-of-view spatial model corresponding to the rear-view camera and the side-view field-of-view spatial model corresponding to the side-view camera; The overlapping region is determined based on the overlapping field of view coordinates.

3. The method according to claim 2, characterized in that, The extraction of matching feature points includes: Determine a preset reference marker point in the overlapping area; The pixel coordinates of the reference marker point in the rear view and the side view are obtained by using a camera calibration algorithm; The matching feature points are determined based on the pixel coordinates.

4. The method according to claim 3, characterized in that, Determining the mapping relationship between the rear view and the side view based on the matching feature points includes: A system of linear equations between the rear view and the side view is constructed based on the pixel coordinates of the matched feature points; The system of linear equations is solved using a preset linear transformation algorithm to obtain a transformation matrix based on the pixel coordinates; The mapping relationship is determined based on the transformation matrix.

5. The method according to claim 4, characterized in that, The step of stitching the rear-view image and the side-view image together using a preset algorithm based on the mapping relationship to obtain a holographic image includes: Based on the transformation matrix, determine the rotation angle and translation distance of the side view relative to the rear view; The relative position information of the side view and the rear view is determined based on the rotation angle and the translation distance. Based on the relative position information, the overlapping areas are stitched together using image fusion technology to generate an overlapping area image; Based on the relative position information, determine the non-overlapping area images corresponding to the non-overlapping areas of the rear view and the side view; The holographic image is determined based on the overlapping area image and the non-overlapping area image.

6. The method according to claim 1, characterized in that, When the camera is a wide-angle camera, it also includes: Acquire the captured images from the camera; Determine the distortion parameters corresponding to the camera; Based on the distortion parameters, a distortion correction algorithm is used to eliminate the distortion of the acquired image to generate the rear view and / or the side view.

7. A working device for a holographic electronic rearview mirror system, characterized in that, The holographic electronic rearview mirror system includes a camera, a display, and a controller. The camera includes a rearview camera and a sideview camera. The rearview camera is used to capture the rear view of the target vehicle. The side-view camera is used to capture side-view images of the target vehicle; the device includes: The image receiving module is configured to receive the rear view image and the side view image; The mapping relationship determination module is configured to extract matching feature points in the overlapping area of ​​the rear view and the side view, so as to determine the mapping relationship between the rear view and the side view based on the matching feature points; The holographic image determination module is configured to stitch the rear view and the side view together using a preset algorithm based on the mapping relationship to obtain a holographic image; The display module is configured to send the holographic image to the display so that the display shows the holographic image; including: mapping the holographic image to a preset coordinate system to generate a mapped image, the preset coordinate system being able to locate a specific area in the image according to the driver's adjustment; and sending the mapped image to the display.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

9. A readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images