Vehicle A-pillar blind spot perspective method, system and vehicle

By extracting and processing the real-life video stream information of the A-pillar blind spot of the vehicle, calculating the distance between the obstacles and the vehicle, performing three-dimensional reconstruction and delay compensation, combining the driver's eyebrow positioning information, and dynamically mapping the image, the A-pillar blind spot environment in the existing technology is solved. Identify difficulties and delay problems, achieving a visual experience with high fusion and low latency.

CN112896045BActive Publication Date: 2024-11-05ANXINTONG TECH MACAO
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110108643.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-26
Publication Date
2024-11-05
Estimated Expiration
2041-01-26

AI Technical Summary

Technical Problem

The prior art cannot effectively identify and judge the A-pillar blind spot environment during driving, which makes it difficult for the driver to identify the environmental hazard factors around the vehicle, and the display screen does not integrate with the real scene, which has obvious delays and cannot be based on the driver. Dynamic transformation of perspective.

Method used

By obtaining the real-life video stream information of the A-pillar imaging area of ​​the vehicle, extracting the first processing screen, calculating the distance and size of the obstacles and the vehicle, performing three-dimensional reconstruction, performing picture delay compensation and dynamic mapping, combining the driver's eyebrow positioning information, adjusting the image Perspective deformation realizes dynamic mapping and display of A-pillar blind spot images.

Benefits of technology

It improves the integration of the display screen and the real scene, reduces the picture delay phenomenon, adapts to the difference in perspective of drivers of different body types, provides a natural visual experience, and achieves the transparent effect of the A-pillar.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112896045B_ABST
    Figure CN112896045B_ABST
Patent Text Reader

Abstract

The present invention provides a vehicle A-pillar blind spot perspective method, which obtains real-scene video stream information of the vehicle A-pillar imaging area, extracts a first processing picture according to a sampling frequency; determines a number of obstacle images from the first processing picture, calculates the distance between the obstacle and the vehicle and the size of the obstacle, outputs a three-dimensional reconstruction picture, performs picture delay compensation calculation, obtains the driver's eyebrow center positioning information, dynamically maps the A-pillar blind spot image to the picture, and performs image effect enhancement processing to obtain the A-pillar display picture; transmits the A-pillar display picture to the A-pillar screen; provides a vehicle A-pillar blind spot perspective system, including an image acquisition module, a processing module and a display module; and provides a vehicle using the vehicle A-pillar blind spot perspective method. The present invention improves the fusion degree of obstacles at different distances and real-scene pictures on the display screen, improves the picture delay phenomenon, and dynamically maps the corresponding A-pillar display picture according to the driver's perspective. The method is simple and has a good user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle driving image display, and in particular relates to a vehicle A-pillar blind spot perspective method, system and vehicle. Background Art

[0002] There are several blind spots when driving a car, including the A-pillar blind spot, which is located on both sides directly in front of the driver. Especially when turning, it is easy for the driver to be unable to identify and judge the dangerous factors in the vehicle's surrounding environment.

[0003] In order to solve this problem, the industry has some solutions for the perspective of the A-pillar of automobiles. The mainstream technical solution is to obtain external images through a camera placed outside the car, and then cut and splice the images of the A-pillar blind area onto the screen covering the A-pillar. However, this technical solution usually does not have the ability to recognize the depth of field of the image. The image display only has flat information, which is difficult to naturally adapt to the size and display angle of the blind area object according to the driver's line of sight, and it is difficult to overlap with the real scene naturally. The combination of the scene outside the car and the display screen in the blind area of ​​the car is poor, and the natural fusion is low; and when the vehicle speed is fast, from camera acquisition to image processing to the final A-pillar display, the A-pillar display screen has a more obvious delay phenomenon in the whole process, that is, the scene has passed the driver's field of vision before it is displayed on the A-pillar screen, affecting the driver's normal judgment; and when drivers of different body shapes drive, the perspective relationship between them and the A-pillar is also different. The current technology cannot perform perspective deformation correction according to the driver's perspective. Summary of the invention

[0004] The purpose of the present invention is to overcome the shortcomings of the above-mentioned prior art and provide a vehicle A-pillar blind spot perspective method, system and vehicle, which solves the problem of imperfect ranging algorithm for obstacles at different distances outside the vehicle in the prior art, overcomes the defect of non-integration of the displayed image and the real scene image, improves the problem of obvious time delay at different vehicle speeds, and solves the problem of not being able to dynamically change the image according to the driver's perspective.

[0005] In order to achieve the above-mentioned object, in a first aspect, the present invention provides a vehicle A-pillar blind spot perspective method, comprising:

[0006] Acquire real-scene video stream information of the A-pillar imaging area of ​​the vehicle, and extract a first processing picture according to the sampling frequency;

[0007] Determine a plurality of obstacle images from the first processed picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, perform image effect enhancement processing, and obtain the A-pillar display picture;

[0008] Transmit the A-pillar display to the A-pillar screen.

[0009] Further, a first processing picture coordinate system is established in the first processing picture, and a plurality of obstacle images are determined, each of the obstacle images includes positioning information, and the positioning information is used to mark the position of the obstacle in the A-pillar imaging area;

[0010] Determining image distance information between the obstacle image and a reference line of the vehicle in a first processing screen coordinate system;

[0011] The actual distance information between the vehicle and the obstacle is determined according to the image distance information.

[0012] Furthermore, the positioning information includes pixel position information of the obstacle image in the first processing screen coordinate system, and the pixel position information includes the lower left corner coordinates (x1, y1) of the obstacle image, the width w and the height h of the obstacle image, and the rectangular bounding box of the obstacle image is determined by the x1, y1, w and h.

[0013] Furthermore, an image distance coefficient R is defined, and the image distance coefficient R is configured to be used for conversion between the image distance information and the actual distance information. The image distance coefficient R is a conventional conversion coefficient between the actual distance from the external obstacle to the vehicle determined by measurement experiments and the coordinate distance in the first processing screen.

[0014] Furthermore, the obstacle near-end distance information a is determined according to the rectangular bounding box of the obstacle image, and the actual distance information includes a vertical absolute distance A between the vehicle and the obstacle in the forward direction of the vehicle body, and the vertical absolute distance A=a*R.

[0015] Furthermore, the sampling frequency is 3 frames per second.

[0016] Furthermore, when there are overlapping areas between the rectangular bounding boxes of the plurality of obstacle images, the rectangular bounding box with the smallest obstacle near-end distance information a is selected as the conversion object.

[0017] Further, the real-scene video stream information is subjected to picture extraction according to the splicing frequency to obtain a second processed picture;

[0018] The extracted second processed images are stitched and synthesized to obtain a panoramic image;

[0019] Based on the fixed delay parameters and the real-time speed information of the vehicle, a delay-free image is captured from the panoramic image.

[0020] Furthermore, the fixed delay parameter is determined by comprehensively collecting real-scene video stream information from the camera, converting the digital-to-analog signal, and the time consumed in transmission and displaying the signal.

[0021] Furthermore, after calculating the distance between the obstacle and the vehicle, the Warp Transformation image processing algorithm is used to correct and stitch the images according to the actual distance information between each obstacle and the vehicle.

[0022] According to the positioning information of the driver's eyebrow center, the angle and position of the driver viewing the A-pillar screen are calculated, and the image is transformed using the warpPerspective method to adjust the image perspective deformation and mapping range, and dynamically map the image of the A-pillar blind spot.

[0023] Furthermore, based on the imaging area information of the vehicle's A-pillar and the actual distance information between the vehicle and the obstacle, combined with the driver's eyebrow positioning information, the real-scene video stream information is three-dimensionally reconstructed and the perspective is transformed, and the A-pillar display image is transmitted to the A-pillar screen.

[0024] Furthermore, the vehicle interior image information is obtained, the interior image information including the driver's eyebrow center positioning information and A-pillar shape information; the blind spot image area information in the first processing screen is determined by the vehicle interior image information, and the corrected blind spot image area information is sent to the A-pillar screen.

[0025] In a second aspect, the present invention provides a system applied to the above-mentioned vehicle A-pillar blind spot perspective method, comprising an image acquisition module, a processing module and a display module, wherein the image acquisition module is connected to the processing module by signal, and the processing module is connected to the display module by signal;

[0026] The image acquisition module is used to obtain real-scene video stream information of the imaging area of ​​the vehicle A-pillar and extract the first processing picture according to the sampling frequency;

[0027] The processing module is used to determine a number of obstacle images from the first processed picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, and perform image effect enhancement processing to obtain the A-pillar display picture;

[0028] The display module is used to display the A-pillar display screen.

[0029] Further, the processing module includes a distance measuring unit, a delay compensation unit and a dynamic mapping unit, the image acquisition module is connected to the distance measuring unit signal, the distance measuring unit is connected to the delay compensation unit signal, the delay compensation unit is connected to the dynamic mapping unit signal, and the dynamic mapping unit is connected to the display module signal;

[0030] The distance measuring unit is used to determine the actual distance information between the vehicle and the obstacle;

[0031] The time delay compensation unit is used to capture a delay-free picture from the panoramic picture;

[0032] The dynamic mapping unit is used to dynamically map the A-pillar blind spot image on the screen according to the driver's eyebrow center positioning information.

[0033] Furthermore, it also includes an interface module, which is used to receive the driver's eyebrow center positioning information and the car CAN data, and the car CAN data includes the vehicle's real-time speed information.

[0034] In a third aspect, the present invention also provides a vehicle applying the above-mentioned vehicle A-pillar blind spot perspective method, comprising a left camera, a right camera, a left A-pillar display screen, a right A-pillar display screen and a control system, wherein the left camera is arranged on the outside of the left A-pillar of the vehicle, and the left camera is used to obtain real-scene video stream information of the left A-pillar imaging area of ​​the vehicle, the right camera is arranged on the outside of the right A-pillar of the vehicle, and the right camera is used to obtain real-scene video stream information of the right A-pillar imaging area of ​​the vehicle, the left camera and the right camera are respectively connected to the control system, and the control system processes the video information collected by the left camera and the right camera and transmits them to the left A-pillar display screen and the right A-pillar display screen respectively, the left A-pillar display screen is arranged on the inside of the left A-pillar of the vehicle, and the left A-pillar display screen is used to display a left A-pillar blind spot display screen, the right A-pillar display screen is arranged on the inside of the right A-pillar of the vehicle, and the right A-pillar display screen is used to display a right A-pillar blind spot display screen.

[0035] Furthermore, the left A-pillar display screen and the right A-pillar display screen are both flexible screens.

[0036] Beneficial effects of the present invention:

[0037] 1. The present invention extracts a first processing picture from the real-scene video stream information captured by the camera, determines each obstacle image based on the first processing picture, and performs distance correction on each obstacle image, so that the actual distance information between each obstacle and the vehicle is closer to reality, thereby improving the integration between the display picture and the real-scene picture.

[0038] 2. The present invention captures and splices real-scene video stream information into a panoramic picture. In this panoramic picture, a delay-free picture is captured based on a fixed delay parameter and the real-time speed information of the vehicle, thereby effectively reducing the picture delay phenomenon. In particular, when the vehicle speed is fast, the picture delay problem is solved by means of picture delay compensation.

[0039] 3. The present invention combines the driver's eyebrow positioning information and the A-pillar shape information inside the vehicle to adapt to the differences in the A-pillar blind spots that can be seen by drivers of different heights and body shapes, and performs dynamic perspective transformation correction on the image to provide a good visual experience and a more natural integration with the real environment, truly achieving the transparent effect of the A-pillar.

[0040] 4. The implementation method of the present invention is simple, the hardware cost is low, and the precision is high. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0042] Figure 1 It is a schematic diagram of the overall process framework of a vehicle A-pillar blind spot perspective method provided in Example 1.

[0043] Figure 2 It is a schematic diagram of a process framework for calculating the distance between an obstacle and a vehicle in a vehicle A-pillar blind spot perspective method provided in Example 1.

[0044] Figure 3 It is a schematic diagram of the process framework when performing image delay compensation processing in a vehicle A-pillar blind spot perspective method provided in Example 1.

[0045] Figure 4 It is a schematic diagram of the framework of a vehicle A-pillar blind spot perspective system provided in Example 2. DETAILED DESCRIPTION

[0046] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", and "third" are used for descriptive purposes only, and cannot be understood as indicating or implying relative importance.

[0048] Embodiment 1:

[0049] Reference Figure 1 , in order to solve the problems in the prior art, first, for different depths of field, that is, different distances between different external obstacles and the vehicle, if the same correction standard is used, the final display images of different obstacles will not be integrated with the actual scene outside the vehicle; second, in actual situations, there will be a more obvious delay in the A-pillar display image from camera acquisition to image processing and then to the final A-pillar display, especially when the vehicle speed is fast, the image delay is serious, affecting the driver's normal judgment; third, it cannot adapt to the driving of drivers of different body shapes, and the perspective relationship between them and the A-pillar cannot adapt to a more natural blind spot image display effect, the dynamic transformation effect is poor, and the A-pillar screen is installed at an inclined angle, and perspective deformation correction is also required. Therefore, this embodiment 1 provides a vehicle A-pillar blind spot perspective method, including:

[0050] Acquire real-scene video stream information of the A-pillar imaging area of ​​the vehicle, and extract a first processing picture according to the sampling frequency;

[0051] Determine a plurality of obstacle images from the first processed picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, perform image effect enhancement processing, and obtain the A-pillar display picture;

[0052] Transmit the A-pillar display to the A-pillar screen.

[0053] Based on the monocular camera, the real-life video stream information of the A-pillar imaging area of ​​the vehicle outside the vehicle can be collected. After calculating the distance between the obstacle and the vehicle and the size of the obstacle through the algorithm, a three-dimensional correction reconstruction is performed. For the three-dimensional reconstruction image, the image delay compensation calculation is performed based on the vehicle speed data and the inherent delay characteristics of the processing system. Based on the coordinate information of the center of the eyebrows, the image of the A-pillar blind spot is dynamically mapped to the image, and the image effect is enhanced for the dynamically mapped A-pillar display image, and then the A-pillar display image is transmitted to the A-pillar screen. Not only is the distance correction made to each obstacle image, so that the actual distance information between each obstacle and the vehicle is closer to reality, the integration between the display image and the real scene image is improved, and the image delay phenomenon is effectively reduced. Especially when the vehicle speed is fast, the image delay problem is solved by the image delay compensation method. It also adapts to the difference in the A-pillar blind spot that drivers of different heights and body shapes can see, and the image is dynamically perspective-transformed and corrected. The A-pillar display image is seamlessly integrated with the real scene, and dynamically changes with the driver's perspective to provide a good visual experience.

[0054] Reference Figure 2, in the process of calculating the distance between the obstacle and the vehicle, establishing a first processing picture coordinate system in the first processing picture, determining a plurality of obstacle images, each of the obstacle images including positioning information, the positioning information being used to mark the position of the obstacle in the imaging area of ​​the A-pillar;

[0055] Determining image distance information between the obstacle image and a reference line of the vehicle in a first processing screen coordinate system;

[0056] The actual distance information between the vehicle and the obstacle is determined according to the image distance information.

[0057] It should be noted that each camera takes real-life photos of the A-pillar blind spot outside the vehicle and a certain area in front and behind it, and forms corresponding real-life video stream information. The real-life video stream information is extracted by using a sampling frequency of 3 frames per second, and a first processing picture is extracted from the real-life video stream information; then, in each first processing picture, a corresponding first processing picture coordinate system is established, and a number of obstacle images are determined in the first processing picture coordinate system. The obstacle represented by each obstacle image may be at different distances from the vehicle, some are far, some are near, and some are even overlapping. Each obstacle image needs to be positioned, wherein in the first processing picture coordinate system, a baseline representing the outermost edge of the vehicle is set, and the image distance information between the obstacle image and the baseline is determined, and the image distance information is converted into actual distance information.

[0058] In this embodiment, the real-scene video stream information is corrected according to the actual distance information, and the corrected real-scene video stream information is transmitted to the A-pillar screen to realize the distance correction of each obstacle in the first processing picture, so that the actual distance information between each obstacle and the vehicle is closer to reality, thereby improving the integration between the display picture and the real-scene picture.

[0059] As a preferred manner, the positioning information includes pixel position information of the obstacle image in the first processing screen coordinate system, the pixel position information includes the lower left corner coordinates (x1, y1) of the obstacle image, the width w and the height h of the obstacle image, and the rectangular bounding box of the obstacle image is determined by the x1, y1, w and h; each obstacle is converted into a rectangular bounding box in the first processing screen, and each rectangular bounding box is represented by an array (x1, y1, w, h). More specifically, the lower left corner coordinates of the rectangular bounding box are (x1, y1), the lower right corner coordinates are (x1+w, y1), the upper left corner coordinates are (x1, y1+h), and the upper right corner coordinates are (x1+w, y1+h); wherein the width w and the height h of the obstacle image refer to the pixel size in the first processing screen, not the actual size.

[0060] In this embodiment, an image distance coefficient R is predefined, and the image distance coefficient R is configured to be used for conversion between the image distance information and the actual distance information. The image distance coefficient R is a conventional conversion coefficient between the actual distance from the obstacle outside the vehicle to the vehicle and the coordinate distance in the first processing picture determined by a measurement experiment. Through a measurement experiment, a reference object is placed at a set distance outside the vehicle in advance, and the actual distance information between the reference object and the front of the vehicle is measured. After shooting with a camera, the image distance information of the reference object is calculated in the first processing picture, and the image distance coefficient R is inferred using the actual distance information and image distance information of the reference object, so as to determine the image distance coefficient R of the reference object at different directions and distances for the vehicle.

[0061] After determining the value range of this image distance coefficient R, in actual operation, the obstacle near-end distance information a is determined according to the rectangular bounding box of the obstacle image, and the actual distance information includes the vertical absolute distance A between the vehicle and the obstacle in the forward direction of the vehicle body, and the vertical absolute distance A=a*R, wherein the obstacle near-end distance information a can be the bottom line y1 coordinate of the rectangular bounding box, that is, this vertical absolute distance A=y1*R.

[0062] Of course, as another implementation method, the proportional relationship b of the most protruding part of the rear of the vehicle relative to the height of the vehicle body can be determined according to different vehicle models, and then the corresponding vertical coordinate y1+b*h is converted in the rectangular boundary box according to this proportional relationship b, that is, the vertical absolute distance A=(y1+b*h)*R at this time.

[0063] In this embodiment, when there is an overlapping area between the rectangular bounding boxes of the plurality of obstacle images, the rectangular bounding box with the smallest obstacle near-end distance information a is selected as the conversion object. That is, if there are multiple obstacles in front of the vehicle, and since each obstacle is moving and changing, there will be overlaps between the obstacle images. At this time, for safety and convenience considerations, within a certain set range, for the obstacle images within the set range, the data information represented by the rectangular bounding box with the smallest obstacle near-end distance information a is uniformly corrected.

[0064] Reference Figure 3There will inevitably be a certain time delay between the image captured by the camera outside the vehicle and the image displayed on the display inside the vehicle. Especially when the vehicle speed is fast, the image delay will cause the displayed image to be inconsistent with the real scene. For example, when the vehicle speed is 60km / h and the delay is 100ms, the corresponding object has moved about 1.7m when the image is displayed on the screen. Therefore, in this embodiment, more specifically, when performing the image delay compensation processing, the real-scene video stream information is extracted according to the splicing frequency, and several frames of real-scene images are dynamically extracted from the video stream collected by the camera to obtain a second processed image; the extracted second processed images are spliced ​​and synthesized to obtain a panoramic image, which includes a panoramic image of a section of the journey and is a continuous panoramic image. Since this panoramic image includes the image that will appear on the A-pillar display screen in the future, according to the fixed delay parameter and the real-time speed information of the vehicle, a non-delayed image is intercepted from the panoramic image, wherein this non-delayed image is actually a future image, and the image image and the real-scene image are synchronously displayed by using the future image synthesis, thereby solving the image delay problem.

[0065] As an implementation method, the fixed delay parameter is a parameter obtained by integrating the time consumed by the camera to collect real-scene video stream information, digital-to-analog conversion of signals, transmission and display. Generally, this is a fixed parameter, and the entire processing process takes a certain amount of time. Combined with the real-time speed information of the vehicle, if the vehicle speed is fast, the delay will be long, and if the vehicle speed is slow, the delay will be short.

[0066] In order to adapt to the different viewing angles between drivers of different heights and body shapes and the A-pillar, in this embodiment, after calculating the distance between the obstacle and the vehicle, the Warp Transformation image processing algorithm is used to correct and splice the images according to the actual distance information between each obstacle and the vehicle;

[0067] According to the positioning information of the driver's eyebrow center, the angle and position of the driver viewing the A-pillar screen are calculated, and the image is transformed using the warpPerspective method to adjust the image perspective deformation and mapping range, and dynamically map the image of the A-pillar blind spot.

[0068] According to the imaging area information of the vehicle's A-pillar and the actual distance between the vehicle and the obstacle, combined with the driver's eyebrow positioning information, the real-scene video stream information is three-dimensionally reconstructed and the perspective is transformed, and the A-pillar display image is transmitted to the A-pillar screen.

[0069] According to the driver's eyebrow positioning information, the A-pillar blind spot area information seen from different driver perspectives is determined, and the corresponding picture is selected from the vehicle's A-pillar imaging area information. After dynamic perspective transformation and correction, the picture is displayed on the A-pillar screen with strong adaptability and high integration.

[0070] As an implementation method, vehicle interior image information is obtained, the interior image information including the driver's eyebrow center positioning information and A-pillar shape information; the blind spot image area information in the first processing screen is determined by the vehicle interior image information, and the corrected blind spot image area information is sent to the A-pillar screen.

[0071] Since the A-pillar screen is installed on the A-pillar at an oblique angle, considering the different shapes and viewing angles of the A-pillar, the blind spot image area information is automatically and adaptively determined in the first processing screen. In order to better project the collected real-scene image onto the display screen, the vehicle interior image information is first obtained, the interior image information includes the driver position information and the A-pillar shape information, wherein the driver position information includes the position information of the driver's head and eyes, and the A-pillar shape information includes the position information of the boundaries of each structure of the A-pillar; the blind spot image area information in the first processing screen is determined by the vehicle interior image information, and the corrected blind spot image area information is sent to the A-pillar screen.

[0072] It should be noted that a three-dimensional coordinate system is established inside the vehicle, with the position information of the driver's head and eyes as the reference point, and the position information of the boundaries of the A-pillar structures as the middle point. A straight line is determined through the two points, and these multiple straight lines form a straight line set. The projection of this straight line set on the first processing picture is used to separate the blind spot image area information in the first processing picture to adapt to drivers of different heights and body shapes whose eye positions are different. With the eyes as the starting point and the A-pillar as the obstacle within the visual range, the blind spot image area information separated by the straight line set is exactly the place blocked by the A-pillar. This blind spot image area information is accurately and concisely extracted and displayed on the A-pillar screen, truly achieving the transparent effect of the A-pillar.

[0073] Embodiment 2:

[0074] Reference Figure 4 , this embodiment 2 provides a system applied to the vehicle A-pillar blind spot perspective method in embodiment 1, comprising an image acquisition module, a processing module and a display module, wherein the image acquisition module is signal-connected to the processing module, and the processing module is signal-connected to the display module;

[0075] The image acquisition module is used to obtain real-scene video stream information of the imaging area of ​​the vehicle's A-pillar, the real-scene video stream information includes the area where the driver is blocked by the A-pillar, and extract the first processing picture according to the sampling frequency;

[0076] The processing module is used to determine a number of obstacle images from the first processing picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, and perform image effect enhancement processing to obtain the A-pillar display picture; the image ranging, delay compensation and dynamic mapping processing processes are respectively completed in this processing module;

[0077] The display module is used to display the A-pillar display screen.

[0078] As an implementation mode, the processing module includes a ranging unit, a delay compensation unit and a dynamic mapping unit, the image acquisition module is connected to the ranging unit signal, the ranging unit is connected to the delay compensation unit signal, the delay compensation unit is connected to the dynamic mapping unit signal, and the dynamic mapping unit is connected to the display module signal;

[0079] The distance measuring unit is used to establish a first processing picture coordinate system in the first processing picture, determine a plurality of obstacle images, each of the obstacle images includes positioning information, and the positioning information is used to mark the position of the obstacle in the imaging area of ​​the A-pillar; determine image distance information between the obstacle image and a baseline of the vehicle in the first processing picture coordinate system; and determine actual distance information between the vehicle and the obstacle based on the image distance information;

[0080] The delay compensation unit is used to extract the real-scene video stream information according to the splicing frequency to obtain a second processed picture; splice and synthesize the extracted second processed pictures to obtain a panoramic picture; and intercept a delay-free picture from the panoramic picture according to the fixed delay parameter and the real-time speed information of the vehicle;

[0081] The dynamic mapping unit is used to use the Warp Transformation image processing algorithm to correct and splice the picture according to the actual distance information between each obstacle and the vehicle; calculate the angle and position of the driver viewing the A-pillar screen according to the driver's eyebrow positioning information, transform the picture using the warpPerspective method, adjust the image perspective deformation and mapping range, and dynamically map the A-pillar blind spot image on the picture.

[0082] In this embodiment, the vehicle A-pillar blind spot perspective system also includes an interface module, which is used to receive the driver's eyebrow positioning information and vehicle CAN data. The vehicle CAN data includes the vehicle's real-time speed information and other operating data. After receiving the vehicle CAN data, the interface module is connected to the processing module signal and uses the data for calculation processing.

[0083] As an embodiment, it also includes an in-vehicle modeling module, which is used to obtain vehicle interior image information, wherein the interior image information includes driver position information and A-pillar shape information, and the blind spot image area information in the first processing screen is determined through the vehicle interior image information.

[0084] Embodiment 3:

[0085] This embodiment 3 provides a vehicle to which the vehicle A-pillar blind spot perspective method in embodiment 1 is applied, including a left camera, a right camera, a left A-pillar display screen, a right A-pillar display screen and a control system;

[0086] The left camera is arranged outside the left A-pillar of the vehicle, and the left camera is used to obtain real-view video stream information of the imaging area of ​​the left A-pillar of the vehicle. The right camera is arranged outside the right A-pillar of the vehicle, and the right camera is used to obtain real-view video stream information of the imaging area of ​​the right A-pillar of the vehicle.

[0087] The left camera and the right camera are connected to the control system respectively, and the control system processes the video information collected by the left camera and the right camera and transmits them to the left A-pillar display screen and the right A-pillar display screen respectively. The processing process and steps of the control system refer to the description in Example 1 and are not described in detail here. More specifically, the control system is arranged on the center console of the vehicle, receives the real-scene video stream information of the A-pillar imaging area, the vehicle CAN data, the driver's eyebrow positioning information, etc., performs a series of object distance determination, image shearing, correction and other calculations, and then transmits the processed images to the left A-pillar display screen and the right A-pillar display screen respectively;

[0088] The left A-pillar display screen is arranged on the inner side of the left A-pillar of the vehicle, and the left A-pillar display screen is used to display the left A-pillar blind spot display screen. The right A-pillar display screen is arranged on the inner side of the right A-pillar of the vehicle, and the right A-pillar display screen is used to display the right A-pillar blind spot display screen.

[0089] Wherein, the left A-pillar display screen and the right A-pillar display screen are both flexible screens.

[0090] Embodiment 4:

[0091] This embodiment 4 provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the vehicle A-pillar blind spot perspective method in the above-mentioned embodiment 1.

[0092] Embodiment 5:

[0093] This embodiment 5 provides a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of the method in the above embodiment 1 are implemented.

[0094] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device or a combination of any of these devices.

[0095] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0096] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0097] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0098] Compared with the prior art, the present invention extracts a first processed picture from the real-scene video stream information captured by the camera, determines each obstacle image based on the first processed picture, and performs distance correction on each obstacle image, so that the actual distance information between each obstacle and the vehicle is closer to reality, thereby improving the fusion between the displayed picture and the real-scene picture.

[0099] The present invention captures and splices real-scene video stream information into a panoramic picture, and captures a delay-free picture in the panoramic picture based on a fixed delay parameter and the real-time speed information of the vehicle, thereby effectively reducing the picture delay phenomenon. In particular, when the vehicle speed is fast, the picture delay problem is solved by means of picture delay compensation.

[0100] The present invention combines the driver's eyebrow positioning information and the A-pillar shape information inside the vehicle to adapt to the differences in the A-pillar blind spots that can be seen by drivers of different heights and body shapes, and performs dynamic perspective transformation correction on the image to provide a good visual experience and a more natural integration with the real environment, truly achieving the transparent effect of the A-pillar.

[0101] The implementation method of the present invention is simple, the hardware cost is low and the precision is high.

[0102] Finally, it should be emphasized that the present invention is not limited to the above-mentioned implementation mode. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle A-pillar blind spot perspective method, characterized in that: include: Acquire real-scene video stream information of the A-pillar imaging area of ​​the vehicle, and extract a first processing picture according to the sampling frequency; Determine a plurality of obstacle images from the first processed picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, perform image effect enhancement processing, and obtain the A-pillar display picture; Transmit the A-pillar display to the A-pillar screen; Establishing a first processing picture coordinate system in the first processing picture, determining a plurality of obstacle images, each of the obstacle images including positioning information, the positioning information being used to mark the position of the obstacle in the A-pillar imaging area; Determining image distance information between the obstacle image and a reference line of the vehicle in a first processing screen coordinate system; Determine the actual distance information between the vehicle and the obstacle according to the image distance information; The positioning information includes pixel position information of the obstacle image in the first processing screen coordinate system, the pixel position information includes the coordinates (x1, y1) of the lower left corner of the obstacle image, the width w and the height h of the obstacle image, and the rectangular bounding box of the obstacle image is determined by the x1, y1, w and h; Determine the obstacle near-end distance information a according to the rectangular bounding box of the obstacle image, and when there is an overlapping area between the rectangular bounding boxes of the plurality of obstacle images, select the rectangular bounding box with the smallest obstacle near-end distance information a as the conversion object; Extracting the real-scene video stream information according to the splicing frequency to obtain a second processed picture; The extracted second processed images are stitched and synthesized to obtain a panoramic image; According to the fixed delay parameters and the real-time speed information of the vehicle, the delay-free image is captured from the panoramic image; The fixed delay parameter is determined by comprehensively collecting the real-scene video stream information of the camera, converting the digital-to-analog signal, and transmitting and displaying the signal; After calculating the distance between the obstacle and the vehicle, the Warp Transformation image processing algorithm is used to correct and stitch the images according to the actual distance information between each obstacle and the vehicle. According to the driver's eyebrow positioning information, the angle and position of the driver's viewing of the A-pillar screen are calculated, and the image is transformed using the warpPerspective method to adjust the image perspective deformation and mapping range, and dynamically map the image of the A-pillar blind spot. According to the imaging area information of the vehicle's A-pillar and the actual distance between the vehicle and the obstacle, combined with the driver's eyebrow positioning information, the real-scene video stream information is 3D reconstructed and the perspective is transformed, and the A-pillar display image is transmitted to the A-pillar screen; Acquire vehicle interior image information, the interior image information including driver's eyebrow center positioning information and A-pillar shape information; determine blind spot image area information in the first processing screen through the vehicle interior image information, and send the corrected blind spot image area information to the A-pillar screen.

2. A vehicle A-pillar blind spot perspective method as claimed in claim 1, characterized in that: An image distance coefficient R is defined, and the image distance coefficient R is configured to be used for conversion between the image distance information and the actual distance information. The image distance coefficient R is a conventional conversion coefficient between the actual distance from the external obstacle to the vehicle determined by measurement experiments and the coordinate distance in the first processing picture.

3. A vehicle A-pillar blind spot perspective method as claimed in claim 2, characterized in that: The actual distance information includes a vertical absolute distance A between the vehicle and the obstacle in the forward direction of the vehicle body, and the vertical absolute distance A=a*R.

4. A vehicle A-pillar blind spot perspective method as claimed in claim 3, characterized in that: The sampling frequency is 3 frames per second.

5. A system for the vehicle A-pillar blind spot perspective method according to any one of claims 1 to 4, characterized in that: It includes an image acquisition module, a processing module and a display module, wherein the image acquisition module is connected to the processing module by signal, and the processing module is connected to the display module by signal; The image acquisition module is used to obtain real-scene video stream information of the imaging area of ​​the vehicle A-pillar and extract the first processing picture according to the sampling frequency; The processing module is used to determine a number of obstacle images from the first processed picture, calculate the distance between the obstacle and the vehicle and the size of the obstacle, output a three-dimensional reconstructed picture, perform picture delay compensation calculation, obtain the driver's eyebrow center positioning information, dynamically map the A-pillar blind spot image to the picture, and perform image effect enhancement processing to obtain the A-pillar display picture; The display module is used to display the A-pillar display screen.

6. A system as claimed in claim 5, characterized in that: The processing module includes a distance measuring unit, a delay compensation unit and a dynamic mapping unit, the image acquisition module is connected to the distance measuring unit by signal, the distance measuring unit is connected to the delay compensation unit by signal, the delay compensation unit is connected to the dynamic mapping unit by signal, and the dynamic mapping unit is connected to the display module by signal; The distance measuring unit is used to determine the actual distance information between the vehicle and the obstacle; The time delay compensation unit is used to capture a delay-free picture from the panoramic picture; The dynamic mapping unit is used to dynamically map the A-pillar blind spot image on the screen according to the driver's eyebrow center positioning information.

7. A system according to claim 6, characterized in that: It also includes an interface module, which is used to receive the driver's eyebrow center positioning information and the car CAN data, and the car CAN data includes the vehicle's real-time speed information.

8. A vehicle using the vehicle A-pillar blind spot perspective method as described in any one of claims 1 to 4, characterized in that: The vehicle comprises a left camera, a right camera, a left A-pillar display screen, a right A-pillar display screen and a control system, wherein the left camera is arranged on the outer side of the left A-pillar of the vehicle, and is used to obtain real-scene video stream information of the imaging area of ​​the left A-pillar of the vehicle; the right camera is arranged on the outer side of the right A-pillar of the vehicle, and is used to obtain real-scene video stream information of the imaging area of ​​the right A-pillar of the vehicle; the left camera and the right camera are respectively connected to the control system, and the control system processes the video information collected by the left camera and the right camera and transmits them to the left A-pillar display screen and the right A-pillar display screen respectively; the left A-pillar display screen is arranged on the inner side of the left A-pillar of the vehicle, and is used to display a display picture of the blind spot of the left A-pillar; the right A-pillar display screen is arranged on the inner side of the right A-pillar of the vehicle, and is used to display a display picture of the blind spot of the right A-pillar.

9. A vehicle as claimed in claim 8, characterized in that: The left A-pillar display screen and the right A-pillar display screen are both flexible screens.