Visual field capturing method of electronic rearview mirror

Abstract

The invention relates to the technical field of electronic rearview mirrors, in particular to a visual field capturing method of an electronic rearview mirror. By optimizing the data processing flow, the power consumption and the cost of the electronic rearview mirror of the commercial vehicle are effectively reduced, and wide application of the electronic rearview mirror is promoted; the video is cut through the vehicle kinematics model and the driving data, so that the data processing amount is reduced, and the power consumption is reduced; a preset detection algorithm and result analysis enable processing to be more targeted, and power consumption is further reduced. In the aspect of cost, an economic treatment mode is adopted, and the overall cost is reduced; meanwhile, the scheme provides clear and coherent visual field pictures outside the vehicle, the visual field can be automatically adjusted especially in visual field blind areas such as steering, the driving safety is improved, the requirements of laws and regulations are met, and safe driving of the commercial vehicle is guaranteed.

Description

Technical Field

[0001] The invention relates to the technical field of electronic rearview mirrors, and in particular to a method for capturing the visual field of an electronic rearview mirror. Background Art

[0002] In the field of commercial vehicles, electronic rearview mirrors are widely used to meet regulatory requirements and improve driving safety. Due to the large size of commercial vehicles, there are many blind spots when driving, especially when turning. Traditional rearview mirrors cannot automatically adjust the field of view to adapt to different driving conditions. Therefore, it is necessary to install multiple cameras to provide the driver with a view outside the vehicle so as to better understand the environmental conditions around the vehicle.

[0003] At present, the existing technical solutions mainly use SoC (system on chip) or high-performance FPGA (field programmable gate array) containing SoC and DDR (double data rate synchronous dynamic random access memory) memory to process camera data. However, this processing method has the problems of high power consumption and high cost, which is not conducive to the large-scale promotion and application of commercial vehicle electronic rearview mirrors. Summary of the invention

[0004] In order to solve the above-mentioned shortcomings in the prior art, the present invention proposes a method, device, equipment and storage medium for capturing the field of view of an electronic rearview mirror.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0006] A method for capturing the visual field of an electronic rearview mirror comprises: acquiring sensing data from various sensors, and obtaining five-channel video data according to the sensor data analysis; acquiring vehicle driving data in the sensing data; performing a cropping operation on the five-channel video data based on a preset vehicle kinematic model and the vehicle driving data to obtain cropped video data; performing defect detection on the cropped video data using a preset detection algorithm to obtain a detection result; performing result analysis on the detection result; when the detection result shows that there are defective cropped video data, pre-processing the cropped video data based on the detection result to obtain spliced ​​video data; otherwise, splicing the cropped video data to obtain spliced ​​video data; rendering the spliced ​​video data to obtain rendered video data; parsing the rendered video data to obtain three-channel video data; and generating display video information according to the three-channel video data. Starting from the optimization of data processing flow, power consumption and cost are effectively reduced, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; based on the vehicle kinematic model combined with vehicle driving data, the video data is cropped, which can accurately remove irrelevant images, focus on key visual areas, reduce unnecessary data processing, and thus reduce system power consumption; using preset detection algorithms, result analysis and other steps, it can intelligently determine whether further processing and cropping of video data is required, making the entire processing process more targeted and efficient, and further reducing system power consumption; secondly, it also has obvious advantages in terms of cost, adopting a more economical processing method, reducing the overall cost, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; through precise processing and splicing rendering of video data, it provides drivers with a clear and coherent view of the outside of the vehicle; especially in the case of blind spots that are prone to occur when commercial vehicles turn, the field of view display can be automatically adjusted, which greatly improves driving safety and enables drivers to better understand the surrounding environment of the vehicle, meeting regulatory requirements while providing a strong guarantee for the safe driving of commercial vehicles.

[0007] Further, the sensor data is obtained from various sensors, and five channels of video data are obtained according to the analysis of the sensor data, including: obtaining various sensor data to obtain sensor data; obtaining video data in the sensor data, and packaging the video data to obtain the first CSI-2 data; converting the first CSI-2 data into serial data; deserializing and aggregating the serial data to obtain two channels of second CSI-2 data; parsing the second CSI-2 data to obtain five channels of video data. The acquisition of various sensor data lays the foundation for video processing; then, packaging the video data to obtain the first CSI-2 data ensures information integrity and provides high-quality raw materials for subsequent video processing; data serial conversion is conducive to efficient transmission between different devices; deserializing and aggregating to obtain two channels of second CSI-2 data, improves processing flexibility and prepares for analysis; finally, the second CSI-2 data is parsed to obtain five channels of video data, demonstrating the system's deep data processing capabilities, ensuring the accuracy and availability of the output video, and providing reliable protection for subsequent video applications.

[0008] Furthermore, the cropping operation of the five-channel video data based on the preset vehicle kinematic model and vehicle driving data to obtain the cropped video data includes: judging whether the vehicle is in a reversing state based on the vehicle driving data; when the vehicle is in a reversing state, obtaining the steering angle data in the vehicle driving data; judging whether the vehicle is in a turning state according to the steering angle data; when the vehicle is in a turning state, calculating the offset view angle screen coordinates based on the vehicle kinematic model and the vehicle driving data; obtaining the preset reversing view angle display ratio; obtaining all the video data except the V-type video data in the five-channel video data to obtain four-channel video data; and calculating the offset view angle screen coordinates and the reversing view angle display ratio according to the offset view angle screen coordinates and the reversing view angle display ratio. Each video frame in the four-channel video data is cropped to obtain preliminary cropping data; when the vehicle is not in a reversing state, the steering angle data in the vehicle driving data is obtained; whether the vehicle is in a turning state is determined based on the steering angle data; when the vehicle is not in a turning state, the normal viewing angle picture coordinates are calculated based on the vehicle kinematic model and the vehicle driving data; the preset normal viewing angle display ratio is obtained; each video frame is cropped according to the normal viewing angle picture coordinates and the normal viewing angle display ratio to obtain preliminary cropping data; V-class video data in the five-channel video data is obtained to obtain the first V-class video data; and cropped video data is generated based on the preliminary cropping data and the first V-class video data. The reversing state and steering state are judged through vehicle driving data, providing key decision-making basis for video display screen adjustment; the corresponding viewing angle display ratio is obtained according to different vehicle states, and then the vehicle is judged whether it is in a turning state or a non-turning state according to the corresponding viewing angle display ratio; for the turning and non-turning states, the relevant screen coordinates are calculated, and each video frame is cropped according to the relevant screen coordinates and display ratio to obtain preliminary cropping data, which can accurately remove irrelevant pictures, reduce unnecessary data processing and video data transmission, optimize system resource usage, and reduce system power consumption; finally, combined with Class V video data, cropped video data is generated to provide the driver with clearer, more accurate and optimized visual information, improve driving safety and comfort, and reflect the advanced technical level of intelligent vehicle systems in video processing.

[0009] Furthermore, the cropping operation is performed on each video frame in the four-channel video data according to the offset view angle picture coordinates and the reversing view angle display ratio to obtain preliminary cropping data, including: obtaining the clock cycle in the sensor data, and obtaining a pixel point coordinate set of each video frame according to the clock cycle count; judging whether there is a pixel point coordinate not within a preset area according to the pixel point coordinate set; when there is a pixel point coordinate not within the area, marking the pixel point whose pixel point coordinate is not within the area as an invalid pixel point to obtain an invalid pixel point set; calculating the cropping starting coordinates of each video frame according to the offset view angle picture coordinates and the reversing view angle display ratio to obtain a cropping starting coordinate set; obtaining a preset cropping height and width for each video frame; and cropping each video frame according to the invalid pixel point set, the cropping starting coordinate set, the cropping height and the width to obtain preliminary cropping data. By combining the offset perspective image coordinates of the vehicle's driving state and the reversing perspective display ratio to calculate the cropping start coordinates of each video frame, the starting reference parameters for the cropping operation are provided for the subsequent further processing of the video frames, which facilitates the precise adjustment of the video picture to adapt to the vehicle's driving conditions; then, the pixel point coordinate set is determined using the clock cycle of the sensor data to ensure the accurate time reference of the pixel resolution; the pixel point coordinates are calculated to clarify their positions, providing a basis for judging the validity, which helps to refine the processing of the video picture and improve the quality and clarity; by judging whether the pixel points are within the preset area, the problem pixels can be quickly screened to improve processing efficiency and accuracy; marking invalid pixels can remove interference in a targeted manner and improve the picture quality and validity; finally, each video frame is cropped based on the invalid pixel point set, the cropping start coordinate set, the cropping height and width to remove the invalid area, reduce the amount of data processing and transmission, and reduce system power consumption.

[0010] Furthermore, the cropping operation is performed on each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio to obtain preliminary cropping data, including: obtaining the clock cycle in the sensor data, and obtaining a pixel point coordinate set of each video frame according to the clock cycle count; judging whether there is a pixel point coordinate not within a preset area according to the pixel point coordinate set; when there is a pixel point coordinate not within the area, marking the pixel point whose pixel point coordinate is not within the area as an invalid pixel point to obtain an invalid pixel point set; calculating the cropping starting coordinates of each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio to obtain a cropping starting coordinate set; obtaining a preset cropping height and width for each video frame; and cropping each video frame according to the invalid pixel point set, the cropping starting coordinate set, the cropping height and the width to obtain preliminary cropping data. By combining the normal viewing angle picture coordinates and the normal viewing angle display ratio, the cropping start coordinates of each video frame are calculated, which provides the starting reference parameters for the cropping operation for the subsequent further processing of the video frame, which helps to adapt to the vehicle driving conditions for precise picture adjustment; the sensor data clock cycle is used to determine the pixel point coordinate set to ensure the accurate time reference of pixel resolution; the pixel point coordinates are calculated to clarify the position, provide a basis for judging the effectiveness, realize refined processing, and improve the video picture quality and clarity; by judging whether the pixel point is in the preset area, the problem pixel points can be quickly screened to improve processing efficiency and accuracy; invalid pixels are marked to facilitate targeted removal of interference and improve picture quality and effectiveness; finally, each video frame is cropped based on the invalid pixel point set, the cropping start coordinate set, the cropping height and width, the invalid area is removed, the data processing and transmission volume is reduced, the system power consumption is reduced, the video picture focuses on the effective content, and the overall effect and performance of video processing are improved.

[0011] Furthermore, the use of a preset detection algorithm to perform defect detection on the cropped video data to obtain a detection result includes: using the detection algorithm to perform a round-robin defect detection on each channel of video data in the cropped video data to obtain a detection result. By using the detection algorithm to round-robinly detect the cropped video data, some basic problems of each channel of video data, such as color deviation, uneven brightness or inconsistent frame rate, can be comprehensively and accurately analyzed, providing reliable detection results for subsequent processing.

[0012] Furthermore, when the detection result is that there are defective cropped video data, the cropped video data is preprocessed based on the detection result to obtain spliced ​​video data, including: when the detection result is that there are defective cropped video data, all defective cropped video data are set as defective video data; splicing operations are performed on the defective video data to obtain first video data; splicing operations are performed on the remaining non-defective cropped video data to obtain second video data; and spliced ​​video data is generated based on the first video data and the second video data. When defective cropped video data is detected, splicing operations are performed on the defective video data and the non-defective cropped video data respectively to obtain the first video data and the second video data. This divide-and-conquer processing method helps to better integrate video clips in different states; finally, spliced ​​video data is generated based on the first video data and the second video data, realizing the organic integration of defective and non-defective videos; at the same time, the entire process reduces memory bandwidth requirements through logical integration, optimizes system resource usage, and improves the efficiency of overall video transmission processing.

[0013] Furthermore, the splicing operation on the second rendered video data to obtain the first video data includes: obtaining left-side Class II video data and left-side Class IV video data in the defective video data, and performing a splicing operation on the left-side Class II video data and the left-side Class IV video data to obtain first left-side spliced ​​video data; obtaining right-side Class II video data and right-side Class IV video data in the defective video data, and performing a splicing operation on the right-side Class II video data and the right-side Class IV video data to obtain first right-side spliced ​​video data; obtaining Class V video data in the defective video data to obtain second Class V video data; and generating the first video data based on the first left-side spliced ​​video data, the first right-side spliced ​​video data and the second Class V video data. The left-side Class II and left-side Class IV video data in the defective video data are spliced ​​separately, and the video resources of the specific categories on the left are integrated to make the left-side perspective video content more complete and coherent, which is conducive to subsequent transmission; similarly, the right-side Class II and right-side Class IV video data are spliced ​​to ensure that the right-side perspective video has better coherence and integrity, which is convenient for the driver to obtain rich information from different angles; then, the Class V video data in the defective video data is obtained to provide an important component for video data integration; finally, the first video data is generated according to the first left-side spliced ​​video data, the first right-side spliced ​​video data and the second Class V video data, realizing the organic integration of video data of different types and different perspectives, providing the driver with a comprehensive video picture, improving the overall quality and value of the video, and meeting the diverse needs in practical applications.

[0014] Furthermore, the stitching operation is performed on the remaining cropped video data without defects to obtain the second video data, including: setting the remaining cropped video data without defects as the third video data; obtaining the left-side Class II video data and the left-side Class IV video data in the third video data, and performing a stitching operation on the left-side Class II video data and the left-side Class IV video data to obtain the second left-side stitched video data; obtaining the right-side Class II video data and the right-side Class IV video data in the third video data, and performing a stitching operation on the right-side Class II video data and the right-side Class IV video data to obtain the second right-side stitched video data; obtaining the Class V video data in the third video data to obtain the third Class V video data; and generating the second video data according to the first left-side stitched video data, the first right-side stitched video data and the third Class V video data. Setting the defect-free cropped video data as the third video data saves processing time and computing resources, improves the overall video processing efficiency, and makes the system respond more quickly. Then, by splicing the specific categories of video data on the left and right sides respectively, the second left and right spliced ​​video data are obtained, so that the left and right side perspective videos are more complete and coherent, which is convenient for the driver to clearly and comprehensively obtain multi-perspective information; obtaining the third V-class video data further enriches the video content dimension; finally, the second video data is generated based on these data to achieve organic integration, providing the driver with multi-perspective and content-rich video data to assist observation and judgment during driving.

[0015] Furthermore, in contrast, the cropped video data is spliced ​​to obtain spliced ​​video data, including: obtaining left-side Class II video data and left-side Class IV video data in the cropped video data, and performing a splicing operation on the left-side Class II video data and the left-side Class IV video data to obtain third left-side spliced ​​video data; obtaining right-side Class II video data and right-side Class IV video data in the cropped video data, and performing a splicing operation on the right-side Class II video data and the right-side Class IV video data to obtain third right-side spliced ​​video data; obtaining Class V video data in the cropped video data to obtain fourth Class V video data; and generating spliced ​​video data based on the third left-side spliced ​​video data, the third right-side spliced ​​video data and the fourth Class V video data. By splicing the specific categories of video data on the left and right sides of the cropped video data respectively, the third left and right spliced ​​video data are obtained, so that the video content of the left and right perspectives can be integrated, and the picture is more coherent and complete, which is convenient for the driver to clearly and comprehensively obtain the corresponding perspective information from different sides and broaden the observation field; then the fourth V-category video data is obtained to enrich the content dimension of the video, and finally the spliced ​​video data is generated based on these data to achieve the organic integration of video data of different perspectives and different categories, presenting multi-perspective video data to the driver, comprehensively assisting observation and judgment during driving, and improving the driving experience.

[0016] Further, the spliced ​​video data is rendered to obtain rendered video data; the first video data in the spliced ​​video data is obtained, and the first video data is enhanced and rendered to obtain the fourth video data; the second video data, the third left-side spliced ​​video data, the third right-side spliced ​​video data and the fourth V-class video data in the spliced ​​video data are obtained; the fourth video data, the second video data, the third left-side spliced ​​video data, the third right-side spliced ​​video data and the fourth V-class video data are UI rendered to obtain rendered video data. By performing enhanced rendering on the defective first video data, its quality is improved, its value is enhanced, and the foundation for the overall video quality is laid; at the same time, multiple types of video data are obtained and UI rendered, the color tone style is unified to make the video more coordinated and beautiful and enhance the clarity, and OSD information such as prompts and warning icons are added to provide important assistance to the driver, which improves the functionality and information transmission ability of the video; overall, the overall value and availability of video data are improved, which helps to improve driving safety and operation accuracy, and brings better viewing experience and practical value to the driver.

[0017] The beneficial effects of the electronic rearview mirror field of view capture method of the present invention are:

[0018] This solution starts with the optimization of the data processing flow, effectively reducing power consumption and costs, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; based on the vehicle kinematic model combined with the vehicle driving data, the video data is cropped, which can accurately remove irrelevant images, focus on the key field of view area, reduce unnecessary data processing, and thus reduce system power consumption; using preset detection algorithms, result analysis and other steps, it can intelligently determine whether further processing and cropping of video data is required, making the entire processing process more targeted and efficient, and further reducing system power consumption; secondly, it also has obvious advantages in terms of cost, adopting a more economical processing method, reducing the overall cost, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; through the precise processing and splicing rendering of video data, it provides drivers with a clear and coherent view of the outside of the vehicle; especially in the case of blind spots in the field of vision such as turning of commercial vehicles, it can automatically adjust the field of view display, greatly improving driving safety, enabling drivers to better understand the surrounding environment of the vehicle, meeting regulatory requirements, and providing a strong guarantee for the safe driving of commercial vehicles. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0020] Figure 1 A first flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0021] Figure 2 A second flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0022] Figure 3 A third flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0023] Figure 4 A fourth flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0024] Figure 5 A fifth flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0025] Figure 6 A sixth flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0026] Figure 7 A seventh flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0027] Figure 8 An eighth flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0028] Fig. 9 A ninth flow chart of a method for capturing the visual field of an electronic rearview mirror provided by an embodiment of the present invention;

[0029] Fig.10 The present invention provides a tenth flow chart of a method for capturing the field of view of an electronic rearview mirror according to an embodiment of the present invention.

[0030] Fig.11 The present invention provides an eleventh flow chart of an electronic rearview mirror field of view capturing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] like Figure 1-11 As shown, the present invention provides a method for capturing the field of view of an electronic rearview mirror.

[0034] To solve the above technical problems, the technical solution adopted by the present invention is as follows. Figure 1 , an embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0035] 101: Acquire sensor data from various sensors, and obtain five channels of video data according to the sensor data analysis;

[0036] 102: Obtain vehicle driving data from the sensor data;

[0037] 103: Performing a cropping operation on the five-channel video data based on a preset vehicle kinematics model and vehicle driving data to obtain cropped video data;

[0038] The video data can be cropped based on the vehicle kinematics model combined with the vehicle driving data, which can accurately remove irrelevant images, focus on the key field of view, and reduce unnecessary data processing;

[0039] 104: Perform defect detection on the cropped video data using a preset detection algorithm to obtain a detection result;

[0040] 105: Analyze the test results;

[0041] By using preset detection algorithms, result analysis and other steps, it can intelligently determine whether the video data needs to be further processed and cropped, making the entire processing process more targeted and efficient;

[0042] 106: When the detection result indicates that there are defective cropped video data, preprocessing the cropped video data based on the detection result to obtain spliced ​​video data;

[0043] 107: On the contrary, performing splicing processing on the cropped video data to obtain spliced ​​video data;

[0044] 108: Rendering the spliced ​​video data to obtain rendered video data;

[0045] Through precise processing and splicing of video data, it provides the driver with a clear and coherent view of the outside world.

[0046] 109: parsing the rendered video data to obtain three channels of video data;

[0047] 110: Generate display video information according to the three-channel video data.

[0048] By analyzing and transmitting the three-way video data to generate display video information, it provides the driver and passengers with a clear multi-angle view outside the vehicle, improving driving safety and experience.

[0049] In this embodiment, the system architecture covers the left rearview mirror, the right rearview mirror, the left field of view display, the right field of view display and the blind spot display; wherein the left rearview mirror includes one camera each for the Class II field of view and the Class IV field of view, and the right rearview mirror includes one camera each for the Class II field of view, the Class IV field of view and the Class V field of view;

[0050] The image of the left-side visual field display is obtained by detecting, rendering, cutting and splicing the images captured by the Class II visual field camera and the Class IV visual field camera of the left rearview mirror;

[0051] The image of the right-side visual field display is obtained by detecting, rendering, cropping and splicing the images captured by the Category II and Category IV visual field cameras of the right rearview mirror;

[0052] The image of the blind spot display is obtained by the image captured by the V-view camera of the right rearview mirror after detection and rendering;

[0053] In step 101, the five-channel video data includes image data captured by the Class II field of view camera and the Class IV field of view camera of the left rearview mirror, image data captured by the Class II field of view camera and the Class IV field of view camera of the right rearview mirror, and image data captured by the Class V field of view camera of the right rearview mirror;

[0054] In step 108, rendering processing operations are performed on the spliced ​​video data, including rendering vehicle distance auxiliary lines, reversing auxiliary lines, etc. for the II and IV video data in the spliced ​​video data; rendering OSD information such as prompts and warning icons for the II, IV and V video data in the spliced ​​video data;

[0055] The system architecture also covers a controller, which includes an FPGA chip and FLASH, wherein the FPGA is a type of FPGA chip that does not include an SoC system-on-chip, and this chip has the advantages of low power consumption and low cost. In step 110, the three-channel video data are respectively transmitted to their respective GMSL serializers in the form of LVDS signals through different LVDS interfaces of the FPGA chip; each GMSL serializer serially packages the LVDS signal from the corresponding LVDS interface; the packaged signal is transmitted to the corresponding GMSL deserializer in the display terminal; each GMSL deserializer deserializes the received serial signal and transmits it to the corresponding display, and the display types include a left field of view display, a right field of view display, and a blind spot display; the left field of view display is used to display the left-side spliced ​​video data of the three-channel video, the right field of view display is used to display the right-side spliced ​​video data of the three-channel video, and the blind spot display is used to display the Class V video data of the three-channel video. The driver can observe the required outside view of the vehicle through the presentation images of the left field of view display, the right field of view display, and the blind spot display.

[0056] Starting from the optimization of data processing flow, power consumption and cost are effectively reduced, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; based on the vehicle kinematic model combined with vehicle driving data, the video data is cropped, which can accurately remove irrelevant images, focus on key visual areas, reduce unnecessary data processing, and thus reduce system power consumption; using preset detection algorithms, result analysis and other steps, it can intelligently determine whether further processing and cropping of video data is required, making the entire processing process more targeted and efficient, and further reducing system power consumption; secondly, it also has obvious advantages in terms of cost, adopting a more economical processing method, reducing the overall cost, which is conducive to the large-scale promotion and application of electronic rearview mirrors for commercial vehicles; through precise processing and splicing rendering of video data, it provides drivers with a clear and coherent view of the outside of the vehicle; especially in the case of blind spots that are prone to occur when commercial vehicles turn, the field of view display can be automatically adjusted, which greatly improves driving safety and enables drivers to better understand the surrounding environment of the vehicle, meeting regulatory requirements while providing a strong guarantee for the safe driving of commercial vehicles.

[0057] See also Figure 2 A second embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0058] 201: Acquire various sensor data to obtain sensor data, which lays a solid foundation for subsequent video data processing;

[0059] 202: Obtain video data from the sensor data, and package the video data to obtain first CSI-2 data. This step ensures the integrity and availability of video information and provides high-quality raw materials for further processing.

[0060] 203: Convert the first CSI-2 data into serial data. This data serial conversion process is a key link in data transmission and processing, which enables data to be efficiently transmitted between different systems and devices.

[0061] 204: deserialize and aggregate the serial data to obtain two channels of second CSI-2 data. This step not only improves the flexibility of data processing, but also provides a solid foundation for subsequent video data analysis.

[0062] 205: The second CSI-2 data is parsed to obtain five channels of video data. This parsing process demonstrates the system's deep understanding and processing capabilities of video data, and also ensures that the final output video data has a high degree of accuracy and availability.

[0063] In this embodiment, in step 202, the video data in the sensor data is obtained, that is, the real-time image information of the road condition collected by the vehicle camera module is obtained. M IPI-CSI-2 is a camera data transmission interface, which can aggregate multiple channels of video data into one channel for packaging and output to obtain the first CSI-2 data; the FPGA chip is connected to the camera and the display using SerDes (Serializer-Deserializer) technology to achieve long-distance, anti-interference and high-speed transmission, and GMSL (Gigabit Multimedia Ser ia lLi nk, Gigabit Multimedia Serial Link) is one of the solutions using SerDes technology. In step 203, the first CSI-2 data is serialized by the GMSL serializer to obtain serial data, and the serial data is transmitted through the GMSL link; in step 204, the GMSL deserializer deserializes the GMSL link data received by the controller and aggregates them into two channels of CSI-2 data, and then transmits the two channels of CSI-2 data to the FPGA chip; in step 205, the two channels of second CSI-2 data are received by the MIPI CSI-2D-PHY module in the FPGA chip, and the second CSI-2 data is parsed into five channels of video data;

[0064] Acquiring data from various sensors lays the foundation for video processing. Next, the video data is packaged to obtain the first CSI-2 data to ensure information integrity and provide high-quality raw materials for subsequent video processing. Data serial conversion facilitates efficient transmission between different devices. Deserialization and aggregation obtain two channels of second CSI-2 data, which improves processing flexibility and prepares for analysis. Finally, the second CSI-2 data is analyzed to obtain five channels of video data, demonstrating the system's deep data processing capabilities, ensuring the accuracy and availability of the output video, and providing reliable protection for subsequent video applications.

[0065] See also Figure 3 A third embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0066] 301: Determine whether the vehicle is in a reverse state based on the vehicle driving data;

[0067] The reversing status is determined by the vehicle driving data, which provides a key decision-making basis for the subsequent acquisition of display ratios of different viewing angles and further processing;

[0068] 302: When the vehicle is in a reversing state, obtaining steering angle data from the vehicle driving data;

[0069] 303: judging whether the vehicle is in a turning state according to the steering angle data;

[0070] 304: When the vehicle is in a turning state, the offset view angle screen coordinates are calculated based on the vehicle kinematics model and the vehicle driving data;

[0071] The vehicle's steering state is determined based on the steering angle data, which further refines the analysis of the vehicle's driving state. When in the steering state, the offset view coordinates are calculated based on the vehicle's kinematic model and vehicle driving data, which can provide a basis for subsequent accurate adjustment of the video screen.

[0072] 305: Obtaining a preset reversing viewing angle display ratio;

[0073] 306: Acquire all video data except the V-type video data from the five-channel video data to obtain four-channel video data;

[0074] 307: performing a cropping operation on each video frame in the four-channel video data according to the offset view angle screen coordinates and the reversing view angle display ratio to obtain preliminary cropping data;

[0075] 308: When the vehicle is not in a reverse state, obtaining steering angle data in the vehicle driving data;

[0076] 309: Determine whether the vehicle is in a turning state according to the steering angle data;

[0077] 310: When the vehicle is not in a turning state, the normal viewing angle screen coordinates are calculated based on the vehicle kinematics model and the vehicle driving data;

[0078] 311: Get the preset normal viewing angle display ratio;

[0079] When the vehicle is in a reversing state, the reversing viewing angle display ratio is obtained, and when the vehicle is in a non-reversing state, the normal viewing angle display ratio is obtained. The acquisition of these ratios provides accurate data support for subsequent video data adjustment;

[0080] 312: Perform a cropping operation on each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio calculation to obtain preliminary cropping data;

[0081] Through the cropping operation, irrelevant images can be accurately removed to focus on the key field of view, reducing the subsequent unnecessary data processing and unnecessary video data transmission, optimizing the use of system resources, and thus reducing system power consumption;

[0082] 313: Obtain V-category video data from five channels of video data to obtain first V-category video data;

[0083] 314: Generate cropped video data according to the cropped preliminary data and the first V-type video data.

[0084] In this embodiment, the vehicle kinematic model is a mathematical model that describes the vehicle motion state, and is used to analyze the vehicle motion in a plane. It is mainly based on geometric relationships and kinematic principles, and does not consider factors such as the vehicle's power system and tire forces. The components of the vehicle kinematic model include a vehicle coordinate system, kinematic equations, vehicle geometric parameters, and a reversing state. The vehicle driving data includes steering angle data, steering wheel angle, vehicle speed signal, speed-related state, steering-related state, and vehicle posture state. The FPGA chip can be connected to the vehicle MCU (microcontroller unit) to obtain information such as vehicle driving data. The controller also includes a HyperRAM memory, which is a type of RAM memory and is suitable for vehicle-mounted applications that have strict requirements on power consumption and volume and do not have particularly large bandwidth throughput. After obtaining the preliminary cut data in steps 307 and 312, the computer caches the preliminary cut data in the HyperRAM memory for subsequent video splicing output and image detection.

[0085] The reversing state and steering state are judged through vehicle driving data, providing key decision-making basis for video display screen adjustment; the corresponding viewing angle display ratio is obtained according to different vehicle states, and then the vehicle is judged whether it is in a turning state or a non-turning state according to the corresponding viewing angle display ratio; for the turning and non-turning states, the relevant screen coordinates are calculated, and each video frame is cropped according to the relevant screen coordinates and display ratio calculation to obtain preliminary cropping data, which can accurately remove irrelevant pictures, reduce unnecessary data processing and video data transmission, optimize system resource usage, and reduce system power consumption; finally, combined with Class V video data, cropped video data is generated to provide the driver with clearer, more accurate and optimized visual information, improve driving safety and comfort, and reflect the advanced technical level of intelligent vehicle systems in video processing.

[0086] See also Figure 4 A fourth embodiment of a method for capturing a visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0087] 401: Obtain the clock cycle in the sensor data, and obtain a pixel point coordinate set of each video frame according to the clock cycle count;

[0088] Using the clock cycle in the sensor data to determine the pixel coordinate set of each video frame can ensure that the pixel resolution of each video frame has an accurate time reference;

[0089] 402: Determine, according to the pixel point coordinate set, whether there is a pixel point whose coordinates are not within a preset area;

[0090] Calculate the coordinates of each pixel point, clarify the specific position of each pixel point in the video frame, and determine whether the pixel point is effective according to the area range. Provide accurate position information, which helps to achieve refined processing of the video picture and improve the quality and clarity of the video picture;

[0091] 403: When there is a pixel point whose coordinates are not within the region range, mark the pixel point whose coordinates are not within the region range as an invalid pixel point to obtain an invalid pixel point set;

[0092] By judging whether the pixel coordinates are within the preset area, the pixels that may have problems can be quickly screened out, providing a basis for the subsequent processing of invalid pixels, which helps to improve the efficiency and accuracy of video processing; marking invalid pixels can clarify which pixels contribute less to the video image or may interfere, which helps to remove these invalid pixels in a targeted manner and improve the quality and effectiveness of the video image;

[0093] 404: Calculate the cropping start coordinates of each video frame according to the offset view angle screen coordinates and the reversing view angle display ratio to obtain a cropping start coordinate set;

[0094] Combined with the offset view coordinates and reversing view display ratio related to the vehicle driving state, the cropping start coordinates of each video frame can be accurately determined, providing the starting reference parameters for the cropping operation for subsequent further processing of the video frame, which helps to achieve more accurate video picture adjustment to adapt to the vehicle driving conditions;

[0095] 405: Obtain the preset crop height and width of each video frame;

[0096] 406: Perform a cropping operation on each video frame according to the invalid pixel point set, the cropping start coordinate set, the cropping height and width to obtain preliminary cropping data.

[0097] Crop each video frame based on the invalid pixel set, crop start coordinate set, crop height and width, which can accurately remove the area where the invalid pixels are located, so that the video image is more focused on the effective content, reduce the subsequent data processing and transmission volume, reduce system power consumption, and improve the quality and clarity of the video image;

[0098] By combining the offset perspective image coordinates of the vehicle's driving state and the reversing perspective display ratio to calculate the cropping start coordinates of each video frame, the starting reference parameters for the cropping operation are provided for the subsequent further processing of the video frames, which facilitates the precise adjustment of the video picture to adapt to the vehicle's driving conditions; then, the pixel point coordinate set is determined using the clock cycle of the sensor data to ensure the accurate time reference of the pixel resolution; the pixel point coordinates are calculated to clarify their positions, providing a basis for judging the validity, which helps to refine the processing of the video picture and improve the quality and clarity; by judging whether the pixel points are within the preset area, the problem pixels can be quickly screened to improve processing efficiency and accuracy; marking invalid pixels can remove interference in a targeted manner and improve the picture quality and validity; finally, each video frame is cropped based on the invalid pixel point set, the cropping start coordinate set, the cropping height and width to remove the invalid area, reduce the amount of data processing and transmission, and reduce system power consumption.

[0099] See also Figure 5 A fifth embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0100] 501: Obtain the clock cycle in the sensor data, and obtain a pixel point coordinate set of each video frame according to the clock cycle count;

[0101] Using the clock cycle in the sensor data to determine the pixel coordinate set of each video frame can ensure that the pixel resolution of each video frame has an accurate time reference;

[0102] 502: Determine, according to the pixel point coordinate set, whether there is a pixel point whose coordinates are not within a preset area;

[0103] Calculate the coordinates of each pixel point, clarify the specific position of each pixel point in the video frame, and determine whether the pixel point is effective according to the area range. Provide accurate position information, which helps to achieve refined processing of the video picture and improve the quality and clarity of the video picture;

[0104] 503: When there are pixels whose coordinates are not within the region, the pixels whose coordinates are not within the region are marked as invalid pixels to obtain an invalid pixel set;

[0105] By judging whether the pixel coordinates are within the preset area, the pixels that may have problems can be quickly screened out, providing a basis for the subsequent processing of invalid pixels, which helps to improve the efficiency and accuracy of video processing; marking invalid pixels can clarify which pixels contribute less to the video image or may interfere, which helps to remove these invalid pixels in a targeted manner and improve the quality and effectiveness of the video image;

[0106] 504: Calculate the cropping start coordinates of each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio to obtain a cropping start coordinate set;

[0107] Combined with the normal viewing angle image coordinates and normal viewing angle display ratio related to the vehicle driving state, the cropping start coordinates of each video frame can be accurately determined, providing the starting reference parameters for the cropping operation for subsequent further processing of the video frame, which helps to achieve more accurate video image adjustment to adapt to the vehicle driving conditions;

[0108] 505: Obtain the preset cropping height and width of each video frame;

[0109] 506: Perform a cropping operation on each video frame according to the invalid pixel point set, the cropping start coordinate set, the cropping height and width to obtain preliminary cropping data.

[0110] Crop each video frame based on the invalid pixel set, crop start coordinate set, crop height and width, which can accurately remove the area where the invalid pixels are located, so that the video image is more focused on the effective content, reduce the subsequent data processing and transmission volume, reduce system power consumption, and improve the quality and clarity of the video image;

[0111] By combining the normal viewing angle picture coordinates and the normal viewing angle display ratio, the cropping start coordinates of each video frame are calculated, which provides the starting reference parameters for the cropping operation for the subsequent further processing of the video frame, which helps to adapt to the vehicle driving conditions for precise picture adjustment; the sensor data clock cycle is used to determine the pixel point coordinate set to ensure the accurate time reference of pixel resolution; the pixel point coordinates are calculated to clarify the position, provide a basis for judging the effectiveness, realize refined processing, and improve the video picture quality and clarity; by judging whether the pixel point is in the preset area, the problem pixel points can be quickly screened to improve processing efficiency and accuracy; invalid pixels are marked to facilitate targeted removal of interference and improve picture quality and effectiveness; finally, each video frame is cropped based on the invalid pixel point set, the cropping start coordinate set, the cropping height and width, the invalid area is removed, the data processing and transmission volume is reduced, the system power consumption is reduced, the video picture focuses on the effective content, and the overall effect and performance of video processing are improved.

[0112] See also Figure 6 A sixth embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0113] 601: Use a detection algorithm to perform a round-robin defect detection on each channel of the cropped video data to obtain a detection result.

[0114] By using detection algorithms to detect and crop video data in a round-robin manner, some basic problems of each video data can be comprehensively and accurately analyzed, such as color deviation, uneven brightness or inconsistent frame rate, providing reliable detection results for subsequent processing.

[0115] In this embodiment, in step 601, the detection operation of the detection algorithm includes detecting whether each video frame in each channel of video data is blocked, blurred, etc., for example: using the Laplace operator convolution operation on the video frame image to obtain the variance of the convolution result; judging whether the variance is greater than a preset first threshold; when the variance is greater than the first threshold, generating a judgment result that blur exists; when the variance is less than the first threshold, generating a judgment result that blur does not exist; for example: converting the video frame image into a grayscale image; based on the grayscale image and Sobe l operator calculates a gradient map; writes the gradient map into the HyperRAM memory; reads multiple frames of gradient maps from the HyperRAM memory, and averages the multiple frames of gradient maps to obtain an average gradient map; performs a Gaussian filtering operation on the average gradient map to obtain a filtered gradient map; calculates an inverse binary gradient map based on the filtered gradient map and the inverse binarization method; performs an expansion operation on the inverse binary gradient map to obtain an expanded gradient map; calculates the proportion of white pixels in the expanded gradient map; determines whether the proportion of white pixels exceeds a preset second threshold; when the proportion of white pixels exceeds the preset second threshold, generates a determination result that an occluder exists; otherwise, generates a determination result that an occluder does not exist.

[0116] See also Figure 7 A seventh embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0117] 701: When the detection result shows that defective cropped video data exists, all defective cropped video data are set as defective video data;

[0118] By explicitly classifying defective cropped video data as defective video data, the problematic video data can be clearly identified and distinguished; this helps to process these defective data in a targeted manner in the future, avoiding confusion between them and normal, defect-free data, laying the foundation for further repair, splicing and other operations, and facilitating subsequent processes to accurately know which data are problematic data that need special treatment;

[0119] 702: performing a splicing operation on defective video data to obtain first video data;

[0120] 703: performing a splicing operation on the remaining cropped video data without defects to obtain second video data;

[0121] 704: Generate spliced ​​video data according to the first video data and the second video data.

[0122] In this embodiment, video data that has undergone different processing flows is logically integrated, which can reduce memory bandwidth requirements, optimize the use of system resources, and improve the efficiency of overall video transmission processing.

[0123] In this embodiment, when defective cropped video data is detected, the defective video data and the non-defective cropped video data are spliced ​​together to obtain first video data and second video data. This divide-and-conquer processing method helps to better integrate video clips in different states. Finally, spliced ​​video data is generated based on the first video data and the second video data, thereby achieving an organic integration of defective and non-defective videos. At the same time, the entire process reduces memory bandwidth requirements through logical integration, optimizes system resource usage, and improves the efficiency of overall video transmission processing.

[0124] See also Figure 8 An eighth embodiment of a method for capturing the visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0125] 801: Obtain left-side category II video data and left-side category IV video data in the defective video data, and perform a splicing operation on the left-side category II video data and the left-side category IV video data to obtain first left-side spliced ​​video data;

[0126] The left-side category II video data and the left-side category IV video data are spliced ​​to integrate the left-side video resources of a specific category, making the video content of the left-side perspective more complete and coherent and facilitating the subsequent video data transmission;

[0127] 802: Acquire right-side category II video data and right-side category IV video data in the defective video data, and perform a splicing operation on the right-side category II video data and the right-side category IV video data to obtain first right-side spliced ​​video data;

[0128] The right-side Category II video data and the right-side Category IV video data are spliced ​​together. Similarly, the specific video resources of the right-side perspective can be integrated to ensure that the video content of the right-side perspective has better coherence and integrity, so that the driver can obtain richer information from different angles.

[0129] 803: Acquire the V-class video data in the defective video data to obtain the second V-class video data;

[0130] 804: Generate first video data according to the first left-side spliced ​​video data, the first right-side spliced ​​video data, and the second V-type video data.

[0131] Generate first video data according to first left-side spliced ​​video data, first right-side spliced ​​video data and second V-type video data, thereby realizing organic integration of video data of different types and different viewing angles;

[0132] In this embodiment, the left-side Class II and left-side Class IV video data in the defective video data are spliced ​​respectively, and the video resources of the specific category on the left are integrated to make the left-side perspective video content more complete and coherent, which is conducive to subsequent transmission; similarly, the right-side Class II and right-side Class IV video data are spliced ​​to ensure that the right-side perspective video has better coherence and integrity, which is convenient for the driver to obtain rich information from different angles; then, the Class V video data in the defective video data is obtained to provide an important component for video data integration; finally, the first video data is generated according to the first left-side spliced ​​video data, the first right-side spliced ​​video data and the second Class V video data, thereby realizing the organic integration of video data of different types and different perspectives, providing the driver with a comprehensive video picture, improving the overall quality and value of the video, and meeting the diverse needs in practical applications.

[0133] See also Fig. 9 A ninth embodiment of a method for capturing a visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0134] 901: setting the remaining cropped video data without defects as the third video data;

[0135] Setting the remaining cropped video data without defects as the third video data helps to directly perform a separate splicing operation on the defect-free video resources, avoiding additional preliminary rendering processing on these data, thereby saving processing time and computing resources. This optimization enables the video processing system to respond to and process video streams faster, thereby improving the overall video processing efficiency;

[0136] 902: Obtain left-side category II video data and left-side category IV video data in the third video data, and perform a splicing operation on the left-side category II video data and the left-side category IV video data to obtain second left-side spliced ​​video data;

[0137] The left-side Category II video data and the left-side Category IV video data are spliced ​​to integrate the video content of the specific category on the left side to form a coherent picture, making the video presented from the left perspective more complete and smooth, so that the driver can obtain the information of the corresponding perspective more clearly and comprehensively;

[0138] 903: Acquire right-side category II video data and right-side category IV video data in the third video data, and perform a splicing operation on the right-side category II video data and the right-side category IV video data to obtain second right-side spliced ​​video data;

[0139] The right-side category II video data and the right-side category IV video data in the third video data are obtained and spliced ​​to obtain the second right-side spliced ​​video data, so as to integrate the relevant video resources of the right-side perspective, so as to make the video content of the right-side perspective more coherent and complete, which is beneficial for the driver to fully observe the relevant situation from the right-side perspective and enrich the observation perspective;

[0140] 904: Acquire V-category video data in the third video data to obtain third V-category video data;

[0141] Acquiring the V-category video data in the third video data to obtain the third V-category video data can enrich the content dimension of the video presentation;

[0142] 905: Generate second video data according to the second left-side spliced ​​video data, the second right-side spliced ​​video data and the third V-type video data;

[0143] The second video data is generated according to the second left stitched video data, the second right stitched video data and the third V-category video data, so as to realize the organic integration of video data of different viewing angles and different categories, and provide multi-viewing video data for the driver;

[0144] In this embodiment, the defect-free cropped video data is set as the third video data, which saves processing time and computing resources, improves the overall video processing efficiency, and makes the system respond more quickly. Then, by splicing the left and right specific category video data respectively, the second left and right spliced ​​video data are obtained, so that the left and right side perspective videos are more complete and coherent, which is convenient for the driver to clearly and comprehensively obtain multi-perspective information; obtaining the third V-class video data further enriches the video content dimension; finally, the second video data is generated based on these data to achieve organic integration, providing the driver with multi-perspective and content-rich video data, and assisting observation and judgment during driving.

[0145] See also Fig.10 A tenth embodiment of a method for capturing visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0146] 111: obtaining left-side category II video data and left-side category IV video data in the cropped video data, and performing a splicing operation on the left-side category II video data and the left-side category IV video data to obtain third left-side spliced ​​video data;

[0147] The left-side Category II video data and the left-side Category IV video data are spliced ​​to integrate the video content of the specific category on the left side to form a coherent picture, making the video presented from the left perspective more complete and smooth, so that the driver can obtain the information of the corresponding perspective more clearly and comprehensively;

[0148] 112: Acquire right-side category II video data and right-side category IV video data in the cropped video data, and perform a splicing operation on the right-side category II video data and the right-side category IV video data to obtain third right-side spliced ​​video data;

[0149] The right-side category II video data and the right-side category IV video data in the cropped video data are obtained and spliced ​​to obtain the third right-side spliced ​​video data, which can integrate the relevant video resources of the right-side perspective, so that the video content of the right-side perspective is more coherent and complete, which is beneficial for the driver to fully observe the relevant situation from the right angle and enrich the observation perspective;

[0150] 113: Acquire the V-category video data in the cropped video data to obtain fourth V-category video data;

[0151] Acquiring the V-category video data in the cropped video data to obtain the fourth V-category video data can enrich the content dimension of the video presentation;

[0152] 114: Generate spliced ​​video data according to the third left spliced ​​video data, the third right spliced ​​video data and the fourth V-category video data;

[0153] Generate spliced ​​video data according to the third left spliced ​​video data, the third right spliced ​​video data and the fourth V-category video data, realize the organic integration of video data of different viewing angles and different categories, and provide multi-view video data for the driver;

[0154] In this embodiment, by splicing the specific categories of video data on the left and right sides of the cropped video data respectively, the third left and right spliced ​​video data are obtained, so that the video content of the left and right perspectives can be integrated, and the picture is more coherent and complete, which is convenient for the driver to clearly and comprehensively obtain the corresponding perspective information from different sides and broaden the observation field; then the fourth V-category video data is obtained to enrich the content dimension of the video, and finally the spliced ​​video data is generated based on these data to achieve the organic integration of video data of different perspectives and different categories, presenting multi-perspective video data to the driver, and comprehensively assisting observation and judgment during driving, thereby improving the driving experience.

[0155] See also Fig.11 , an eleventh embodiment of a method for capturing visual field of an electronic rearview mirror in an embodiment of the present invention includes:

[0156] 115: Obtain first video data from the spliced ​​video data, and perform enhanced rendering processing on the first video data to obtain fourth video data;

[0157] In this embodiment, since the first video data is formed by splicing originally defective video data, the enhanced rendering processing is performed by using a histogram equalization algorithm, etc., and key indicators such as the contrast of the video data can be adjusted in a targeted manner. This can correct problems such as dark images and unclear layers caused by the original defects, making the details of the video images clearer and more recognizable, effectively improving the quality of defective videos, and maximally mining and enhancing the value of this part of the problematic video data so that it can be better integrated into the overall video, laying the foundation for the subsequent generation of high-quality final videos, and reducing the adverse effects of some data defects on the overall video viewing experience;

[0158] 116: Acquire second video data, third left-side spliced ​​video data, third right-side spliced ​​video data, and fourth V-category video data in the spliced ​​video data;

[0159] 117: Perform UI rendering processing on the fourth video data, the second video data, the third left-side spliced ​​video data, the third right-side spliced ​​video data, and the fourth V-category video data to obtain rendered video data.

[0160] In this embodiment, the UI rendering processing operation includes rendering color, rendering prompts, warning icons and other OSD information for the video data; by rendering color operations on various types of video data, the overall color tone style of the video can be unified and optimized, making the video screen look more coordinated, beautiful, and visually impactful, and enhancing the clarity of the video; rendering prompts and warning icons and other OSD (screen display) information can provide the driver with important auxiliary information, such as timely prompts of some key content, precautions, or warning signs when abnormal situations occur in the video, thereby improving the functionality and information transmission capabilities of the video. In this way, the driver can better understand the content conveyed by the video, so that the video is not just a simple picture display, but also has more guidance and prompt functions, which helps to improve driving safety and operation accuracy.

[0161] By performing enhanced rendering processing on the defective first video data, its quality is improved and its value is enhanced, laying the foundation for the overall video quality. At the same time, multiple types of video data are obtained and UI rendering processing is performed, and the unified color style makes the video more coordinated and beautiful and enhances clarity. OSD information such as prompts and warning icons are added to provide important assistance to the driver, which improves the functionality and information transmission ability of the video. Overall, the overall value and usability of video data are improved, which helps to improve driving safety and operation accuracy, and brings better viewing experience and practical value to the driver.

[0162] Finally, it should be noted that the above description is only a preferred example of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for capturing the visual field of an electronic rearview mirror, characterized in that: include: Acquire sensor data from various sensors and parse the sensor data to obtain five channels of video data; Acquire vehicle driving data from sensor data; Performing a cropping operation on the five-channel video data based on a preset vehicle kinematics model and vehicle driving data to obtain cropped video data; Using a preset detection algorithm to perform defect detection on the cropped video data to obtain a detection result; Analyze the test results; When the detection result indicates that there are defective cropped video data, the cropped video data is preprocessed based on the detection result to obtain spliced ​​video data; On the contrary, the cropped video data is spliced ​​to obtain spliced ​​video data; Rendering the spliced ​​video data to obtain rendered video data; Parsing the rendered video data to obtain three channels of video data; Generate display video information based on three channels of video data.

2. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 1, characterized in that: The sensor data is obtained from various sensors, and five channels of video data are obtained according to the sensor data analysis, including: Acquire various sensor data to obtain sensor data; Acquire video data from the sensor data, and pack the video data to obtain first CSI-2 data; Converting the first CSI-2 data into serial data; Deserialize and aggregate the serial data to obtain two-way second CSI-2 data; The second CSI-2 data is parsed to obtain five channels of video data.

3. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 1, characterized in that: The cropping operation on the five-channel video data based on the preset vehicle kinematics model and the vehicle driving data to obtain cropped video data includes: Determining whether the vehicle is in a reversing state based on vehicle driving data; When the vehicle is in a reversing state, the steering angle data in the vehicle driving data is obtained; Determine whether the vehicle is in a steering state according to the steering angle data; When the vehicle is in a turning state, the offset view angle screen coordinates are calculated based on the vehicle kinematic model and vehicle driving data; Get the preset reversing viewing angle display ratio; Acquire all video data except the V-type video data from the five-channel video data to obtain four-channel video data; Performing a cropping operation on each video frame in the four-channel video data according to the offset view angle screen coordinates and the reversing view angle display ratio to obtain preliminary cropping data; When the vehicle is not in a reverse state, the steering angle data in the vehicle driving data is obtained; Determine whether the vehicle is in a steering state according to the steering angle data; When the vehicle is not in a turning state, the normal viewing angle coordinates are calculated based on the vehicle kinematic model and vehicle driving data; Get the preset normal viewing angle display ratio; Perform cropping operation on each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio calculation to obtain preliminary cropping data; Acquire V-category video data from five channels of video data to obtain first V-category video data; The cropped video data is generated based on the cropped preliminary data and the first V-type video data.

4. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 3, characterized in that: The step of performing a cropping operation on each video frame in the four-channel video data according to the offset view angle screen coordinates and the reversing view angle display ratio to obtain preliminary cropping data includes: Obtain the clock cycle in the sensor data, and obtain the pixel coordinate set of each video frame according to the clock cycle count; Determine whether there is a pixel point coordinate outside the preset area according to the pixel point coordinate set; When there are pixels whose coordinates are not within the region, the pixels whose coordinates are not within the region are marked as invalid pixels to obtain an invalid pixel set; Calculate the cropping start coordinates of each video frame according to the offset view angle screen coordinates and the reversing view angle display ratio to obtain a cropping start coordinate set; Get the preset crop height and width of each video frame; A cropping operation is performed on each video frame according to the invalid pixel point set, the cropping start coordinate set, the cropping height and the cropping width to obtain preliminary cropping data.

5. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 3, characterized in that: The cropping operation is performed on each video frame according to the normal viewing angle picture coordinates and the normal viewing angle display ratio to obtain preliminary cropping data, including: Obtain the clock cycle in the sensor data, and obtain the pixel coordinate set of each video frame according to the clock cycle count; Determine whether there is a pixel point coordinate outside the preset area according to the pixel point coordinate set; When there are pixels whose coordinates are not within the region, the pixels whose coordinates are not within the region are marked as invalid pixels to obtain an invalid pixel set; Calculate the cropping start coordinates of each video frame according to the normal viewing angle screen coordinates and the normal viewing angle display ratio to obtain a cropping start coordinate set; Get the preset crop height and width of each video frame; A cropping operation is performed on each video frame according to the invalid pixel point set, the cropping start coordinate set, the cropping height and the cropping width to obtain preliminary cropping data.

6. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 1, characterized in that: The method of using a preset detection algorithm to perform defect detection on the cropped video data to obtain a detection result includes: A detection algorithm is used to perform a round-robin defect detection on each channel of video data in the cropped video data to obtain a detection result.

7. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 1, characterized in that: When the detection result is that there is defective cropped video data, the cropped video data is preprocessed based on the detection result to obtain spliced ​​video data, including: When the detection result is that there are defective cropped video data, all defective cropped video data are set as defective video data; Performing a splicing operation on the defective video data to obtain first video data; Performing a splicing operation on the remaining cropped video data without defects to obtain second video data; The spliced ​​video data is generated according to the first video data and the second video data.

8. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 7, characterized in that: The step of performing a splicing operation on the defective video data to obtain the first video data includes: Acquire left-side class II video data and left-side class IV video data in the defective video data, and perform a splicing operation on the left-side class II video data and the left-side class IV video data to obtain first left-side spliced ​​video data; Acquire right-side class II video data and right-side class IV video data in the defective video data, and perform a splicing operation on the right-side class II video data and the right-side class IV video data to obtain first right-side spliced ​​video data; Acquire the V-class video data in the defective video data to obtain the second V-class video data; The first video data is generated according to the first left-side spliced ​​video data, the first right-side spliced ​​video data and the second V-type video data.

9. The method for capturing the visual field of an electronic rearview mirror as claimed in claim 7, characterized in that: The step of performing a splicing operation on the remaining cropped video data without defects to obtain the second video data includes: The remaining cropped video data without defects is set as the third video data; Acquire left-side category II video data and left-side category IV video data in the third video data, and perform a splicing operation on the left-side category II video data and the left-side category IV video data to obtain second left-side spliced ​​video data; Acquire right-side category II video data and right-side category IV video data in the third video data, and perform a splicing operation on the right-side category II video data and the right-side category IV video data to obtain second right-side spliced ​​video data; Acquire the V-class video data in the third video data to obtain third V-class video data; The second video data is generated according to the first left-side spliced ​​video data, the first right-side spliced ​​video data and the third V-class video data.

10. The method for capturing the visual field of an electronic rearview mirror according to claim 1, characterized in that: On the contrary, the cropped video data is spliced ​​to obtain spliced ​​video data, including: Acquire left-side category II video data and left-side category IV video data in the cropped video data, and perform a splicing operation on the left-side category II video data and the left-side category IV video data to obtain third left-side spliced ​​video data; Acquire right-side category II video data and right-side category IV video data in the cropped video data, and perform a splicing operation on the right-side category II video data and the right-side category IV video data to obtain third right-side spliced ​​video data; Acquire the V-category video data in the cropped video data to obtain fourth V-category video data; The spliced ​​video data is generated according to the third left-side spliced ​​video data, the third right-side spliced ​​video data and the fourth V-type video data.

11. The method for capturing the visual field of an electronic rearview mirror according to claim 1, characterized in that: The spliced ​​video data is rendered to obtain rendered video data; Acquire first video data from the spliced ​​video data, and perform enhanced rendering processing on the first video data to obtain fourth video data; Acquire the second video data, the third left-side spliced ​​video data, the third right-side spliced ​​video data, and the fourth V-category video data in the spliced ​​video data; The fourth video data, the second video data, the third left-side spliced ​​video data, the third right-side spliced ​​video data and the fourth V-class video data are subjected to UI rendering processing to obtain rendered video data.

Images