Electronic outside rear-view mirror control method and device, vehicle and storage medium

Abstract

The invention provides an electronic outside rear-view mirror control method and device, a vehicle and a storage medium, and relates to the technical field of vehicles. The method comprises the following steps: according to an obtained original rear-view image, carrying out image partition processing on the original rear-view image, and determining a plurality of sub-images; according to the content characteristics of each sub-image, allocating processing resources to each sub-image; based on the allocated processing resources, performing differential image processing on each sub-image, and determining a to-be-displayed image corresponding to each sub-image; and generating and outputting a complete target rearview image according to all the to-be-displayed images. According to the method, the original rear-view image is subjected to partition and differentiation processing, intelligent scheduling and efficient utilization of image processing resources are achieved, the display quality of key sub-images is improved, the processing process of non-key sub-images is simplified, the overall processing efficiency and output real-time performance of the rear-view image are remarkably improved, and the user experience is improved. And the driving experience and the driving safety of the user are improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle technology, and more specifically, to an electronic exterior rearview mirror control method, device, vehicle, and storage medium. Background Technology

[0002] In traditional automobiles, exterior rearview mirrors are typically optical, relying on mirror reflection to provide the driver with visual information about the sides and rear of the vehicle. However, optical rearview mirrors, limited by their structural characteristics, usually suffer from the following problems: 1. Limited field of view: Due to the limited imaging angle of the optical mirror, the optical rearview mirror has obvious blind spots. 2. High environmental sensitivity, meaning that the mirror surface of the optical rearview mirror is easily affected by environmental factors such as rain, dust, fog and strong light reflection.

[0003] In response, related technologies utilize camera-based electronic rearview mirrors to overcome the aforementioned problems of traditional optical rearview mirrors. However, electronic rearview mirrors still face new technical challenges in practical applications. For example, image signals acquired from cameras undergo multiple stages, including compression, transmission, decoding, and processing. With limited processing resources, this can introduce significant processing delays, especially in complex traffic environments or high-speed driving scenarios. The corresponding processing speed often fails to meet real-time display requirements, leading to image lag or even dynamic distortion, which in turn affects the driver's judgment and operation, posing safety hazards and reducing the user's driving experience. Summary of the Invention

[0004] The problem this invention addresses is how to improve the real-time display of electronic rearview mirror images in order to enhance the user's driving experience and driving safety.

[0005] To address the aforementioned problems, this invention provides an electronic exterior rearview mirror control method, device, vehicle, and storage medium.

[0006] In a first aspect, the present invention provides an electronic exterior rearview mirror control method, comprising: Based on the acquired original rear view image, the original rear view image is subjected to image partitioning processing to determine multiple sub-images; Based on the content features of each sub-image, allocate processing resources to each sub-image; Based on the allocated processing resources, differentiated image processing is performed on each sub-image to determine the image to be displayed corresponding to each sub-image; Based on all the images to be displayed, generate and output a complete target rear view image.

[0007] Optionally, the step of performing image partitioning processing on the original rear view image to determine multiple sub-images includes: Based on a preset image partitioning rule, the original rear view image is divided into multiple grid units, and the sub-image corresponding to each grid unit is determined.

[0008] Optionally, allocating processing resources to each sub-image based on its content features includes: Image recognition is performed on each of the sub-images to determine the content features of each sub-image; wherein, the content features include environmental features and moving target features; Based on the content characteristics, a preset priority is determined for each sub-image, and processing resources corresponding to the preset priority are allocated to each sub-image.

[0009] Optionally, determining a preset priority for each sub-image based on each content feature, and allocating processing resources corresponding to the preset priority for each sub-image includes: Based on the preset risk coefficients corresponding to the content features of each of the sub-images and the acquired vehicle motion state information, determine the motion weights corresponding to each of the sub-images. Based on the preset motion weight-preset priority correspondence, the preset priority corresponding to the motion weight of the sub-image is determined; According to the preset priority-processing resource allocation ratio correspondence, each sub-image is allocated processing resources corresponding to the processing resource allocation ratio of the preset priority.

[0010] Optionally, after determining the preset priority corresponding to the motion weight of the sub-image, the step of determining the preset priority corresponding to each sub-image based on each content feature and allocating the processing resources corresponding to the preset priority to each sub-image further includes: For the sub-images whose corresponding preset priority meets the upsampling condition, perform upsampling processing; And / or, perform downsampling processing on the sub-images whose corresponding preset priority satisfies the downsampling condition.

[0011] Optionally, the step of performing image partitioning processing on the acquired original rear view image to determine multiple sub-images includes: Based on the acquired vehicle motion state information and the original rear view image, predict the changing trend of the original rear view image; Based on the changing trend, preload the preprocessing pipeline corresponding to the changing trend; In response to the original rear view image corresponding to the changing trend, the original rear view image is preprocessed based on the preprocessing pipeline to determine the optimized rear view image; The optimized post-view image is subjected to image partitioning to determine multiple sub-images.

[0012] Optionally, predicting the trend of change of the original rearview image based on the acquired vehicle motion state information and the original rearview image includes: Based on the vehicle motion state information and the original rear view image within the current first preset period, predict the change trend of the original rear view image in the next second preset period.

[0013] Optionally, the step of performing differentiated image processing on each sub-image based on the allocated processing resources to determine the image to be displayed corresponding to each sub-image includes: Simultaneously, differential image processing is performed on each sub-image to determine the image to be displayed corresponding to each sub-image.

[0014] Optionally, the original rear view image is acquired through the image acquisition mechanism of the vehicle's electronic exterior rearview mirror system; the image acquisition mechanism includes a left image acquisition structure and a right image acquisition structure, and both the left and right image acquisition structures are provided in multiple ways; The step of performing image partitioning processing on the acquired original rear view image to determine multiple sub-images includes: The first original rear view images acquired simultaneously by multiple left-side image acquisition structures are fused together to determine the original rear view image about the left side of the vehicle; the second original rear view images acquired simultaneously by multiple right-side image acquisition structures are fused together to determine the original rear view image about the right side of the vehicle.

[0015] Optionally, the target rear view image is output through the display mechanism of the electronic exterior rearview mirror system, the display mechanism including a left display structure and a right display structure; The step of generating and outputting a complete target rear view image based on all the images to be displayed includes: All images to be displayed corresponding to the original rear view image on the left side of the vehicle are fused together to determine a complete target rear view image on the left side of the vehicle, and then displayed through the left-side display structure; all images to be displayed corresponding to the original rear view image on the right side of the vehicle are fused together to determine a complete target rear view image on the right side of the vehicle, and then displayed through the right-side display structure.

[0016] In a second aspect, the present invention provides an electronic exterior rearview mirror control device, comprising: An image partitioning unit is used to perform image partitioning processing on the acquired original rear view image to determine multiple sub-images; A resource allocation unit is configured to allocate processing resources to each sub-image based on the content features of each sub-image. An image processing unit is configured to perform differentiated image processing on each sub-image based on the allocated processing resources, and determine the image to be displayed corresponding to each sub-image; The image output unit is used to generate and output a complete target rear view image based on all the images to be displayed.

[0017] Thirdly, the present invention provides a vehicle including a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the electronic exterior rearview mirror control method as described in the first aspect when executing the computer program.

[0018] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that is read and executed by a processor to implement the electronic exterior rearview mirror control method as described in the first aspect.

[0019] The beneficial effects of the electronic rearview mirror control method, device, vehicle, and storage medium of the present invention are as follows: The present invention improves the processing efficiency and real-time display of rearview images by partitioning and differentiating the original rearview image, while ensuring the display effect and user viewing experience, thereby enhancing the user's driving experience and driving safety. Specifically, firstly, by partitioning the original rearview image, the entire original rearview image is divided into multiple sub-images, providing a structural basis for subsequent processing; secondly, based on the content characteristics of each sub-image, processing resources are intelligently allocated to each sub-image, achieving differentiated scheduling and efficient utilization of image processing resources, avoiding the waste of computing power and reduced efficiency caused by average processing; then, according to the allocated processing resources, differentiated image processing is performed on each sub-image. For example, higher-precision processing operations can be performed on key sub-images to improve their display quality, while simplified processing is used for non-key sub-images to reduce overall computing power consumption, ensuring that users can efficiently obtain key images. The content contained in the sub-images avoids the drag on the overall processing efficiency of the original rearview image from non-critical sub-images. This ensures both the display effect of the rearview image (i.e., the user can efficiently obtain the content contained in the critical sub-images from the displayed rearview image) and the user's viewing experience, while improving the processing efficiency and real-time output of the rearview image, thus providing effective protection for driving safety. Furthermore, based on all the processed images to be displayed, a complete target rearview image is generated and output, enabling the user to perceive the vehicle's surrounding environment more clearly and accurately, and to obtain key information related to driving safety in a timely manner, thereby improving the readability, recognizability, and safety assistance capabilities of the overall rearview image. Moreover, based on differentiated image processing, the amount of data in the images to be displayed generated after processing non-critical sub-images can be effectively reduced, thereby reducing the burden of image transmission and storage, shortening the time required for image acquisition, processing, and display, and thus improving the operating efficiency and response speed of the invention, and enhancing the real-time output of the rearview image. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating an electronic exterior rearview mirror control method in an embodiment of the present invention; Figure 2 This is a schematic diagram of a sub-process of step 200 in an embodiment of the present invention; Figure 3 This is a schematic diagram of a sub-process of step 220 in an embodiment of the present invention; Figure 4 This is a schematic diagram of a sub-process of step 100 in an embodiment of the present invention; Figure 5 This is a structural block diagram of the electronic exterior rearview mirror control device in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of the vehicle's memory and processor in an embodiment of the present invention. Detailed Implementation

[0021] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.

[0023] Combination Figure 1 As shown, this embodiment of the invention provides an electronic exterior rearview mirror control method, including the following steps: Step 100: Based on the acquired original rear view image, perform image partitioning processing on the original rear view image to determine multiple sub-images.

[0024] The method described in this embodiment can be applied to vehicles equipped with electronic exterior rearview mirror systems to improve the processing efficiency and real-time display of rearview images, thereby enhancing the user's driving experience and driving safety.

[0025] Specifically, in step 100, the image acquisition mechanism (such as a camera) of the vehicle's electronic rearview mirror system, used to acquire rearview (or side-rearview) images, acquires (or captures) images of the vehicle's current rear (or side-rear) field of view (referred to as the original rearview image) in real time. Based on the acquired original rearview image, image partitioning processing is performed, such as dividing the entire original rearview image into multiple images (referred to as sub-images). This facilitates subsequent processing of multiple sub-images, enabling targeted analysis and optimization of different image regions. This provides a structural foundation and data support for subsequent steps in this embodiment, thereby improving the efficiency and response speed of overall image processing while ensuring the processing quality of key image regions. It also avoids unnecessary waste of computing resources or processing delays caused by directly processing the entire original rearview image. In some embodiments, the original rearview image on the left side of the vehicle and the original rearview image on the right side of the vehicle are processed separately and ultimately displayed through different display mechanisms of the electronic rearview mirror system, or through partitions (such as left and right partitions) of the same display mechanism of the electronic rearview mirror system. This avoids mutual interference, ensures the independence of the left and right fields of view, and thus guarantees the user experience.

[0026] Step 200: Allocate processing resources to each sub-image based on its content features.

[0027] Specifically, in step 200, based on all sub-images corresponding to the original rearview image determined in step 100, corresponding image processing resources are allocated to each sub-image according to its content characteristics. For example, more processing resources can be allocated to sub-images involving key content features (such as content features that have an important impact on driving safety). In this way, intelligent scheduling and differentiated use of processing resources for image processing are realized. This avoids the waste of resources and computational power caused by averaging or indiscriminately processing the entire original rearview image when processing resources are limited. As a result, efficient use of processing resources is achieved, the efficiency of rearview image processing and the real-time performance of rearview image output are improved, and effective protection is provided for driving safety.

[0028] The content features include environmental features and moving target features. Environmental features include static or relatively static road and traffic environment elements around the vehicle, such as lane lines, road signs, curbs, sidewalks, traffic lights, and fixed obstacles, which are used to help determine the vehicle's environment and provide navigation reference. Moving target features include information on dynamic objects around the vehicle, such as other vehicles, pedestrians, and other moving targets, which are used to help achieve moving target detection and tracking.

[0029] Step 300: Based on the allocated processing resources, perform differentiated image processing on each sub-image to determine the image to be displayed for each sub-image.

[0030] Specifically, in step 300, based on the resource allocation in step 200, differentiated image processing is performed on each sub-image. For example, according to the specific characteristics of each sub-image (reflected by content features), the allocated processing resources are used to call suitable processing algorithms such as image enhancement, noise reduction, edge detection, and target recognition to process each sub-image accordingly, thereby obtaining (determining) the image to be displayed corresponding to each sub-image.

[0031] For example, for sub-images involving key content features, more processing resources can be allocated to perform higher-precision or more complex image processing operations. This could include employing high-resolution image enhancement algorithms, more refined target recognition models, or more complex image segmentation and target tracking algorithms to improve image clarity and target recognition accuracy in that sub-image region. Conversely, for sub-images not involving key content features, fewer processing resources can be allocated to perform only basic image processing operations, such as simple denoising and brightness adjustment, to reduce the overall computational burden, minimize unnecessary computations, and optimize energy consumption. This not only improves the display quality and perception capability of the original rearview image in key areas but also achieves differentiated allocation and intelligent scheduling of processing resources, ensuring an optimal balance between image processing performance and the real-time output of the rearview image under limited computing power.

[0032] Step 400: Generate and output the complete target rear view image based on all images to be displayed.

[0033] Specifically, in step 400, the images to be displayed corresponding to the sub-images processed in step 300 are fused (e.g., stitched) to generate a complete rear-view image for display (denoted as the target rear-view image), which is then displayed through the display mechanism (e.g., a display screen) of the electronic exterior rearview mirror system. This provides the user with a clear, continuous, real-time, and highly recognizable rear (or side-rear) field of view image, thereby improving the user's driving experience and driving safety. The fusion process can be based on the original positional relationship of each sub-image to ensure natural image edge transitions and no obvious stitching marks, thus maintaining overall image consistency and visual continuity. This ensures that the user is not disturbed by image misalignment, breakage, or other issues when viewing the target rear-view image. It helps improve the readability and credibility of the target rear-view image, enhances the user's intuitive perception of the vehicle's surrounding environment, and further ensures the accuracy and safety of driving operations.

[0034] For example, for sub-images involving key content features, after differential image processing in step 300, a displayable image with high display quality is obtained, which is then displayed in step 400. This makes the sub-image have higher clarity, recognizability and visual prominence in the overall rear view image, helping users to obtain key information closely related to driving safety more quickly and accurately, such as nearby vehicles, pedestrians or road boundaries, thereby improving the practicality and safety assistance capabilities of the overall rear view image.

[0035] In summary, the method in this embodiment improves the processing efficiency and real-time display of rearview images by partitioning and differentiating the original rearview image, while ensuring the display effect and user viewing experience, thus enhancing the user's driving experience and driving safety. Specifically, firstly, by partitioning the original rearview image, the entire original rearview image is divided into multiple sub-images, providing a structural foundation for subsequent processing; secondly, based on the content characteristics of each sub-image, processing resources are intelligently allocated to each sub-image, achieving differentiated scheduling and efficient utilization of image processing resources, avoiding the waste of computing power and reduced efficiency caused by average processing; then, based on the allocated processing resources, differentiated image processing is performed on each sub-image. For example, higher-precision processing operations can be performed on key sub-images to improve their display quality, while simplified processing is used for non-key sub-images to reduce overall computing power consumption, ensuring that users can efficiently obtain key images. The content contained in the sub-images can avoid non-critical sub-images dragging down the overall processing efficiency of the original rearview image. This ensures the display effect of the rearview image (i.e., the user can at least efficiently obtain the content contained in the key sub-images from the displayed rearview image) and the user's viewing experience, while improving the processing efficiency and real-time output of the rearview image, providing effective protection for driving safety. Furthermore, based on all the processed images to be displayed, a complete target rearview image is generated and output, enabling the user to perceive the vehicle's surrounding environment more clearly and accurately, and to obtain key information related to driving safety in a timely manner, thereby improving the readability, recognizability, and safety assistance capabilities of the overall rearview image. Moreover, based on differentiated image processing, the amount of data in the images to be displayed generated after processing non-critical sub-images can be effectively reduced, thereby reducing the burden of image transmission and storage, shortening the time required for image acquisition, processing, and display, and thus improving the operating efficiency and response speed of the method in this embodiment, and enhancing the real-time output of the rearview image.

[0036] Optionally, the original rear view image is partitioned to determine multiple sub-images, including: Based on the preset image partitioning rules, the original rear view image is divided into multiple grid units, and the sub-image corresponding to each grid unit is determined.

[0037] Specifically, according to the pre-set image partitioning rules, the entire original rear view image is divided into several grid units of the same or different sizes. The image in the corresponding area of ​​each grid unit on the original rear view image is determined as a sub-image. That is to say, each sub-image contains the pixel data in the corresponding grid unit, which serves as the basic unit for subsequent differential processing.

[0038] For example, image partitioning rules can be set based on the shape and size of the display screen, the resolution of the original rear view image, driving environment characteristics, or user needs. For instance, if the display screen is rectangular, the original rear view image can be divided into multiple (e.g., 16×9) rectangular grid units of equal area to achieve uniform image partitioning, facilitating unified and standardized processing operations among the sub-images. Furthermore, this uniform partitioning method simplifies boundary calculations during image partitioning, improves processing efficiency, and provides consistent and clearly structured image sub-units for subsequent content feature extraction, processing resource allocation, and differentiated image processing, reducing processing complexity. Alternatively, the image can be non-uniformly partitioned based on the spatial distribution characteristics of key visual areas (e.g., lane line areas, blind spot areas, and areas on both sides of the vehicle's rear). This involves dividing key areas into smaller grid units to improve the accuracy of subsequent processing of critical image areas, while dividing non-critical areas into larger grid units to reduce processing redundancy and computational overhead.

[0039] Optionally, combined Figure 1 , Figure 2 As shown, step 200 includes: Step 210: Perform image recognition on each sub-image to determine the content features of each sub-image; the content features include environmental features and moving target features.

[0040] Specifically, in step 210, image recognition processing is performed on each sub-image to extract its corresponding content features. For example, a corresponding image recognition algorithm (such as a target detection algorithm based on a convolutional neural network) can be used to analyze and extract features from the visual information in the sub-image, identify whether there are corresponding visual environment features (such as road boundaries, lane lines, road signs, obstacles, etc.) and moving target features (such as pedestrians, vehicles, non-motorized vehicles, and other dynamic objects) in the current sub-image, and determine the relevant information of the content features of each sub-image (such as the type of content features and related feature parameters, etc.) based on the recognition results.

[0041] Step 220: Determine the preset priority of each sub-image based on the characteristics of each content, and allocate processing resources corresponding to the preset priority to each sub-image.

[0042] Specifically, in step 220, based on the content features of each sub-image determined in step 210, such as the type of content features and related feature parameters of each sub-image, the processing priority (i.e., preset priority) of each sub-image in subsequent image processing tasks is determined. Then, according to the preset priority of each sub-image, corresponding processing resources are allocated to it in the overall image processing resource pool. For example, sub-images with higher priority will be allocated more processing resources to support the efficient execution of higher precision and complex processing procedures; sub-images with lower priority will be allocated fewer processing resources to ensure the smooth execution of the corresponding processing procedures.

[0043] This approach enables dynamic, flexible scheduling and differentiated allocation of image processing resources, improving overall resource utilization and ensuring the processing quality and response speed of critical image regions (such as sub-images containing moving targets or complex environmental information). This enhances the readability and usability of rear-view images in critical scenarios. Simultaneously, it avoids the problem of non-critical image regions consuming excessive processing resources, leading to reduced overall image processing efficiency or limited processing accuracy in critical regions. Furthermore, it achieves reasonable allocation and optimized scheduling of image processing resources, improving overall image processing efficiency and responsiveness. This ensures that critical driving information is extracted and presented in a timely and accurate manner, thereby enhancing the stability and reliability of the method in this embodiment under complex driving conditions.

[0044] Optionally, for image recognition, the vehicle can be configured with an image change perception module to achieve efficient detection of moving targets in the vehicle's surrounding environment and accurate extraction of environmental features and other content features in the sub-image based on the electronic exterior rearview mirror system.

[0045] For example, for moving target detection, an improved YOLOv8-Tiny model (a target detection model) can be used to detect key targets such as pedestrians and vehicles in real time, with an inference time of less than 5 milliseconds, ensuring rapid response capabilities. Simultaneously, optical flow methods can be combined to calculate pixel-level motion vector fields, achieving motion detection accuracy of ±0.1 pixels, enhancing the accuracy and timeliness of capturing dynamic targets. For environmental feature detection, feature point matching and tracking based on ORB (Oriented FAST and Rotated BRIEF, a feature extraction algorithm combining fast corner detection and rotation-invariant binary descriptors) can achieve continuous monitoring and localization of key regions. Furthermore, background modeling algorithms (such as the ViBe algorithm) can be used to separate dynamic and static regions in the corresponding scene of the image, improving sensitivity and accuracy to environmental changes.

[0046] Optionally, different computing architectures (or processing resources) can be flexibly selected for the image processing process in this embodiment to adapt to diverse application needs and performance requirements. For example, cloud processing resources can be used, based on high-speed networks (such as 5G, V2X communication, etc.), to directly or after corresponding processing, upload the original rearview image acquired by the image acquisition mechanism of the electronic rearview mirror system to the cloud server for high-performance image processing, thereby reducing the demand for and dependence on local hardware resources. Alternatively, local processing resources can be used, such as high-performance SoC chips or image processing units integrated in the vehicle computing platform (vehicle infotainment system), to perform real-time local processing of image data (such as the original rearview image), thereby reducing the latency of image data transmission and processing and improving the real-time performance of the rearview image output. In this case, an edge computing unit with computing power can be deployed at the image acquisition mechanism, allowing some image processing tasks such as image preprocessing, feature extraction, or preliminary recognition to be completed directly at the image acquisition end, thereby effectively reducing the data transmission burden and central processing pressure, and further improving the corresponding response speed and processing efficiency.

[0047] Optionally, during image processing, GPUs or dedicated hardware accelerators (such as NPUs, DSPs, FPGAs, etc.) with graphics processing capabilities can be selected to accelerate image processing operations, based on the complexity and real-time requirements of the image processing task, thereby significantly improving image processing speed and corresponding response efficiency. The aforementioned acceleration hardware can be deployed in in-vehicle computing platforms, edge computing units, or cloud servers, and can be used to perform computationally intensive tasks such as image partitioning, feature extraction, object detection, and image fusion. This can improve the processing accuracy of key image areas, reduce latency, enhance the real-time performance and stability of rear-view image output, and provide users with a smoother and more efficient driving assistance experience.

[0048] Optionally, during image processing, pre-trained lightweight artificial intelligence models (such as models based on MobileNet, YOLOv8-Tiny, EfficientNet-Lite, etc.) can be used to significantly reduce computational complexity and dependence on computing resources while ensuring recognition accuracy, thereby improving image processing efficiency and response speed. These lightweight models can be deployed on in-vehicle computing platforms, edge computing units, or cloud servers, and can be used to perform tasks such as object detection, feature extraction, and dynamic region recognition, balancing performance and real-time capabilities. They are suitable for resource-constrained in-vehicle environments with high processing efficiency requirements, contributing to the efficient and stable operation of electronic rearview mirror systems.

[0049] Optionally, during image processing, preprocessing and caching techniques can be combined to preprocess image data of common scenes or high-frequency areas, and the processing results can be cached in local storage or a high-speed cache. This helps to directly call the cached results during actual operation, avoiding redundant calculations and effectively reducing the real-time computing load. For example, for image areas with stable structures such as lane lines, parking spaces, and obstacle boundaries, feature extraction, recognition model inference, or image enhancement can be pre-completed, and the results can be quickly matched and called when similar scenes appear in the future, significantly improving response speed and image processing efficiency, and further enhancing the stability and practicality of the electronic rearview mirror system in complex road environments.

[0050] Optionally, combined Figure 2 , Figure 3 As shown, step 220 includes: Step 221: Determine the motion weight of each sub-image based on the preset risk coefficient corresponding to each content feature of each sub-image and the acquired vehicle motion state information.

[0051] Specifically, considering that factors affecting driving safety come from both the vehicle's surroundings, such as the features of each sub-image, and the vehicle's own operating state, such as its speed and acceleration, it is necessary to comprehensively consider these factors when determining the preset priority for each sub-image.

[0052] For example, based on the content features of each sub-image determined in step 210, a preset risk coefficient corresponding to different content features of each sub-image is determined. Content features with specific types and specific feature parameters have corresponding preset risk coefficients (denoted as preset risk coefficients). In some embodiments, the preset risk coefficients can be pre-set based on factors such as experience data, accident statistics, scene complexity, or driving safety level. Additionally, the current actual vehicle motion state information (including vehicle speed, acceleration, etc.) is obtained.

[0053] The following example illustrates how to determine the motion weight W for each sub-image based on the acquired vehicle speed V and the preset risk coefficient R corresponding to each content feature of each sub-image. The following weighting model exists: W=α*V+β*(R1+R2+…+Rn), Among them, α is the vehicle speed coefficient, which is used to represent the influence degree of vehicle speed in the overall weight calculation, reflecting the actual requirement that the higher the vehicle speed, the higher the requirement for the timeliness of rear or side-rear image perception; β is the content feature coefficient, which is used to represent the influence degree of the content risk coefficient in the overall weight calculation, reflecting the weight priority consideration of target type, environmental complexity, etc. in the content features during image processing; R1 to Rn are the preset risk coefficients corresponding to the n content features identified in each sub-image. In some embodiments, the parameters α and β can be flexibly adjusted according to vehicle type, processing ability, and user-set strategies. For example, α can be taken as 0.6 and β can be taken as 0.4.

[0054] In this way, through the above weighted model, the current dynamic state of the vehicle can be fused with the static / dynamic elements of the environment for evaluation, so as to dynamically determine the comprehensive processing weight (i.e., motion weight) of each sub-image to support the differential resource allocation strategy.

[0055] Step 222: Determine the preset priority corresponding to the motion weight of the sub-image according to the preset motion weight-preset priority correspondence.

[0056] Specifically, after determining the motion weights corresponding to each sub-image, the processing priority (i.e., preset priority) of each sub-image in the subsequent image processing process can be further determined according to the preset motion weight-preset priority correspondence.

[0057] Among them, the motion weight-preset priority correspondence can be set by using the interval mapping method. The motion weight is divided into several intervals, and each interval corresponds to a priority level; when the motion weight W of a certain sub-image falls into a certain priority interval, the sub-image can be classified into the corresponding priority level. Exemplarily, the following motion weight-preset priority correspondence is set: If W ≥ Th1, the priority is Critical; If Th2 ≤ W < Th1, the priority is High; If Th3 ≤ W < Th2, the priority is Normal; If W < Th3, the priority is Low; Among them, Th1, Th2, and Th3 are preset threshold parameters, which can be preset according to factors such as image processing ability, application scenario requirements for response speed and processing accuracy, or user requirements, or can be adaptively adjusted dynamically through machine learning and other methods.

[0058] Thus, by determining the preset priority of each sub-image based on the correspondence between preset motion weights and preset priorities, the importance of different regions of the image can be automatically identified and dynamically graded during image processing, thereby improving the intelligent processing and adaptive capabilities of the method in this embodiment. Furthermore, by mapping motion weights to multiple priority levels, a structured processing grading system is formed, facilitating the automatic identification and priority processing of key image regions based on factors such as the dynamic complexity and potential risk level of the environment reflected by the sub-images. This enables refined management and rational allocation of processing resources. The introduction of priority levels not only avoids resource waste caused by over-processing of non-critical regions but also ensures the response speed and processing quality of key image regions, improving overall image processing efficiency and reliability. In addition, the priority determination results can serve as input for subsequent image processing strategies, configuring different processing algorithm complexities, recognition accuracy, or resource scheduling strategies for image sub-regions of different priorities. This constructs a closed-loop image processing system oriented towards real-time and security requirements, particularly suitable for multi-task, multi-target concurrent processing scenarios, effectively enhancing the practicality, robustness, and stability of the rear-view imaging system under complex dynamic conditions.

[0059] Step 223: Based on the preset priority-processing resource allocation ratio correspondence, allocate processing resources to each sub-image according to the processing resource allocation ratio corresponding to the preset priority.

[0060] Specifically, after determining the preset priority for each sub-image, image processing resources matching its priority can be allocated to each sub-image according to the preset priority-processing resource allocation ratio, thereby achieving differentiated allocation of image processing resources. The preset priority-processing resource allocation ratio can be set through a configuration table or constructed based on empirical rules, scene data analysis results, and trained machine learning models. Different priority levels correspond to different proportions of processing resources; for example, high-priority sub-images can be allocated more processing resources to support higher precision or higher frequency processing tasks, while low-priority sub-images can be allocated relatively fewer resources to meet basic processing needs.

[0061] For example, the following allocation relationship can be set: sub-images with a priority of "Critical" are allocated the largest proportion of resources (e.g., 45%), sub-images with a priority of "High" are allocated the second highest proportion (e.g., 30%), sub-images with a priority of "Normal" are allocated a medium proportion (e.g., 20%), and sub-images with a priority of "Low" are allocated only a small amount of resources (e.g., 5%). This achieves efficient allocation and utilization of limited processing resources, optimizing image processing performance, improving image processing efficiency, effectively reducing processing latency, and enhancing the real-time performance of rearview image output, thereby improving the user's driving experience and driving safety.

[0062] Optionally, combined Figure 1 , Figure 4 As shown, step 100 includes: Step 110: Based on the acquired vehicle motion state information and the original rear view image, predict the changing trend of the original rear view image.

[0063] Specifically, in step 110, based on the acquired vehicle motion state information (such as vehicle speed, acceleration, steering angle, angular velocity, angular acceleration, etc.), and combined with relevant information from the currently acquired original rear view image (such as the curvature and slope of the lane in the environmental features of the original rear view image), the subsequent change trend of the rear view image is predicted (which is also the prediction of the subsequent change trend of the vehicle's motion state), so as to predict the changes in the rear view field of view. Based on the prediction results, the image processing process is optimized in advance, the real-time computing load is reduced, the image processing efficiency is further improved, and the real-time performance of the rear view image output is improved.

[0064] For example, a time-series analysis-based prediction model (such as a Kalman filter, a recurrent neural network, or a deep learning-based time-series prediction model) can be used to predict the changing trend of the rear view image; this prediction is used to capture the dynamic changes that the rear view image content may undergo as the vehicle moves, such as the trajectory of the target object, changes in ambient lighting, or changes in scene structure.

[0065] Step 120: Based on the changing trend, preload the preprocessing pipeline corresponding to the changing trend.

[0066] Specifically, in step 120, based on the predicted future trend of the original rear view image, a preprocessing pipeline matching this trend is dynamically preloaded. The preprocessing pipeline refers to an ordered combination of a series of image preprocessing steps or modules performed before formal image processing to improve overall image processing efficiency and image quality. It may include image enhancement, denoising, distortion correction, motion compensation, bandwidth reservation, and other processing modules optimized for specific trends. By preloading the preprocessing pipeline, corresponding computing resources and algorithm parameters can be prepared in advance, reducing loading latency in subsequent image processing, thereby improving response speed and the real-time performance of the rear view image output.

[0067] For example, when predicting a vehicle's left turn, the distortion correction look-up table (LUT) of the left image acquisition mechanism of the electronic exterior rearview mirror system is loaded in advance so that when the vehicle actually turns left, the image geometry can be quickly corrected according to the distortion correction look-up table to ensure the image quality and geometric accuracy of the vehicle's left rear view.

[0068] For example, to improve the response speed and efficiency of image processing tasks in different driving scenarios, the weight files required by the corresponding neural network processing unit (NPU) can be classified and managed according to the driving scenario. When a specific scenario (such as a left turn or an emergency lane change) is predicted to occur, the weight file for the corresponding scenario can be preloaded into the high-speed memory cache to enable the rapid retrieval and deployment of model parameters, avoid processing delays caused by temporary loading, and ensure the real-time performance and stability of subsequent image analysis and recognition tasks.

[0069] For example, when a high-risk maneuver such as an emergency lane change is predicted, a bandwidth reservation strategy can be adopted to improve real-time response capabilities and image processing efficiency. For instance, the bandwidth used for image and control data transmission in the vehicle's CAN FD (Controller Area Network Flexible Data Rate) bus can be increased, such as dynamically increasing it from 50% to 80%, to ensure that critical data can be transmitted faster and more stably. At the same time, the priority of the thread responsible for image rendering in the corresponding graphics processing unit (GPU) can be increased to the Real-Time level, so that the image processing task can obtain higher execution priority in task scheduling, thereby achieving fast rendering and stable output of rear view images, improving driving safety and system response speed.

[0070] Step 130: In response to the original rear view image corresponding to the changing trend, preprocess the original rear view image based on the preprocessing pipeline to determine the optimized rear view image.

[0071] Specifically, in step 130, based on the previously predicted trend of change in the original rearview image, when the actual trend matches the prediction, the pre-loaded preprocessing pipeline can be directly invoked to preprocess the currently acquired original rearview image (such as image enhancement, denoising, distortion correction, motion compensation, etc.), thereby significantly improving the speed and efficiency of image processing, reducing computational latency, and ensuring the real-time and continuous nature of image processing. The preprocessed image (denoted as the optimized rearview image) has higher clarity and accuracy, and can more effectively reflect the actual situation of the vehicle's surrounding environment, providing a reliable foundation for subsequent image partitioning and target recognition. By preprocessing the original rearview image based on the preprocessing pipeline, this mechanism effectively reduces the instantaneous demand on computing resources, improves resource response speed and resource utilization, especially by fully utilizing idle resources during preloading, which helps improve overall image processing performance and energy consumption, further ensuring driving safety and user experience.

[0072] Step 140: Perform image partitioning on the optimized post-view image to determine multiple sub-images.

[0073] Specifically, in step 140, the optimized rear view image is divided into multiple sub-images, which facilitates the implementation of differentiated processing strategies for different regions based on their image features and importance, thus laying the foundation for improving the efficiency of rear view image processing and the real-time performance of display.

[0074] Optionally, step 110 includes: Based on the vehicle motion status information and the original rear view image within the current first preset cycle, predict the change trend of the original rear view image in the next second preset cycle.

[0075] To ensure the accuracy and reliability of trend prediction, predictions can be made based on the vehicle's historical motion state information and historical rearview images. Specifically, based on some or all of the vehicle motion state information and original rearview images acquired within the current corresponding period (denoted as the first preset period), or based on the vehicle motion state information and original rearview images within the previous time period (i.e., the first preset period), time series analysis methods or deep learning models, such as Long Short-Term Memory Networks (LSTM), Recurrent Neural Networks (RNN), or other deep learning models, are used to comprehensively model and analyze the trends in vehicle driving status and rear-view field of view changes, so as to achieve accurate prediction of the trend of original rearview image changes in the next preset period (denoted as the second preset period). The durations of the first and second preset periods can be equal or different, and can be set or dynamically adjusted according to design requirements, such as adjusting and setting them according to the dynamic characteristics of the actual application scenario, to balance the real-time nature and accuracy of the prediction.

[0076] For example, a trend prediction model based on a Long Short-Term Memory (LSTM) network can be used. The input dimension is set as a sequence of original rear-view images with a time series window length of 20 frames within a first preset period (which can reflect the dynamic changes of the rear-view images within a duration of approximately 333ms). By modeling the temporal and spatial change features between images in this sequence, the potential patterns of the evolution of the rear-view images over time during vehicle movement can be extracted to achieve trend prediction. Specifically, a neural network structure consisting of two layers of LSTM units (each layer containing 128 hidden nodes) can be constructed to deeply mine the dynamic change features in the image sequence. A fully connected layer is then added to integrate and transform the extracted feature vectors. Finally, the model can output a probability distribution map of the hotspots of visual field changes within a future second preset period (e.g., 100ms). This prediction result reflects the location areas in the original rear-view images most likely to undergo significant changes under the current driving environment. This provides effective support for subsequent preprocessing pipeline scheduling and resource preloading strategies, thereby improving image processing efficiency and resource utilization efficiency.

[0077] Optionally, step 300 includes: Simultaneously, differentiated image processing is performed on each sub-image to determine the image to be displayed for each sub-image.

[0078] Specifically, to improve the overall processing speed and efficiency of the original rearview image, it is preferable to adopt a parallel processing mechanism based on the processing resources allocated to each sub-image, and to perform image processing on all sub-images simultaneously (or concurrently) to achieve differentiated image processing on all sub-images, thereby determining (obtaining) the image to be displayed for each sub-image. In this way, the total time required to obtain each image to be displayed is significantly reduced, and the image processing efficiency is significantly improved, thereby improving the real-time performance of the rearview image output and providing effective protection for driving safety.

[0079] For example, a multi-core processor can be used to process all sub-images in parallel. This could be based on a multi-core heterogeneous processor (e.g., a system-on-a-chip (SoC) integrating multiple CPU cores, GPUs, and NPUs) provided by the vehicle to provide processing resources. Through task allocation and scheduling mechanisms, all sub-images to be processed are divided and concurrently distributed to multiple computing cores of the processor. Each core then executes the corresponding image processing task (including image decoding, object detection, image enhancement, etc.) for that sub-image, achieving independent and parallel processing of each sub-image. Specifically, basic image processing operations (such as image decoding, denoising, and color adjustment) can be allocated to CPU cores; high-load tasks involving neural network inference (such as the recognition and detection of vehicles, pedestrians, or obstacles in sub-images) are preferably allocated to the NPU cluster; and graphics processing tasks such as image rendering and display adaptation can be allocated to GPU cores. Especially when the GPU supports graphics rendering standards such as OpenGL ES3.2, the smoothness and visual quality of the processed image display can be effectively improved. Thus, by adopting a parallel processing architecture, the overall throughput of image processing can be significantly improved, enabling efficient support for implementing differentiated processing strategies for each sub-image, thereby enhancing the targeting and intelligence of image processing.

[0080] Optionally, after step 222, step 220 further includes: Upsampling is performed on the sub-images that meet the upsampling conditions according to the corresponding preset priority. And / or, perform downsampling processing on the sub-images that meet the downsampling conditions according to the corresponding preset priority.

[0081] Specifically, based on the preset priority corresponding to each sub-image, it is determined whether the sub-image meets the upsampling condition. If it does, upsampling processing is performed on the sub-image, such as using bilinear interpolation to increase the resolution of the sub-image to a certain multiple (e.g., 4 times) of the original resolution, so as to enhance the detail and image clarity of the region.

[0082] And / or, based on the preset priority corresponding to each sub-image, determine whether the sub-image meets the downsampling condition. If it does, perform downsampling processing on the sub-image, such as reducing the resolution of the sub-image to a certain proportion of the original resolution (e.g., 1 / 4), to reduce the image detail level in that area, reduce the computational load and storage requirements, thereby optimizing the allocation of processing resources, improving the overall image processing efficiency, and ensuring that the visual effect of non-critical areas and the accuracy of subsequent analysis are not significantly affected, thus guaranteeing the display effect of the rear view image and the user's viewing experience.

[0083] For example, the upsampling condition is set to a preset priority of at least high, and the downsampling condition is set to a preset priority of at most medium; or, the upsampling condition is set to a preset priority of important, and the downsampling condition is set to a preset priority of low; and so on. The upsampling and downsampling conditions can be set according to actual needs, or flexibly adjusted according to the dynamic changes in vehicle resources and driving environment, so as to achieve optimal allocation of image processing resources.

[0084] Optionally, during image processing, DMA (Direct Memory Access) technology can be used to achieve zero-copy data transfer. This means that when image data is read from the sensor or memory into the processing unit, it is directly transferred, bypassing the CPU. This avoids traditional data copying operations, significantly reducing processing latency and CPU load, and improving data transfer efficiency and response speed. Furthermore, DMA enables seamless data switching and efficient collaboration during multi-task parallel processing, further enhancing overall image processing performance and resource utilization, and ensuring the real-time performance and stability of post-processing image processing.

[0085] Optionally, to achieve low-latency transmission of corresponding data during image processing, image data can be transmitted using efficient transmission protocols (such as CAN FD or Ethernet) to ensure the stability and real-time performance of data transmission. For example, an improved version of the AVB (Audio Video Bridging) protocol stack can be used, with an end-to-end transmission latency of less than 8 milliseconds. Combined with CRC (Cyclic Redundancy Check) and selective retransmission mechanisms, the bit error rate can be effectively controlled to be below [a certain threshold]. This ensures data integrity and reliability.

[0086] Optionally, to achieve low-latency display of the target rear-view image, a low-latency display control algorithm can be used to ensure rapid image display. Simultaneously, for the display stage, a Panel self-refresh mode (MIPI DSI Command Mode) can be employed, combined with an Overdrive voltage pre-compensation algorithm, to improve display response speed and image quality, further reducing display latency and achieving fast, clear image display, thereby meeting the stringent requirements of real-time performance and stability for vehicle rear-view images.

[0087] Optionally, the original rear view image is acquired through the image acquisition mechanism of the vehicle's electronic exterior rearview mirror system; the image acquisition mechanism includes a left image acquisition structure and a right image acquisition structure, and both the left and right image acquisition structures are provided with multiple structures.

[0088] Specifically, the original rear-view image is acquired through the image acquisition mechanism of the vehicle's electronic exterior rearview mirror system. To improve the user's field of view coverage on the left and right rear sides of the vehicle, and to enhance the stability and redundancy of the acquired field of view, the image acquisition mechanism is equipped with multiple image acquisition structures respectively located on the left and right sides of the vehicle. These multiple image acquisition structures can work collaboratively at different installation angles and positions, covering a wider monitoring area, reducing blind spots, and ensuring that even if some image acquisition structures are interfered with, obstructed, or damaged, the remaining structures can still provide redundant images, guaranteeing the integrity and continuity of critical image information and avoiding single-point failures.

[0089] Step 100 includes: The first original rear view image acquired simultaneously by multiple left-side image acquisition structures is fused to determine the original rear view image about the left side of the vehicle; the second original rear view image acquired simultaneously by multiple right-side image acquisition structures is fused to determine the original rear view image about the right side of the vehicle.

[0090] Specifically, to effectively utilize the original rear-view images acquired by multiple left-side and right-side image acquisition structures, the first original rear-view images simultaneously acquired by multiple left-side image acquisition structures are fused to obtain a fused rear-view image of the left side of the vehicle; similarly, the second original rear-view images simultaneously acquired by multiple right-side image acquisition structures are fused to obtain a fused rear-view image of the right side of the vehicle. The image fusion process may include at least one of image registration, feature matching, distortion correction, and weighted superposition to effectively improve the overall quality of both the fused original rear-view image and the final target rear-view image, enhancing image clarity and consistency, while expanding the field of view coverage to the side and rear of the vehicle and reducing blind spots. Furthermore, even if the image quality of a particular image acquisition structure deteriorates or malfunctions, the fusion process can still reconstruct the image based on the remaining image data, thereby improving the robustness and fault tolerance of the vehicle's electronic rearview mirror system in complex road scenarios or extreme conditions, ensuring that the final output target rear-view image has higher reliability and real-time performance.

[0091] It is worth noting that the original rear view images of the left side of the vehicle and the original rear view images of the right side of the vehicle can be processed independently and ultimately displayed through different display mechanisms of the electronic exterior rearview mirror system, or through the same display mechanism partition (such as left and right partitions) of the electronic exterior rearview mirror system, to avoid mutual interference, ensure the independence of the left and right field of vision, and thus guarantee the user experience.

[0092] For example, both the left and right image acquisition structures employ high-resolution cameras to acquire images of the vehicle's surrounding environment in real time. The camera modules in both structures may use global shutter CMOS sensors with a resolution of at least 8 megapixels, support for high dynamic range (HDR) technology, a dynamic range greater than 120dB, and a frame rate of at least 60 frames per second.

[0093] The layout design of multiple left-side and multiple right-side image acquisition structures is illustrated below using an example where both left and right sides of the vehicle are equipped with wide-angle cameras and rear cameras. The wide-angle camera has a field of view (FOV) of no less than 120 degrees to cover the vehicle's blind spots; the rear camera has night vision enhancement capabilities to ensure image quality in nighttime and low-light environments. Furthermore, the multiple cameras can achieve time synchronization via hardware trigger signals, with a time error of less than 1 millisecond, ensuring the synchronicity and consistency of image acquisition.

[0094] Optionally, to improve the display effect of the target rearview image, the display mechanism of the electronic exterior rearview mirror system can use a high-performance display screen, such as a high-resolution display screen, to display the processed target rearview image in real time. It can also be combined with Mini-LED backlight local dimming technology to achieve higher brightness and contrast, such as a peak brightness of up to 1000 nits, ensuring good visibility of the display screen even in bright light environments. Furthermore, to reduce visual fatigue caused by image flicker, the display mechanism can also employ a PWM frequency modulation anti-flicker algorithm, such as a drive frequency greater than 2000Hz, thereby effectively suppressing flicker.

[0095] For example, this display solution possesses the following performance indicators: end-to-end image processing and display latency of less than 50ms (compliant with ISO 16505 standard), target detection accuracy (mAP) higher than 92%, overall power consumption less than 8W, and stable operation in a wide temperature range of -40℃ to 85℃, meeting the environmental adaptability requirements of the automotive industry. Furthermore, to further enhance the stability and smoothness of image display, the display mechanism can also employ a dynamic partitioning elastic resource scheduling mechanism, combined with intelligent prediction, to maintain a stable high frame rate even under severe driving conditions such as rapid acceleration (e.g., acceleration greater than 0.5g), ensuring the driver receives a continuous, smooth, and high-quality image display experience.

[0096] Optionally, the target rear view image is output through the display mechanism of the electronic exterior rearview mirror system, the display mechanism including a left display structure and a right display structure; Based on all images to be displayed, generate and output a complete target rear view image, including; All images to be displayed corresponding to the original rear view image of the left side of the vehicle are fused together to determine the complete target rear view image of the left side of the vehicle, and then displayed through the left-side display structure; all images to be displayed corresponding to the original rear view image of the right side of the vehicle are fused together to determine the complete target rear view image of the right side of the vehicle, and then displayed through the right-side display structure.

[0097] Specifically, to ensure the independence of left and right views and guarantee user experience, the display mechanism of the electronic exterior rearview mirror system for displaying (or outputting) target rearview images has a left-side display structure and a right-side display structure. The left-side display structure displays the target rearview image from the left side of the vehicle, and the right-side display structure displays the target rearview image from the right side of the vehicle. By displaying the left and right target images in corresponding display structures, users can form an intuitive left-right correspondence when observing, which helps improve judgment efficiency and driving safety. For example, the left-side and right-side display structures can be arranged in ergonomic positions such as near the A-pillars on the left and right sides of the driver or on the extension of the dashboard, and use high-resolution, low-latency displays. With image stabilization processing and adaptive brightness adjustment functions, the target rearview image can be clearly presented under different ambient lighting conditions, thereby enhancing the user's perception of the surrounding environment.

[0098] The following explanation of the image fusion process uses the fusion of all images to be displayed corresponding to the original rear-view image on the left side of the vehicle as an example. The images to be displayed corresponding to each sub-image of the original rear-view image on the left side of the vehicle are fused (e.g., stitched together) to generate a complete rear-view image for display (denoted as the target rear-view image). This image is then displayed through the left-side display structure, providing users with a clear, coherent, real-time, and highly recognizable rear (or side-rear) view, thereby improving the user's driving experience and safety. The fusion process can be based on the original positional relationships of each sub-image, ensuring natural edge transitions and no obvious stitching marks, thus maintaining overall image consistency and visual coherence. This ensures that users are not disturbed by image misalignment or breaks when viewing the target rear-view image; it also helps improve the readability and credibility of the target rear-view image, enhances the user's intuitive perception of the vehicle's surroundings, and further ensures the accuracy and safety of driving operations.

[0099] Combination Figure 5 As shown, another embodiment of the present invention provides an electronic exterior rearview mirror control device, comprising: The image partitioning unit is used to perform image partitioning processing on the acquired original rear view image to determine multiple sub-images; The resource allocation unit is used to allocate processing resources to each sub-image based on the content characteristics of each sub-image; The image processing unit is used to perform differentiated image processing on each sub-image based on the allocated processing resources, and determine the image to be displayed corresponding to each sub-image; The image output unit is used to generate and output a complete target rear view image based on all images to be displayed.

[0100] The electronic exterior rearview mirror control device of this embodiment is used to implement the above-described electronic exterior rearview mirror control method. Its advantages over the prior art are the same as the advantages of the above-described electronic exterior rearview mirror control method over the prior art, and will not be repeated here.

[0101] Optionally, the image partitioning unit is specifically used to: divide the original rear view image into multiple grid units based on a preset image partitioning rule, and determine the sub-image corresponding to each grid unit.

[0102] Optionally, the resource allocation unit is specifically used to: perform image recognition on each sub-image to determine the content features of each sub-image; wherein, the content features include environmental features and moving target features; Based on the characteristics of each content, a preset priority is determined for each sub-image, and processing resources corresponding to the preset priority are allocated to each sub-image.

[0103] Optionally, the resource allocation unit is specifically used to: determine the motion weight corresponding to each sub-image based on the preset risk coefficient corresponding to each content feature of each sub-image and the acquired vehicle motion state information; Based on the preset motion weight-preset priority correspondence, determine the preset priority corresponding to the motion weight of the sub-image; Based on the preset priority-processing resource allocation ratio correspondence, each sub-image is allocated processing resources according to the processing resource allocation ratio corresponding to the preset priority.

[0104] Optionally, the image partitioning unit is specifically used to: predict the changing trend of the original rear view image based on the acquired vehicle motion state information and the original rear view image; Based on the changing trend, preload the preprocessing pipeline corresponding to the changing trend; In response to the original rear view image corresponding to the changing trend, the original rear view image is preprocessed based on the preprocessing pipeline to determine the optimized rear view image; The optimized post-view image is partitioned to determine multiple sub-images.

[0105] Optionally, the resource allocation unit is specifically used to: predict the change trend of the original rear view image in the next second preset period based on the vehicle motion status information and the original rear view image in the current first preset period.

[0106] Optionally, the image processing unit is specifically used to: simultaneously perform differential image processing on all sub-images to determine the image to be displayed corresponding to each sub-image.

[0107] Optionally, after determining the preset priority corresponding to the motion weight of the sub-image based on the preset motion weight-preset priority correspondence, the resource allocation unit is further used for: Upsampling is performed on the sub-images that meet the upsampling conditions according to the corresponding preset priority. And / or, perform downsampling processing on the sub-images that meet the downsampling conditions according to the corresponding preset priority.

[0108] Optionally, the image is acquired based on the original rearview image through the image acquisition mechanism of the vehicle's electronic exterior rearview mirror system; the image acquisition mechanism includes a left image acquisition structure and a right image acquisition structure, and both the left and right image acquisition structures are provided with multiple units; the image partitioning unit is specifically used for: The first original rear view image acquired simultaneously by multiple left-side image acquisition structures is fused to determine the original rear view image about the left side of the vehicle; the second original rear view image acquired simultaneously by multiple right-side image acquisition structures is fused to determine the original rear view image about the right side of the vehicle. Optionally, the target rearview image is output through the display mechanism of the electronic exterior rearview mirror system, the display mechanism including a left-side display structure and a right-side display structure; the image output unit is specifically used for: All images to be displayed corresponding to the original rear view image of the left side of the vehicle are fused together to determine the complete target rear view image of the left side of the vehicle, and then displayed through the left-side display structure; all images to be displayed corresponding to the original rear view image of the right side of the vehicle are fused together to determine the complete target rear view image of the right side of the vehicle, and then displayed through the right-side display structure.

[0109] Combination Figure 6 As shown, another embodiment of the present invention provides a vehicle, including a memory 601 and a processor 602; Memory 601 is used to store computer programs; Processor 602 is used to implement the above-described electronic exterior rearview mirror control method when executing a computer program.

[0110] Alternatively, a vehicle includes a memory 601 and a processor 602 coupled to the memory 601; the memory 601 is configured to store a computer program; the processor 602 is configured to perform the following operations when the computer program is executed: Based on the acquired original rear view image, perform image partitioning processing on the original rear view image to determine multiple sub-images; Allocate processing resources to each sub-image based on its content features; Based on the allocated processing resources, differentiated image processing is performed on each sub-image to determine the image to be displayed for each sub-image; Generate and output a complete target rear view image based on all images to be displayed.

[0111] The vehicle in this embodiment can be used to implement the above-described electronic exterior rearview mirror control method. Its advantages over the prior art are the same as those of the above-described electronic exterior rearview mirror control method over the prior art, and will not be repeated here.

[0112] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program, which is read and executed by a processor to implement the above-described electronic rearview mirror control method.

[0113] Alternatively, a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the following operations: Based on the acquired original rear view image, perform image partitioning processing on the original rear view image to determine multiple sub-images; Allocate processing resources to each sub-image based on its content features; Based on the allocated processing resources, differentiated image processing is performed on each sub-image to determine the image to be displayed for each sub-image; Generate and output a complete target rear view image based on all images to be displayed.

[0114] The technical solutions of the embodiments of the present invention, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of the embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0115] The computer-readable storage medium of this embodiment can be used to implement the above-described electronic exterior rearview mirror control method. Its advantages over the prior art are the same as those of the above-described electronic exterior rearview mirror control method over the prior art, and will not be repeated here.

[0116] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A method for controlling an electronic exterior rearview mirror, characterized in that, include: Based on the acquired original rear view image, the original rear view image is subjected to image partitioning processing to determine multiple sub-images; Based on the content features of each sub-image, allocate processing resources to each sub-image; Based on the allocated processing resources, differentiated image processing is performed on each sub-image to determine the image to be displayed corresponding to each sub-image; Based on all the images to be displayed, generate and output a complete target rear view image.

2. The electronic exterior rearview mirror control method as described in claim 1, characterized in that, The step of performing image partitioning processing on the original rear view image to determine multiple sub-images includes: Based on a preset image partitioning rule, the original rear view image is divided into multiple grid units, and the sub-image corresponding to each grid unit is determined.

3. The electronic exterior rearview mirror control method as described in claim 1, characterized in that, The step of allocating processing resources to each sub-image based on the content features of each sub-image includes: Image recognition is performed on each of the sub-images to determine the content features of each sub-image; wherein, the content features include environmental features and moving target features; Based on the content characteristics, a preset priority is determined for each sub-image, and processing resources corresponding to the preset priority are allocated to each sub-image.

4. The electronic exterior rearview mirror control method as described in claim 3, characterized in that, The step of determining a preset priority for each sub-image based on its content features, and allocating processing resources corresponding to the preset priority for each sub-image, includes: Based on the preset risk coefficients corresponding to the content features of each of the sub-images and the acquired vehicle motion state information, determine the motion weights corresponding to each of the sub-images. Based on the preset motion weight-preset priority correspondence, the preset priority corresponding to the motion weight of the sub-image is determined; According to the preset priority-processing resource allocation ratio correspondence, each sub-image is allocated processing resources corresponding to the processing resource allocation ratio of the preset priority.

5. The electronic exterior rearview mirror control method as described in claim 4, characterized in that, After determining the preset priority corresponding to the motion weight of the sub-image, the step of determining the preset priority corresponding to each sub-image based on each content feature and allocating the processing resources corresponding to the preset priority to each sub-image further includes: For the sub-images whose corresponding preset priority meets the upsampling condition, perform upsampling processing; And / or, perform downsampling processing on the sub-images whose corresponding preset priority satisfies the downsampling condition.

6. The electronic exterior rearview mirror control method as described in any one of claims 1-5, characterized in that, The step of performing image partitioning processing on the acquired original rear view image to determine multiple sub-images includes: Based on the acquired vehicle motion state information and the original rear view image, predict the changing trend of the original rear view image; Based on the changing trend, preload the preprocessing pipeline corresponding to the changing trend; In response to the original rear view image corresponding to the changing trend, the original rear view image is preprocessed based on the preprocessing pipeline to determine the optimized rear view image; The optimized post-view image is subjected to image partitioning processing to determine multiple sub-images.

7. The electronic exterior rearview mirror control method as described in claim 6, characterized in that, The step of predicting the changing trend of the original rearview image based on the acquired vehicle motion state information and the original rearview image includes: Based on the vehicle motion state information and the original rear view image within the current first preset period, predict the change trend of the original rear view image in the next second preset period.

8. The electronic exterior rearview mirror control method as described in any one of claims 1-5, characterized in that, Based on the allocated processing resources, differential image processing is performed on each sub-image to determine the image to be displayed corresponding to each sub-image, including: Simultaneously, differential image processing is performed on each sub-image to determine the image to be displayed corresponding to each sub-image.

9. The electronic exterior rearview mirror control method as described in any one of claims 1-5, characterized in that, The original rear view image is acquired through the image acquisition mechanism of the vehicle's electronic exterior rearview mirror system; the image acquisition mechanism includes a left image acquisition structure and a right image acquisition structure, and both the left and right image acquisition structures are provided in multiple ways; The step of performing image partitioning processing on the acquired original rear view image to determine multiple sub-images includes: The first original rear view images acquired simultaneously by multiple left-side image acquisition structures are fused to determine the original rear view image about the left side of the vehicle. The second original rear view images acquired simultaneously by multiple right-side image acquisition structures are fused to determine the original rear view image about the right side of the vehicle.

10. The electronic exterior rearview mirror control method as described in claim 9, characterized in that, The target rear view image is output through the display mechanism of the electronic exterior rearview mirror system, the display mechanism including a left display structure and a right display structure; The step of generating and outputting a complete target rear view image based on all the images to be displayed includes: All images to be displayed corresponding to the original rear view image on the left side of the vehicle are fused together to determine a complete target rear view image on the left side of the vehicle, and then displayed through the left-side display structure; all images to be displayed corresponding to the original rear view image on the right side of the vehicle are fused together to determine a complete target rear view image on the right side of the vehicle, and then displayed through the right-side display structure.

11. An electronic exterior rearview mirror control device, characterized in that, include: An image partitioning unit is used to perform image partitioning processing on the acquired original rear view image to determine multiple sub-images; A resource allocation unit is configured to allocate processing resources to each sub-image based on the content features of each sub-image. An image processing unit is configured to perform differentiated image processing on each sub-image based on the allocated processing resources, and determine the image to be displayed corresponding to each sub-image; The image output unit is used to generate and output a complete target rear view image based on all the images to be displayed.

12. A vehicle, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to implement the electronic exterior rearview mirror control method as described in any one of claims 1-10 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is read and executed by a processor to implement the electronic exterior rearview mirror control method as described in any one of claims 1-10.

Images