Methods, devices, terminal equipment, and storage media for processing image lag in electronic rearview mirrors

Abstract

This application is applicable to the technical field of intelligent terminals, and provides an electronic rearview mirror image stuttering processing method, device, terminal device and storage medium. The method includes: during the video stream acquisition process, marking the image data of a preset pixel region of each frame of image to generate a marked video frame containing marked blocks, where the marked blocks change with the frame; during the video stream display process, extracting the display data of the marked blocks from the marked video frame; based on the display data corresponding to the marked blocks of multiple consecutive marked video frames, determining whether the video stream has stuttered; if stuttering occurs, performing a fault recovery operation. This application can achieve precise detection and rapid recovery of image stuttering, improve the reliability of the electronic rearview mirror system, and thus provide a strong guarantee for driving safety.

Description

Technical Field

[0001] This application relates to the field of smart terminal technology, and in particular to a method, apparatus, terminal device and storage medium for processing image lag in electronic rearview mirrors. Background Technology

[0002] Electronic rearview mirrors use cameras to capture images of the vehicle's surroundings and transmit the image data to a display device for real-time display via a video transmission link. This effectively overcomes the field-of-view limitations of traditional optical rearview mirrors, providing drivers with more comprehensive information about the driving environment. The reliability of electronic rearview mirrors is directly related to driving safety, requiring stable image data throughout the entire process from acquisition and transmission to display, ensuring accurate and real-time feedback of changes in the surrounding environment.

[0003] In practical applications, issues such as camera hardware malfunction, transmission link signal interruption, or software deadlock can all cause the display device to fail to update image data, thus continuously displaying the last frame of the image stored in memory, resulting in image lag. This phenomenon can easily lead drivers to misjudge environmental conditions, making it difficult to detect obstacles or pedestrians in time, and causing safety hazards.

[0004] Therefore, how to achieve accurate detection and rapid recovery of image lag and improve the reliability of electronic rearview mirror systems is a problem that needs to be considered. Summary of the Invention

[0005] This application provides an electronic rearview mirror image lag processing method, device, terminal equipment, and storage medium, which can achieve accurate detection and rapid recovery of image lag, improve the reliability of the electronic rearview mirror system, and thus provide strong protection for driving safety.

[0006] In a first aspect, embodiments of this application provide a method for processing image lag in an electronic rearview mirror, including:

[0007] During video stream acquisition, image data of a preset pixel region in each frame is marked to generate a marked video frame containing marked blocks, wherein the marked blocks change with the frame. During the display of the video stream, the display data of the marked blocks is extracted from the marked video frames; Based on the display data corresponding to the marker blocks of multiple consecutive marked video frames, it is determined whether the video stream is experiencing stuttering; If a jam occurs, perform a fault recovery operation.

[0008] In one possible implementation of the first aspect, marking the image data of a preset pixel region in each frame to generate a marked video frame containing the marked blocks includes: Obtain ambient light intensity; Based on the ambient light intensity, the modification bit depth of the chromaticity components of the preset pixel region in each frame of the image is dynamically determined; Based on the frame count and the number of bits modified during the video stream acquisition process, the image data in the preset pixel region is modified to generate the marker block.

[0009] In one possible implementation of the first aspect, the chromaticity components include a first chromaticity component and a second chromaticity component; the step of dynamically determining the modification bit depth of the chromaticity components of the preset pixel region in each frame image based on the ambient light intensity includes: If the ambient light intensity is less than the first threshold, the modification bit of the first chromaticity component is determined to be one bit, and the second chromaticity component is not modified. If the ambient light intensity is greater than or equal to the first threshold and less than or equal to the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be one bit. If the ambient light intensity is greater than the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be two bits.

[0010] In one possible implementation of the first aspect, determining whether the video stream is experiencing stuttering based on the display data corresponding to the marker blocks of a consecutive plurality of marked video frames includes: Calculate the check value based on the extracted display data of the marked blocks; If the checksums of multiple consecutive marked video frames are the same, it is determined that the marked blocks have not changed, and the video stream display is stuck.

[0011] In one possible implementation of the first aspect, the fault recovery operation includes: Perform a first-level recovery operation, which includes resetting the image acquisition device and the display device.

[0012] In one possible implementation of the first aspect, after performing the first-level recovery operation, the method further includes: Re-verify the changes in the marked blocks; If it is determined that the lag has not been eliminated, a second-level recovery operation is performed, which includes restarting the image processing unit.

[0013] In one possible implementation of the first aspect, the method further includes: Obtain the vehicle's real-time speed and the actual frame rate of the video stream; Based on the actual frame rate and the real-time speed of the vehicle, dynamically calculate the number of stuck frames to determine the threshold; When the marker blocks corresponding to multiple consecutive marked video frames remain unchanged, and the number of consecutive marked video frames that remain unchanged reaches the lag frame count threshold, it is determined that the video stream display is lag-prone.

[0014] Secondly, embodiments of this application provide an electronic rearview mirror image lag processing device, including: An image labeling unit is used to label the image data of a preset pixel region of each frame during the video stream acquisition process, and generate a labeled video frame containing a label block, wherein the label block changes with the frame. A marker block extraction unit is used to extract display data of the marker blocks from the marked video frames during the display of the video stream; The stuttering detection unit is used to determine whether the video stream is stuttering based on the display data corresponding to the marker blocks of a series of consecutive marked video frames; The fault recovery unit is used to perform fault recovery operations if a jam occurs.

[0015] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the electronic rearview mirror image lag processing method as described in the first aspect above.

[0016] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the electronic rearview mirror image lag processing method as described in the first aspect above.

[0017] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the electronic rearview mirror image lag processing method described in the first aspect above.

[0018] In this embodiment, image data of a preset pixel region in each frame is marked during video stream acquisition to generate a video frame containing marked blocks. These marked blocks change with each frame, providing a dedicated verification basis for subsequent image lag detection. Targeted detection can be achieved without relying on full-image pixel analysis, effectively simplifying the detection logic. Extracting the display data of the marked blocks from the marked video frames during video stream display allows for precise focusing on key verification objects, avoiding interference from irrelevant image regions and ensuring the validity of the detection data. Based on the display data extracted from multiple consecutive marked video frames, it is determined whether the video stream has experienced image lag. Continuous temporal verification effectively avoids misjudgments caused by momentary interference, ensuring the accuracy of lag determination. Upon determining that lag has occurred, timely fault recovery operations are performed, enabling rapid response to fault scenarios and preventing drivers from misjudging environmental conditions due to image lag, thus minimizing safety hazards. This solution achieves accurate detection and rapid recovery of image lag, improving the reliability of electronic rearview mirror image lag processing and providing strong protection for driving safety. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the implementation of the electronic rearview mirror image lag processing method provided in this application embodiment; Figure 2 This is a flowchart illustrating a specific implementation of step S101 in the electronic rearview mirror image lag processing method provided in this application embodiment; Figure 3 This is a flowchart illustrating a specific implementation of the electronic rearview mirror image lag processing method provided in this application embodiment, which determines the number of bits to be modified. Figure 4 This is a flowchart illustrating a specific implementation of step S103 in the electronic rearview mirror image lag processing method provided in this application embodiment; Figure 5 This is another specific implementation flowchart of the electronic rearview mirror image lag processing method provided in the embodiments of this application for detecting lag; Figure 6.1 This is a flowchart illustrating a specific implementation of the fault recovery operation in the electronic rearview mirror image lag processing method provided in this application embodiment; Figure 6.2This is a flowchart illustrating a specific implementation of the fault recovery operation in the electronic rearview mirror image lag processing method provided in this application embodiment; Figure 7 This is a structural block diagram of the electronic rearview mirror image lag processing device provided in the embodiments of this application; Figure 8 This is a schematic diagram of the terminal device provided in the embodiments of this application. Detailed Implementation

[0021] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0022] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0023] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0024] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0025] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0026] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0027] By way of example and not limitation, the electronic rearview mirror image lag processing method provided in this application is applicable to various types of terminal devices that need to perform electronic rearview mirror image lag processing. Specific terminal devices may include mobile phones, tablets, wearable devices, laptops, ultra-mobile personal computers (UMPCs), desktop computers, and servers, etc. This application does not impose any limitations on the specific type of terminal device.

[0028] Figure 1 The implementation flow of the electronic rearview mirror image lag processing method provided in this application embodiment is illustrated. The method flow includes steps S101 to S105. The specific implementation principle of each step is as follows: Step S101: During the video stream acquisition process, the image data of a preset pixel area of ​​each frame image is marked to generate a marked video frame containing a marked block, wherein the marked block changes with the frame.

[0029] The video stream acquisition process refers to the process by which an image acquisition device captures images of the vehicle's surrounding environment and generates continuous video frames. The image acquisition device includes an in-vehicle camera, mounted on the vehicle, used to acquire images of the vehicle's external environment. Its acquisition range can cover areas in front of, to the sides of, and behind the vehicle, and the acquired image data format is adapted to the image processing requirements of the in-vehicle system.

[0030] A preset pixel region refers to a fixed-size area pre-defined at the edge of each frame of the image (such as the lower left, lower right, upper left, and upper right corners). The size of this preset pixel region can be set to an 8×8 pixel range. Selecting the edge region as the preset pixel region avoids interference with the driver's core field of vision. Its core attribute is positional definition, not specific pixel data. The image data of the preset pixel region refers to the raw pixel data covered by the preset pixel region, that is, the complete YUV data (including the Y component of luminance, U component of chrominance, and V component) acquired by the image acquisition device without any marking or modification. Its state is raw and unprocessed, corresponding only to the pixel information at a specific location.

[0031] A marker block is an image data block containing specific change information formed within a preset pixel area after a marking operation. A marked video frame is a video frame generated during video stream acquisition after the image data in a preset pixel area has been marked; the marker block is a component of the marked video frame. During normal video streaming, the image data of the marker block changes with each frame.

[0032] In this embodiment, the marking of image data in a preset pixel region for each frame is performed at the image signal source (such as an image acquisition device or a processing unit connected to it). The purpose is to actively modify the image data in a fixed preset pixel region within each frame that does not interfere with the main field of view. This modification is not random or based on image content, but rather generates and embeds marker data according to specific rules, making the image data in the preset pixel region form a marker block with expected changes. This generates a marked video frame containing the marker block, providing a dedicated and stable verification object for subsequent lag detection. This avoids the excessive computational burden caused by relying on full-image pixel analysis. Furthermore, the marking method is imperceptible to the human eye, ensuring compatibility between the marking operation and image display, without affecting the driver's observation of the normal image. The marker data is an information encoding representing a specific state, embedded in the original image data.

[0033] As one possible implementation of this application Figure 2 A specific implementation flow of step S101 in the electronic rearview mirror image lag processing method provided in this application embodiment is shown below: A1: Acquiring Ambient Light Intensity. Ambient light intensity refers to the real-time natural or artificial light intensity in the working environment of the electronic rearview mirror. It is a continuously changing physical quantity, and its value directly affects the human eye's sensitivity to changes in image color. For example, at night or in a tunnel, the ambient light intensity may be less than 100 Lux; while on a sunny day, the ambient light intensity may exceed 10,000 Lux. Ambient light intensity can be acquired through the ambient light sensor built into the image acquisition device.

[0034] A2: Based on the ambient light intensity, dynamically determine the modification bit depth of the chromaticity components of the preset pixel region in each frame of the image.

[0035] Chromaticity components refer to the U and V components in image pixel data, which together determine the color representation of the image. The human eye is less sensitive to changes in chromaticity components than to changes in luminance components (Y components). In this embodiment, the chromaticity components are modified to avoid visual interference. The modification bit depth refers to the minimum number of significant bits used to adjust the binary data of the chromaticity components. The bit depth determines the magnitude of changes in the directly associated marker data, thus affecting detectability in strong light environments and concealment in low light environments. In this embodiment, the modification bit depth is individually matched for the ambient light intensity corresponding to each frame of the image, achieving adaptive adjustment for different lighting scenarios.

[0036] As one possible implementation of this application, the chromaticity components include a first chromaticity component and a second chromaticity component; Figure 3 The following is a detailed implementation flow of determining the number of bits to be modified in the electronic rearview mirror image lag processing method provided in this application embodiment: A21: If the ambient light intensity is less than the first threshold, the modification bit of the first chromaticity component is determined to be one bit, and the second chromaticity component is not modified.

[0037] The first threshold is a preset critical value of light intensity that distinguishes between low-light and normal-light environments. In one possible implementation, the value of the first threshold is set based on the human eye's perception threshold for color change and the requirements of the actual driving scenario. For example, the first threshold is specifically set to 1000 lux. Modifying the bit depth to one bit means adjusting only the least significant bit of the color component. The least significant bit is the bit with the smallest weight in binary data. Modifying this bit results in a very small color change, which can ensure that it is not perceptible to the human eye in low-light environments.

[0038] When the ambient light intensity is less than the first threshold, in order to adapt to the human eye's perception characteristics in low-light environments, the markers are modified with minimal amplitude to ensure that the markers exist while minimizing visual interference and ensuring that the driving field of vision is not affected.

[0039] A22: If the ambient light intensity is greater than or equal to the first threshold and less than or equal to the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be one bit.

[0040] The second threshold is a preset critical value of light intensity that distinguishes between normal and strong light environments. In one possible implementation, the value of the second threshold is set based on the detectability requirements of the marked data under strong light conditions. For example, the second threshold is specifically set to 10000 lux. The range where the ambient light intensity is greater than or equal to the first threshold and less than or equal to the second threshold is considered a normal light environment, covering scenarios such as cloudy days and indoor parking lots. The human eye's sensitivity to color changes is at a moderate level. To adapt to the usage scenarios of normal light environments, both the first and second chromaticity components are modified using a single bit. This ensures the effectiveness and concealment of the markings without increasing the modification range, simplifies the processing logic, and controls the computational load.

[0041] A23: If the ambient light intensity is greater than the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be two bits.

[0042] The range where the ambient light intensity exceeds the second threshold is considered a strong light environment, encompassing scenarios such as midday sunlight and snow reflections. Strong light can mask subtle chromaticity changes, potentially causing subsequent detection algorithms to fail to capture label differences. Modifying the bit depth to two refers to adjusting the least significant two bits of the chromaticity components. Compared to a one-bit modification, the change is slightly larger, but still kept within the range imperceptible to the human eye, ensuring the detectability of labeled data under strong light conditions.

[0043] For example, the first chromaticity component is the U component, and the second chromaticity component is the V component. The modification bit depth for the U and V components is determined based on the different ranges of ambient light intensity. If the ambient light intensity is less than a first threshold, the modification bit depth for the U component is determined to be one bit, and the V component is not modified. If the ambient light intensity is greater than or equal to the first threshold and less than or equal to a second threshold, the modification bit depth for both the U and V components is determined to be one bit. If the ambient light intensity is greater than the second threshold, the modification bit depth for both the U and V components is determined to be two bits.

[0044] In this embodiment, to address the problem that tiny markers are easily obscured in strong light environments, the number of bits to be modified is reasonably determined to improve the detectability of the marker data without causing visual interference, thus ensuring the accuracy of jam detection in strong light scenarios.

[0045] A3: Based on the frame count and the number of bits modified during the video stream acquisition process, the image data in the preset pixel area is modified to generate the marker block.

[0046] In this embodiment, image data in a preset pixel region is modified, specifically targeting the least significant bit or the two least significant bits of the chroma components. The number of bits modified is one or two. Frame counting refers to the count value accumulated sequentially for each frame of the video stream during acquisition. Its initial value can be set to 0, and the count value increases by 1 for each acquired frame. The frame count value serves as the seed for the association of the marker data, ensuring that the marker data of adjacent frames are different. The marker data is generated based on the ambient light intensity and the current frame count value. The modification targets are the chroma components (U component, V component) of all pixels within the preset pixel region, prioritizing the modification of the chroma components to avoid visual interference caused by brightness changes.

[0047] In one possible implementation, when the number of bits to be modified is one, the least significant bit of the chroma component is modified to the binary value corresponding to the remainder of the frame count divided by 2; when the number of bits to be modified is two, the least two significant bits of the chroma component are modified to the binary value corresponding to the remainder of the frame count divided by 4.

[0048] For example, the frame counter is called Counter, which is initially 0 and increments by 1 every frame. The modified low-order bits of the UV components are associated with Counter: when only 1 bit is modified: Least Significant Bit (LSB) = Counter%2 (LSB = 0 when Counter is even and LSB = 1 when Counter is odd); when 2 bits are modified: the two least significant bits = Counter%4 (Counter modulo 4 results in 0-3, corresponding to 2 binary bits), ensuring that the UV data of the marker blocks in adjacent frames are different.

[0049] For example, a preset pixel area of ​​8×8 pixels is set in the lower right corner of each frame image. This area only defines the location of the marking operation. The marking method is adjusted according to the light intensity obtained by the ambient light sensor. If the light intensity is 500 lux (low light environment), it is determined that only the least significant bit of the U component in the chromaticity components of all pixels in the preset pixel area is modified. According to the progressive frame count, the value of the least significant bit is set to the result of taking the frame count value modulo 2. The V component remains unchanged. After this marking is completed, the set of pixels in the preset pixel area that carries specific data constitutes the marking block, and then a marked video frame containing the marking block is generated.

[0050] In this embodiment, the modification bit depth is adaptively adjusted by the lighting scene. The modification range is controlled in low-light and normal-light environments to ensure concealment, while the modification range is appropriately increased in strong-light environments to ensure detectability. This solves the problem of adapting the marking effect and concealment under different lighting scenes. By associating the marking data with frame counting, it is ensured that the marking blocks of each frame image have regular differences, providing a stable basis for subsequent continuous frame verification. At the same time, precise modification is performed in combination with the determined modification bit depth to generate marking blocks that are both concealed and detectable, laying the core foundation for lag detection.

[0051] Step S102: During the video stream display process, extract the display data of the marked block from the marked video frame.

[0052] The video stream display process refers to the process by which the display device receives the marked video frames and prepares to output them to the display screen for the driver to observe. The display data of the marked blocks refers to the pixel data located within the preset pixel area, which has been modified by the marking operation in step S101. It is still complete YUV data, but the least significant bit (or the least two significant bits) of the chrominance components has been adjusted according to the ambient light intensity and frame count, and its state is the state after targeted marking processing.

[0053] In this embodiment, by extracting the display data of the marker block, the core data for verification is accurately focused, avoiding interference from data in irrelevant image areas with subsequent detection results. At the same time, the amount of data processing is reduced, the computational load of the system is lowered, and the efficiency of lag detection is improved.

[0054] Step S103: Based on the display data corresponding to the marker blocks of a series of consecutive marked video frames, determine whether the video stream is stuck.

[0055] Multiple consecutive marked video frames refer to multiple video frames that have completed the marking operation during continuous transmission and processing. The specific number of consecutive marked video frames is determined by the lag frame count judgment threshold. This lag frame count judgment threshold can be preset, such as 6 to 24 frames, or it can be dynamically determined according to the actual frame rate fluctuation and the real-time speed of the vehicle.

[0056] In this embodiment of the application, continuous verification in time sequence avoids interference from single data anomalies caused by factors such as instantaneous noise and light fluctuations, thereby improving the accuracy of the stuck judgment.

[0057] As one possible implementation of this application Figure 4 A specific implementation flow of step S103 in the electronic rearview mirror image lag processing method provided in this application embodiment is shown below: B1: Calculate the check value based on the extracted display data of the marker block. The check value is a fixed-length numerical value obtained by processing the display data of the marker block using a preset check algorithm, and is used to quantitatively characterize the display data features of the marker block.

[0058] In one possible implementation, a cyclic redundancy check (CRC) algorithm is performed on the extracted display data to obtain a check value.

[0059] B2: If the check value corresponding to multiple consecutive marked video frames is the same, it is determined that the marked blocks have not changed, and the video stream display is stuck.

[0060] Same checksum means that the checksums corresponding to the display data of the marked blocks are completely consistent across multiple consecutive marked video frames, with no numerical change, indicating that the display data of the marked blocks has not been updated. No change in marked blocks means that the display data with marked features within the preset pixel area does not show the changes associated with the frame count set in step S101 across multiple consecutive marked video frames, which means that the video stream has not been updated normally. If the checksums corresponding to multiple consecutive marked video frames are the same, it is determined that the marked blocks have not changed, and the video stream display is stuck.

[0061] For example, the CRC algorithm uses the standard CRC-16-CCITT (polynomial of 0x1021, initial value of 0xFFFF) to calculate all YUV data of an 8x8 pixel marker block. The CRC values ​​of the current frame are continuously compared with those of historical frames. If the number of frames in which the CRC value of the same marker block remains unchanged exceeds the threshold for judging the number of stuck frames, then it is judged as stuck.

[0062] In this embodiment, the display data of the marker block is converted into a quantized check value, replacing the complex calculation of full pixel comparison, which significantly reduces the amount of data processing and the computational load of the system. At the same time, by continuously comparing the quantized check values, it is possible to accurately determine whether the display data of the marker block has been updated, avoiding the interference of instantaneous noise and illumination fluctuations on single-frame data, and preventing normal static images from being misjudged as stuck.

[0063] As one possible implementation of this application Figure 5 Another specific implementation flow for detecting jamming in the electronic rearview mirror image jamming processing method provided in the embodiments of this application is shown below: C1: Obtain the vehicle's real-time speed and the actual frame rate of the video stream.

[0064] The vehicle's real-time speed can be obtained via the vehicle's controller LAN bus. The actual frame rate of the video stream can be calculated using a built-in timer (1 millisecond precision) on the display host.

[0065] C2: Based on the actual frame rate and the real-time speed of the vehicle, dynamically calculate the threshold for judging the number of stuck frames.

[0066] The stuttering frame count threshold refers to the critical number of consecutive frames in which the marker block remains unchanged when stuttering occurs in the video stream. In this embodiment, the stuttering frame count threshold is dynamically adjusted according to the actual frame rate and the real-time speed of the vehicle, rather than being a fixed value.

[0067] In one possible implementation, a dynamic time window is calculated based on the vehicle's real-time speed; a base threshold is calculated based on the actual frame rate and the dynamic time window; if the fluctuation range between the actual frame rate and the system-set frame rate exceeds a preset amplitude threshold (e.g., 10%), a threshold for determining the number of stuck frames is determined based on the base threshold and the preset frame threshold.

[0068] In one possible implementation, the dynamic time window refers to the shortest continuous time required to determine the jam, and the specific calculation formula is: Dynamic Time Window = Maximum Value (Minimum Time Window, Baseline Dynamic Time Window - Adjustment Coefficient × Real-Time Vehicle Speed); where the baseline dynamic time window is the time window when the real-time vehicle speed is zero, in seconds, and the adjustment coefficient is in seconds per kilometer. The function of the hour is to linearly adjust the reference time window based on the real-time speed of the vehicle. The higher the vehicle speed, the larger the product of the adjustment coefficient and the vehicle speed.

[0069] For example, the dynamic time window = maximum value (0.2, 0.4-0.005 × real-time vehicle speed), where the minimum time window is 0.2 seconds to ensure that the threshold is not too low in high-speed scenarios, the baseline dynamic time window is 0.4, the adjustment coefficient is 0.005, and 0.4-0.005 × real-time vehicle speed increases the time window at low speeds, reducing misjudgments of static images.

[0070] In one possible implementation, the base threshold is equal to (actual frame rate × dynamic time window). The base threshold is rounded up using a function that ensures that the base threshold is an integer frame.

[0071] In one possible implementation, the fluctuation range is calculated as |Actual Frame Rate - System Set Frame Rate| / System Set Frame Rate × 100%, where |.| represents the absolute value. For example, if the fluctuation range exceeds 10%, the threshold for judging the number of stuck frames is temporarily set to the base threshold + 2 frames to compensate for the risk of misjudgment caused by frame rate fluctuations.

[0072] C3: When the marker blocks corresponding to the consecutive multiple marked video frames do not change, and the number of consecutive marked video frames that do not change reaches the lag frame count judgment threshold, it is determined that the video stream display is lag-prone.

[0073] In this embodiment, the changes in the marker blocks are verified based on the display data corresponding to the marker blocks in multiple consecutive marked video frames. Verifying the changes in the marker blocks refers to determining whether there are regular changes in the display data based on the marker block display data extracted from multiple consecutive marked video frames. No change in marker blocks within multiple consecutive marked video frames means that, within a set verification window, the checksums corresponding to the marker block display data extracted from all marked video frames are completely consistent. For example, if the number of consecutive marked video frames with continuously unchanged checksums reaches the lag frame count threshold, then it is determined that the video stream display is lag-prone. Based on the continuous change characteristics of the marker block data, it accurately determines whether the video stream display is lag-prone, avoiding misjudging normal static image scenes as lag, while ensuring that lag phenomena are not missed, providing accurate triggering conditions for subsequent fault handling.

[0074] In this embodiment, dynamic threshold calculation is used to adapt the frame rate judgment threshold to different driving speeds and video stream processing states. The threshold is smaller in high-speed scenarios to ensure the timeliness of frame rate detection; the threshold is larger in low-speed / static scenarios to avoid misjudging normal static images as frame rate drops. The threshold is temporarily adjusted when the frame rate fluctuates to avoid misjudgment caused by frame counter jumps, thereby improving the scene adaptability and robustness of the threshold. Based on the dynamic threshold, accurate frame rate judgment can be achieved, avoiding misjudgment under fixed thresholds in different scenarios.

[0075] In one possible implementation, when it is determined that the video stream display is stuck, system status data is collected; the system status data is stored in a non-volatile memory.

[0076] Step S105: If jamming occurs, perform a fault recovery operation.

[0077] The purpose of fault recovery operations is to quickly restore the normal acquisition and display of video streams.

[0078] In one possible implementation, if a jam occurs, a first-level recovery operation is performed, which includes resetting the image acquisition device and the display device.

[0079] As one possible implementation of this application Figure 6.1 The following is a detailed implementation flow of the first-level recovery operation in the electronic rearview mirror image lag processing method provided in this application embodiment: D1: Output warning information. Warning information refers to messages used to remind the driver that the electronic rearview mirror is experiencing image lag and that the system is recovering. In one possible implementation, outputting warning information includes playing a pre-recorded warning voice message. By outputting warning information to the driver, the driver is informed that the current electronic rearview mirror image is invalid, preventing safety accidents caused by misjudging environmental conditions.

[0080] D2: Controls the display device to output a predetermined image frame.

[0081] The display device includes a screen on an electronic rearview mirror, used to display images of the vehicle's surrounding environment captured and processed by an image acquisition device for the driver to observe. A predetermined image frame refers to a pre-set standard image frame used to clearly indicate to the driver that the current image is invalid. In one possible implementation, the predetermined image frame is a black image frame. By controlling the display device to output the predetermined image frame, the driver is visually alerted that the current image is invalid, forming a dual reminder with the voice warning information, further enhancing the warning effect.

[0082] D3: Reset the image acquisition device and the display device. Resetting refers to restarting the device through hardware control, clearing temporary faults in the device's operation (such as software deadlock or signal abnormality), and restoring the device to its initial working state.

[0083] In one possible implementation, the image acquisition device is reset by controlling its power supply through the general-purpose input / output pins of the display host. A preset power-off period followed by power-on triggers device re-initialization. The display device is reset by sending a reset command through the serial peripheral interface of the display host. After the reset is complete, normal display driving is restored. This hardware reset quickly clears temporary faults, restoring both the image acquisition device and the display device to normal operating status.

[0084] As one possible implementation of this application, such as Figure 6.2 As shown, after performing the first-level recovery operation, the following steps are also included: E1: Re-verify the changes in the marker block. Re-verification means that after the first-level recovery operation is completed, the verification logic corresponding to step S103 is executed again to check the changes in the marker block and verify whether the stuck fault has been eliminated. The verification process is the same as described above and will not be repeated here.

[0085] E2: If it is determined that the jamming has not been eliminated, a second-level recovery operation is performed, which includes restarting the image processing unit.

[0086] If the displayed data of the marked blocks remains unchanged (the checksums within multiple consecutive marked video frames remain consistent), it is determined that the first-level recovery operation cannot resolve the current fault. In this case, the second-level recovery operation is executed. The second-level recovery operation refers to a more in-depth fault repair method compared to the first-level recovery.

[0087] The image processing unit refers to the system-on-a-chip (SoC) built into the display host. In one possible implementation, the image processing unit is restarted via a hardware watchdog timer. The hardware watchdog timer is a hardware module independent of the image processing unit, which can send a reset signal to force a restart of the image processing unit, ensuring that the restart operation is effectively executed. After restarting, the image processing unit automatically executes the initialization process and restores its core functions.

[0088] In this embodiment, for deep faults that cannot be resolved by the first-level recovery operation, the complex faults are cleared by restarting the core processing unit, which further improves the success rate of fault repair. At the same time, the core content of the second-level recovery operation is clarified, forming a hierarchical recovery system to avoid excessive recovery from affecting the system operation.

[0089] As can be seen from the above, in this embodiment, by marking the image data of a preset pixel area in each frame during video stream acquisition, a video frame containing marked blocks is generated. These marked blocks change with each frame, providing a dedicated verification basis for subsequent image lag detection. Targeted detection can be achieved without relying on full-image pixel analysis, effectively simplifying the detection logic. Extracting the display data of the marked blocks in the marked video frames during video stream display allows for precise focusing on key verification objects, avoiding interference from irrelevant image areas and ensuring the validity of the detection data. Based on the display data extracted from multiple consecutive marked video frames, it is determined whether the video stream has experienced image lag. Continuous temporal verification effectively avoids misjudgments caused by instantaneous interference, ensuring the accuracy of lag determination. Upon determining that lag has occurred, timely fault recovery operations are performed, enabling rapid response to fault scenarios and preventing drivers from misjudging environmental conditions due to image lag, thus minimizing safety hazards. This application's solution achieves accurate detection and rapid recovery of image lag, improving the reliability of electronic rearview mirror image lag processing, thereby providing strong protection for driving safety.

[0090] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0091] Corresponding to the electronic rearview mirror image lag processing method described in the above embodiments, Figure 7 A structural block diagram of the electronic rearview mirror image lag processing device provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0092] Reference Figure 7 The electronic rearview mirror image lag processing device includes: an image marking unit 71, a marker block extraction unit 72, a lag detection unit 73, and a fault recovery unit 74, wherein: Image marking unit 71 is used to mark the image data of a preset pixel area of ​​each frame image during the video stream acquisition process, and generate a marked video frame containing a marked block, wherein the marked block changes with the frame; The marker block extraction unit 72 is used to extract the display data of the marker blocks from the marked video frames during the video stream display process; The jamming detection unit 73 is used to perform a fault recovery operation if jamming occurs. The fault recovery unit 74 is used to perform a fault recovery operation if a jam occurs.

[0093] As one possible implementation of this application, the image tagging unit 71 includes: The environmental information acquisition module is used to acquire ambient light intensity; The bit depth determination module is used to dynamically determine the bit depth of the chromaticity components of the preset pixel region in each frame of the image based on the ambient light intensity. The marker block generation module is used to modify the image data in the preset pixel area based on the frame count and the number of bits modified during the video stream acquisition process, and generate the marker block.

[0094] As one possible implementation of this application, the chromaticity components include a first chromaticity component and a second chromaticity component; the modified bit depth determination module is specifically used for: If the ambient light intensity is less than the first threshold, the modification bit of the first chromaticity component is determined to be one bit, and the second chromaticity component is not modified. If the ambient light intensity is greater than or equal to the first threshold and less than or equal to the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be one bit. If the ambient light intensity is greater than the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be two bits.

[0095] As one possible implementation of this application, the jamming detection unit 73 includes: The verification value calculation module is used to calculate the verification value based on the extracted display data of the marker block; The change verification module is used to determine that the marking blocks have not changed and the video stream display is stuck if the verification values ​​corresponding to multiple consecutive marked video frames are the same.

[0096] As one possible implementation of this application, the fault recovery unit 75 includes: The first recovery unit is used to perform a first-level recovery operation, which includes resetting the image acquisition device and the display device.

[0097] As one possible implementation of this application, the fault recovery unit 75 further includes: The second recovery unit is used to re-verify the changes in the marker block; if it is determined that the jamming has not been eliminated, a second-level recovery operation is performed, which includes restarting the image processing unit.

[0098] As one possible implementation of this application, the jamming detection unit 74 includes: The information acquisition module is used to acquire the vehicle's real-time speed and the actual frame rate of the video stream; The threshold calculation module is used to dynamically calculate the number of stuck frames and determine the threshold based on the actual frame rate and the real-time speed of the vehicle. The stuttering verification module is used to determine that the video stream display is stuttering when the marker blocks corresponding to multiple consecutive marked video frames do not change and the number of consecutive marked video frames that do not change reaches the stuttering frame count judgment threshold.

[0099] As can be seen from the above, in this embodiment, by marking the image data of a preset pixel area in each frame during video stream acquisition, a video frame containing marked blocks is generated. These marked blocks change with each frame, providing a dedicated verification basis for subsequent image lag detection. Targeted detection can be achieved without relying on full-image pixel analysis, effectively simplifying the detection logic. Extracting the display data of the marked blocks in the marked video frames during video stream display allows for precise focusing on key verification objects, avoiding interference from irrelevant image areas and ensuring the validity of the detection data. Based on the display data extracted from multiple consecutive marked video frames, it is determined whether the video stream has experienced image lag. Continuous temporal verification effectively avoids misjudgments caused by instantaneous interference, ensuring the accuracy of lag determination. Upon determining that lag has occurred, timely fault recovery operations are performed, enabling rapid response to fault scenarios and preventing drivers from misjudging environmental conditions due to image lag, thus minimizing safety hazards. This application's solution achieves accurate detection and rapid recovery of image lag, improving the reliability of electronic rearview mirror image lag processing, thereby providing strong protection for driving safety.

[0100] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0101] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figure 1 Up to Figure 6 (including) Figure 6.1 , Figure 6.2 ( ) represents the steps of any electronic rearview mirror image lag processing method.

[0102] This application embodiment also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements... Figure 1 Up to Figure 6 (including) Figure 6.1 , Figure 6.2 ( ) represents the steps of any electronic rearview mirror image lag processing method.

[0103] This application also provides a computer program product that, when run on a terminal device, causes the terminal device to execute the following implementation: Figure 1 Up to Figure 6 (including) Figure 6.1 , Figure 6.2 ( ) represents the steps of any electronic rearview mirror image lag processing method.

[0104] Figure 8 This is a schematic diagram of a terminal device provided in an embodiment of this application. For example... Figure 8 As shown, the terminal device 8 in this embodiment includes: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. When the processor 80 executes the computer program 82, it implements the steps in the various embodiments of the electronic rearview mirror image lag processing method described above, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 80 executes the computer program 82, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 7 The functions of units 71 to 74 are shown.

[0105] For example, the computer program 82 may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program 82 in the terminal device 8.

[0106] The terminal device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will understand that... Figure 8 This is merely an example of terminal device 8 and does not constitute a limitation on terminal device 8. It may include more or fewer components than shown, or combine certain components, or different components. For example, terminal device 8 may also include input / output devices, network access devices, buses, etc.

[0107] The processor 80 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0108] The memory 81 can be an internal storage unit of the terminal device 8, such as a hard disk or memory of the terminal device 8. The memory 81 can also be an external storage device of the terminal device 8, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the terminal device 8. Furthermore, the memory 81 can include both internal and external storage units of the terminal device 8. The memory 81 is used to store the computer program and other programs and data required by the terminal device. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0109] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0110] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0112] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for processing image lag in an electronic rearview mirror, characterized in that, include: During video stream acquisition, image data of a preset pixel region in each frame is marked to generate a marked video frame containing marked blocks, wherein the marked blocks change with the frame. During the display of the video stream, the display data of the marked blocks is extracted from the marked video frames; Based on the display data corresponding to the marker blocks of multiple consecutive marked video frames, it is determined whether the video stream is experiencing stuttering; If a jam occurs, perform a fault recovery operation.

2. The method according to claim 1, characterized in that, The step of marking image data in a preset pixel region of each frame to generate a marked video frame containing the marked blocks includes: Obtain ambient light intensity; Based on the ambient light intensity, the modification bit depth of the chromaticity components of the preset pixel region in each frame of the image is dynamically determined; Based on the frame count and the number of bits modified during the video stream acquisition process, the image data in the preset pixel region is modified to generate the marker block.

3. The method according to claim 2, characterized in that, The chromaticity components include a first chromaticity component and a second chromaticity component; the step of dynamically determining the modification bit depth of the chromaticity components of the preset pixel region in each frame image based on the ambient light intensity includes: If the ambient light intensity is less than the first threshold, the modification bit of the first chromaticity component is determined to be one bit, and the second chromaticity component is not modified. If the ambient light intensity is greater than or equal to the first threshold and less than or equal to the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be one bit. If the ambient light intensity is greater than the second threshold, then the modification bit depth of both the first chromaticity component and the second chromaticity component is determined to be two bits.

4. The method according to claim 1, characterized in that, The step of determining whether the video stream is experiencing stuttering based on the display data corresponding to the marker blocks of a series of consecutive marked video frames includes: Calculate the check value based on the extracted display data of the marked blocks; If the checksums of multiple consecutive marked video frames are the same, it is determined that the marked blocks have not changed, and the video stream display is stuck.

5. The method according to claim 4, characterized in that, The fault recovery operation includes: Perform a first-level recovery operation, which includes resetting the image acquisition device and the display device.

6. The method according to claim 5, characterized in that, After performing the first-level recovery operation, the following steps are also included: Re-verify the changes in the marked blocks; If it is determined that the lag has not been eliminated, a second-level recovery operation is performed, which includes restarting the image processing unit.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Obtain the vehicle's real-time speed and the actual frame rate of the video stream; Based on the actual frame rate and the real-time speed of the vehicle, dynamically calculate the number of stuck frames to determine the threshold; When the marker blocks corresponding to multiple consecutive marked video frames remain unchanged, and the number of consecutive marked video frames that remain unchanged reaches the stuttering judgment threshold, it is determined that the video stream display has stuttered.

8. An electronic rearview mirror image lag processing device, characterized in that, include: An image labeling unit is used to label the image data of a preset pixel region of each frame during the video stream acquisition process, and generate a labeled video frame containing a label block, wherein the label block changes with the frame. A marker block extraction unit is used to extract display data of the marker blocks from the marked video frames during the display of the video stream; The stuttering detection unit is used to determine whether the video stream is stuttering based on the display data corresponding to the marker blocks of a series of consecutive marked video frames; The fault recovery unit is used to perform fault recovery operations if a jam occurs.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the electronic rearview mirror image lag processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the electronic rearview mirror image lag processing method as described in any one of claims 1 to 7.

Images