Collected image processing method and system of streaming media rearview mirror

Abstract

The invention relates to the technical field of streaming media rearview mirror image processing, and discloses an acquired image processing method and system for a streaming media rearview mirror, and the method comprises the steps: photographing the rear part of a vehicle through a camera at the tail part of a vehicle body, acquiring an image, marking the image acquired in real time as a real-time frame image, and storing the real-time frame image in a server; and acquiring an image before the preset frame number of the real-time frame image, and marking the image as a preset frame image. According to the method, the similarity between the real-time frame image and the preset frame image is calculated by using the SSIM, abnormal changes caused by vehicle vibration and tiny changes in a normal scene can be quickly distinguished, and when large-area motion blur is caused by vehicle vibration, the image edge is cut, so that the image quality is improved. Therefore, small-size images can be processed by subsequent image processing algorithms such as geometric correction and feature extraction, and the real-time processing performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of streaming media rearview mirror image processing, and in particular to a method and system for processing collected images of a streaming media rearview mirror. Background Art

[0002] A streaming rearview mirror is an automotive feature that utilizes digital technology to achieve rearview vision, replacing traditional glass mirrors. A camera mounted on the rear of the vehicle captures real-time images from behind the vehicle and converts the analog signal into a digital signal. The video data is then transmitted to a display system via a stable internal data network. This transmission process typically requires low latency and high stability to ensure that the driver can receive the latest images in real time.

[0003] Since cameras, especially wide-angle cameras, are prone to barrel distortion, it is necessary to use camera calibration parameters to perform geometric correction on the image to correct the distortion problem at the edges, so as to obtain an image with a true proportion. However, in the actual process of lens distortion correction, the vehicle may be affected by more severe vibrations while driving, making the edge distortion of the image captured by the camera more serious. The correction of the image edge will require more calculation steps and introduce additional delays. The streaming rearview mirror requires that image processing must be completed in an extremely short time. The additional computational burden will affect the timeliness of real-time display and affect driving safety. Summary of the Invention

[0004] In response to the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a method and system for collecting and processing images for a streaming media rearview mirror, so as to adaptively adjust the correction processing method for severely distorted images during the image collection process of the streaming media rearview mirror, thereby ensuring the timeliness of the image displayed to the driver.

[0005] In order to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for processing collected images of a streaming media rearview mirror, comprising the following steps:

[0006] Use the camera at the rear of the vehicle to shoot and collect images from the rear of the vehicle, mark the real-time collected images as real-time frame images, and obtain images before a preset number of frames of the real-time frame images and mark them as preset frame images;

[0007] Calculate the SSIM of the real-time frame image and the preset frame image, and obtain the similarity between the real-time frame image and the preset frame image according to the SSIM result, and mark the similarity as the similarity of the two frame images;

[0008] Set a similarity threshold and compare the similarity between the two frames of images with the similarity threshold. If the similarity between the two frames of images is greater than or equal to the similarity threshold, the images are geometrically corrected according to the calibration parameters. If the similarity between the two frames of images is less than the similarity threshold, the vibration judgment strategy is executed.

[0009] Executing the vibration judgment strategy includes obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vectors of the pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, the image edges are cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror.

[0010] In some embodiments, a motion pixel threshold is set. When pixels on all consecutive image frames have a consistent motion direction, the proportion of pixels with a consistent motion direction is obtained. The specific method for obtaining the consistent motion pixel ratio is: obtain the total number of pixels with a consistent motion direction Rn and the total number of image pixels Tn, divide the two to obtain the consistent motion pixel ratio Rb=Rn÷Tn, and compare the consistent motion pixel ratio with the motion pixel threshold, and make a corresponding response based on the comparison result.

[0011] In some embodiments, when the consistent motion pixel ratio exceeds the motion pixel threshold, the judgment that the vehicle has experienced significant vibration and caused the camera to shake is maintained, and subsequent processing steps are executed; when the consistent motion pixel ratio does not exceed the motion pixel threshold, it is judged that the vehicle has not experienced significant vibration, the captured image is normally corrected and output to the streaming rearview mirror.

[0012] In some embodiments, when the consistent motion pixel ratio exceeds the motion pixel threshold, the distribution and number of pixels that do not generate a consistent motion direction on the continuous image frames are obtained, the pixels that do not generate a consistent motion direction on the continuous image frames are marked as changed pixels, and the total number of changed pixels in the area to be cropped is obtained. By dividing the total number of changed pixels in the area to be cropped by the total pixels in the area to be cropped, the proportion of changed pixels in the area to be cropped can be obtained, and the proportion is marked as the changed pixel proportion. A changed pixel proportion threshold is set, the changed pixel proportion is compared with the changed pixel proportion threshold, and a corresponding response is made based on the comparison result.

[0013] In some embodiments, when the ratio of changed pixels is greater than the ratio threshold of changed pixels, the cropping area retention strategy is executed; when the ratio of changed pixels is less than or equal to the ratio threshold of changed pixels, the edge cropping of the image is canceled, the captured image is corrected normally and output to the streaming rearview mirror.

[0014] In some embodiments, in the process of processing subsequent continuous image frames, each time the edge cropping of the image is canceled due to the condition that the changed pixel ratio is less than or equal to the changed pixel ratio threshold, a cancel cropping count is performed. The cancel cropping count must continue to increase under continuous triggering, and is cleared when the trigger is interrupted, and a cancellation threshold is set. When the cancel cropping count exceeds the cancellation threshold, the image edge cropping is no longer canceled, and the image edge cropping is performed normally to reduce the computational burden of image processing.

[0015] In some embodiments, the cropping area retention strategy includes setting a threshold for the number of changed pixels, dividing the area to be cropped into an upper area, a lower area, a left area, and a right area, obtaining the corresponding area with the least number of changed pixels, and comparing the total number of changed pixels in the area with the threshold for the number of changed pixels. If the total number of changed pixels in the area exceeds the threshold for the number of changed pixels, the area to be cropped is not retained, and cropping of the image edge is performed normally; if the total number of changed pixels in the area does not exceed the threshold for the number of changed pixels, when cropping the image edge, cropping of the area is canceled, that is, the area is retained for normal image processing and output on the streaming media rearview mirror.

[0016] In some embodiments, the corresponding area with the second smallest number of changed pixels is obtained, the total number of changed pixels in the area is summed with the total number of changed pixels in the previously retained area to obtain the cumulative number of changed pixels, and the cumulative number of changed pixels is compared with the threshold number of changed pixels. If the cumulative number of changed pixels exceeds the threshold number of changed pixels, the area is not retained; if the cumulative number of changed pixels does not exceed the threshold number of changed pixels, the cropping of the area is canceled, and so on. The corresponding area with the third smallest number of changed pixels and the corresponding area with the largest number of changed pixels are continued to be obtained, the cumulative number of changed pixels in the corresponding areas are compared with the threshold number of changed pixels, and corresponding judgments are made.

[0017] The present invention also provides the following technical solution: a system for collecting and processing images of a streaming media rearview mirror, comprising:

[0018] An image acquisition module, which includes using a camera at the rear of the vehicle to shoot and capture images from the rear of the vehicle, marking the real-time captured images as real-time frame images, and obtaining images before a preset number of frames of the real-time frame image and marking them as preset frame images;

[0019] A pixel comparison module includes calculating the SSIM of the real-time frame image and the preset frame image, and obtaining the similarity between the real-time frame image and the preset frame image according to the SSIM result, and marking the similarity as the similarity of the two frame images;

[0020] The data processing module includes setting a similarity threshold, comparing the similarity of the two frames of images with the similarity threshold, and performing geometric correction on the images according to the calibration parameters if the similarity of the two frames of images is greater than or equal to the similarity threshold; if the similarity of the two frames of images is less than the similarity threshold, executing the vibration judgment strategy;

[0021] The strategy execution module includes executing the vibration judgment strategy, including obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vectors of the pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, the image edges are cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror.

[0022] The present invention further provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the above-mentioned method for collecting and processing images of a streaming media rearview mirror.

[0023] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0024] First, the present invention uses SSIM to calculate the similarity between the real-time frame image and the preset frame image, which can quickly distinguish between abnormal changes caused by vehicle vibration and minor changes in normal scenes. When vehicle vibration causes large-area motion blur, by cropping the image edges, subsequent image processing algorithms such as geometric correction and feature extraction can process smaller-sized images, thereby improving real-time processing performance and avoiding the delay problem of streaming media rearview mirrors caused by the image processing burden.

[0025] Secondly, the present invention counts the degree of pixel change in the cropped area. When the proportion of changed pixels is small, it indicates that the computational burden brought by cropping is limited. Retaining this area can expand the driver's field of view as much as possible. At the same time, the cropping cancellation counting mechanism triggered by continuous image frames can restore edge cropping in time when continuous jitter causes the system to continue to have a high load, thereby achieving a balanced burden on the overall image processing of the system, improving real-time response capabilities and image stability.

[0026] Third, the present invention divides the area to be cropped into four independent areas: top, bottom, left, and right. It can make targeted retention or cropping decisions based on the number of changed pixels in each area. This local trade-off method avoids a one-size-fits-all global cropping, which can prevent the system from being under high computing load for a long time when the camera vibrates greatly, and retain additional image information as much as possible in relatively stable scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1This is a flow chart of a method for collecting and processing images of a streaming media rearview mirror according to the present invention;

[0028] Figure 2 This is a module schematic diagram of an image acquisition and processing system for a streaming media rearview mirror according to the present invention;

[0029] Figure 3 Schematic diagram of the area division of the area to be cropped according to the present invention. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0031] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the elements may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0032] The present invention provides a method for processing collected images of a streaming media rearview mirror, such as Figure 1 As shown, the method includes the following steps:

[0033] Step 1: Use the camera at the rear of the vehicle to shoot and capture images from the rear of the vehicle, mark the real-time captured images as real-time frame images, and obtain images before a preset number of frames of the real-time frame image and mark them as preset frame images. The preset frame can be 5 to 10 frames. For example, when 10 frames are taken, the preset frame image is the image 10 frames before the real-time frame image.

[0034] Step 2: Calculate the SSIM (structural similarity index) between the real-time frame image and the preset frame image. The obtained index value range is between 0 and 1. Based on the SSIM result, the similarity between the real-time frame image and the preset frame image is obtained. This similarity is marked as the similarity between the two frames. For example, if the SSIM is 0.9, the similarity between the two frames is 90%;

[0035] Step 3: Set a similarity threshold and compare the similarity between the two frames of image with the similarity threshold. If the similarity between the two frames of image is greater than or equal to the similarity threshold, it indicates that the real-time frame image has a small change compared with the preset frame image, and there is only reasonable change between the two images. Then, the image is geometrically corrected according to the calibration parameters, and the corrected image is displayed on the streaming media rearview mirror for the driver to watch. If the similarity between the two frames of image is less than the similarity threshold, it indicates that the real-time frame image has a drastic change compared with the preset frame image, and the two images have changed significantly after a short frame change, then the vibration judgment strategy is executed.

[0036] Step 4. The vibration judgment strategy includes obtaining all continuous image frames between the real-time frame image and the preset frame image, and using an optical flow algorithm (such as Lucas-Kanade) to calculate the motion vectors of pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, it is judged that the vehicle has experienced a large vibration and the camera is shaking. The motion blur generated at this time will cause the details in the image to be lost and the computational pressure for the real-time correction processing of the image to increase sharply. The image edge is cropped to reduce the size of the generated image, and the generated image is output on the streaming media rearview mirror. The range of cropping at the edge can be 5%-20%. After cropping the edge, the image size is reduced and the processing area is narrowed, which can reduce the computational complexity of subsequent geometric correction, feature extraction and other real-time image processing algorithms, thereby improving the real-time processing performance of the system and reducing the problem of sudden increase in computational pressure due to vibration. When the vehicle vibrates significantly or the camera shakes, the edges of the image are more prone to dynamic blur or motion distortion. This is especially true for the wide-angle camera at the rear of the vehicle. After the image captured produces barrel distortion, the deformation and distortion are most severe at the edges. By cropping the edges, the areas with the most severe blur and distortion caused by the vibration can be removed, making the geometrically corrected image more stable and more in line with the actual scene, avoiding the loss of details that interfere with the driver's perception, and improving the timeliness and reliability of the final output image of the streaming rearview mirror.

[0037] Set a motion pixel threshold. When the pixels on all consecutive image frames have a consistent motion direction, obtain the ratio of pixels with a consistent motion direction and mark it as the consistent motion pixel ratio. Specifically, the consistent motion pixel ratio Rb=Rn÷Tn is obtained by combining the total number of pixels Rn with a consistent motion direction and the total number of image pixels Tn. Compare the consistent motion pixel ratio Rb with the motion pixel threshold. When the consistent motion pixel ratio exceeds the motion pixel threshold, it indicates that most of the pixels on all consecutive image frames have produced consistent motion. This is a strong evidence that the camera lens shakes and causes the overall image shake. In this case, the judgment that the vehicle shakes significantly and causes the camera to shake is maintained, and subsequent operations are performed. Processing steps; when the consistent motion pixel ratio does not exceed the motion pixel threshold, it indicates that the proportion of pixels that produce consistent motion in all continuous image frames is not high, which also means that the pixels on the continuous image frames do not produce overall consistent direction of motion. The presence of pixels with consistent motion directions on the continuous image frames may be due to vehicles, roadside objects, etc. entering the camera's shooting range behind the vehicle. The changes produced in this shooting picture are normal shooting methods. The camera lens is not shaken by the vibration of the vehicle, and the captured image will not cause dynamic blur and distortion. It is judged that the vehicle has not produced significant vibration, and the captured image is normally corrected and output to the streaming rearview mirror.

[0038] For example, it is assumed that a frame image captured by the rear camera has 2.5 million pixels, the preset frame image uses the images 10 frames before the real-time frame image, the similarity between the two frames is set to 0.7, and the similarity threshold is set to 0.8. Since the similarity between the two frames is less than the similarity threshold, it indicates that the change amplitude between the real-time frame image and the preset frame image is large. The vibration judgment strategy is executed, and it is set that there are 1 million pixels with consistent motion directions in the continuous image frames between the preset frame image and the real-time frame image. The consistent motion pixel ratio can be obtained to be 0.4, and the motion pixel threshold is set to 0.9. Since the consistent motion pixel ratio does not exceed the motion pixel threshold, it is judged that the vehicle has not generated significant vibration, and the image edge is not cropped, but the captured image is normally corrected and output to the streaming rearview mirror.

[0039] Furthermore, after executing the above steps, when it is determined that the edge of the image needs to be cropped, the distribution and number of pixels that do not produce a consistent motion direction on the continuous image frames are obtained, the pixels that do not produce a consistent motion direction on the continuous image frames are marked as changed pixels, and the total number of changed pixels in the area to be cropped is obtained. The area to be cropped refers to the part that is cropped when the image is cropped. Since the image edge is cropped, the area to be cropped is a frame-shaped area. By dividing the total number of changed pixels in the area to be cropped by the total pixels in the area to be cropped, the proportion of changed pixels in the area to be cropped can be obtained, and the proportion is marked as the changed pixel proportion. A changed pixel proportion threshold is set, and the changed pixel proportion is compared with the changed pixel proportion threshold. When the changed pixel proportion is greater than the changed pixel proportion threshold, it indicates that a large number of pixels that do not generate consistent motion directions on continuous image frames are distributed in the area to be cropped. Although cropping the image edge can effectively reduce the system's computational burden on image processing, the cropped area of ​​the image should be retained as much as possible to improve the field of view of the image presented by the streaming media rearview mirror. For this purpose, the cropped area retention strategy is executed; when the changed pixel proportion is less than or equal to the changed pixel proportion threshold, it indicates that pixels that do not generate consistent motion directions on continuous image frames. A smaller number of pixels are located in the to-be-cropped area, with a greater number located in the center of the image, i.e., the area that will not be cropped. This also means that the image processing computational burden reduced by cropping the image edges is limited. Therefore, the image edge cropping is canceled, the captured image is corrected normally, and output to the streaming rearview mirror. It should be noted that during the processing of subsequent consecutive image frames, each time the image edge cropping is canceled due to the triggering condition that the change pixel ratio is less than or equal to the change pixel ratio threshold, an uncropping count is counted. The uncropping count must continue to increase under continuous triggering. It is reset to zero when the trigger is interrupted, and a cancellation count threshold is set. When the uncropping count exceeds the cancellation count threshold, the image edge cropping is no longer canceled, and image edge cropping is performed normally to reduce the image processing computational burden. This is because the image blurring and distortion caused by continuous camera shake will cause the system image processing computing power to be consistently high when the edge cropping is canceled. To ensure overall image processing speed and stability, the edge cropping is restored after the to-be-cropped area has been retained multiple times in a row.

[0040] The cropping area retention strategy includes setting a threshold value for the number of pixels changed, dividing the area to be cropped into an upper area, a lower area, a left area, and a right area (see Figure 3), obtain the corresponding area with the least number of changed pixels, compare the total number of changed pixels in the area with the threshold number of changed pixels, if the total number of changed pixels in the area exceeds the threshold number of changed pixels, do not retain the area to be cropped, and perform normal cropping of the image edge; if the total number of changed pixels in the area does not exceed the threshold number of changed pixels, then when cropping the image edge, cancel the cropping of the area, that is, retain the area, perform normal image processing on it, and output it on the streaming media rearview mirror;

[0041] Furthermore, continue to obtain the corresponding area with the second smallest number of changed pixels, sum the total number of changed pixels in the area with the total number of changed pixels in the previously retained area to obtain the cumulative number of changed pixels, compare the cumulative number of changed pixels with the threshold number of changed pixels, if the cumulative number of changed pixels exceeds the threshold number of changed pixels, then the area will not be retained; if the cumulative number of changed pixels does not exceed the threshold number of changed pixels, then cancel the cropping of the area, and so on, continue to obtain the corresponding area with the third smallest number of changed pixels and the corresponding area with the largest number of changed pixels, compare the cumulative number of changed pixels in the corresponding area with the threshold number of changed pixels, and make corresponding judgments.

[0042] In summary, the present invention aims to design a method and system for processing captured images for a streaming rearview mirror. This method addresses the problem of blurring and distortion in camera-captured images caused by vehicle vibration, which can lead to image correction delays when outputting the images to the streaming rearview mirror. The present invention uses SSIM (Simultaneous Similarity Calculation) to calculate the similarity between real-time frame images and preset frame images, quickly distinguishing between abnormal changes caused by vehicle vibration and minor changes in normal scenes. When vehicle vibration causes large-area motion blur, the present invention clips the image edges, allowing subsequent image processing algorithms such as geometric correction and feature extraction to process smaller images, improving real-time processing performance. Edge clipping not only reduces computational effort but also prioritizes the image's central region (which typically contains more important information), reducing interference caused by blurring and distortion. By statistically analyzing the degree of pixel change in the clipped area, a low percentage of pixels indicates limited computational burden from clipping. Preserving this region maximizes the driver's field of view. Furthermore, a clipping cancellation counting mechanism, triggered by consecutive image frames, allows for timely recovery of edge clipping when continuous jitter causes sustained high system load, thereby balancing the overall image processing load of the system and improving real-time responsiveness and image stability. By dividing the area to be cropped into four independent regions: top, bottom, left, and right, targeted retention or cropping decisions can be made based on the number of changed pixels in each region. This local trade-off approach avoids a one-size-fits-all global cropping. It can prevent the system from being under high computational load for a long time when the camera vibrates significantly, and it can also retain as much additional image information as possible in relatively stable scenes.

[0043] The present invention provides a system for collecting and processing images of a streaming media rearview mirror, such as Figure 2 As shown, including:

[0044] An image acquisition module, which includes using a camera at the rear of the vehicle to shoot and capture images from the rear of the vehicle, marking the real-time captured images as real-time frame images, and obtaining images before a preset number of frames of the real-time frame image and marking them as preset frame images;

[0045] A pixel comparison module includes calculating the SSIM of the real-time frame image and the preset frame image, and obtaining the similarity between the real-time frame image and the preset frame image according to the SSIM result, and marking the similarity as the similarity of the two frame images;

[0046] The data processing module includes setting a similarity threshold, comparing the similarity of the two frames of images with the similarity threshold, and performing geometric correction on the images according to the calibration parameters if the similarity of the two frames of images is greater than or equal to the similarity threshold; if the similarity of the two frames of images is less than the similarity threshold, executing the vibration judgment strategy;

[0047] The strategy execution module includes executing the vibration judgment strategy, including obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vectors of the pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, the image edges are cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror.

[0048] In the embodiments disclosed herein, the processes described above with reference to the flowcharts can be implemented as computer software programs. The embodiments disclosed herein include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the method illustrated in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from removable media. When the computer program is executed by a central processing unit, the functions defined in the methods of this application are performed. It should be noted that the computer-readable medium referred to herein can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof. More specific examples of computer-readable storage media can include, but are not limited to, an electrical connection having one or more wire segments, a portable computer disk, a hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. Furthermore, in this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, electrical, optical, RF, or any suitable combination thereof.

[0049] The flow charts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0050] Those skilled in the art should understand that the above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered by the scope of protection of the present application.

Claims

1. A method for processing collected images of a streaming media rearview mirror, characterized in that: The method comprises the following steps: Use the camera at the rear of the vehicle to shoot and collect images from the rear of the vehicle, mark the real-time collected images as real-time frame images, and obtain images before a preset number of frames of the real-time frame images and mark them as preset frame images; Calculate the SSIM of the real-time frame image and the preset frame image, and obtain the similarity between the real-time frame image and the preset frame image according to the SSIM result, and mark the similarity as the similarity of the two frame images; Set a similarity threshold and compare the similarity between the two frames of images with the similarity threshold. If the similarity between the two frames of images is greater than or equal to the similarity threshold, the images are geometrically corrected according to the calibration parameters. If the similarity between the two frames of images is less than the similarity threshold, the vibration judgment strategy is executed. Executing the vibration judgment strategy includes obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vectors of the pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, the image edges are cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror.

2. The method for collecting and processing images of a streaming media rearview mirror according to claim 1, characterized in that: Set a motion pixel threshold. When the pixels on all consecutive image frames have a consistent motion direction, obtain the ratio of pixels with a consistent motion direction. The specific method for obtaining the consistent motion pixel ratio is: obtain the total number of pixels with a consistent motion direction Rn and the total number of image pixels Tn, divide the two to obtain the consistent motion pixel ratio Rb=Rn÷Tn, and compare the consistent motion pixel ratio with the motion pixel threshold, and make corresponding responses based on the comparison results.

3. The method for collecting and processing images of a streaming media rearview mirror according to claim 2, characterized in that: When the consistent motion pixel ratio exceeds the motion pixel threshold, the judgment that the vehicle has experienced significant vibration and caused the camera to shake is maintained, and subsequent processing steps are executed; when the consistent motion pixel ratio does not exceed the motion pixel threshold, it is judged that the vehicle has not experienced significant vibration, the captured image is corrected normally and output to the streaming rearview mirror.

4. The method for collecting and processing images of a streaming media rearview mirror according to claim 3, characterized in that: When the consistent motion pixel ratio exceeds the motion pixel threshold, the distribution and number of pixels that do not generate consistent motion directions on continuous image frames are obtained, the pixels that do not generate consistent motion directions on continuous image frames are marked as changed pixels, and the total number of changed pixels in the area to be cropped is obtained. By dividing the total number of changed pixels in the area to be cropped by the total pixels in the area to be cropped, the proportion of changed pixels in the area to be cropped can be obtained, and the proportion is marked as the changed pixel proportion. A changed pixel proportion threshold is set, and the changed pixel proportion is compared with the changed pixel proportion threshold, and a corresponding response is made according to the comparison result.

5. The method for collecting and processing images of a streaming media rearview mirror according to claim 4, characterized in that: When the ratio of changed pixels is greater than the threshold of the ratio of changed pixels, the cropping area retention strategy is executed; when the ratio of changed pixels is less than or equal to the threshold of the ratio of changed pixels, the edge cropping of the image is canceled, the captured image is corrected normally and output to the streaming rearview mirror.

6. The method for collecting and processing images of a streaming media rearview mirror according to claim 5, characterized in that: In the process of processing subsequent continuous image frames, each time the edge cropping of the image is canceled due to the condition that the ratio of changed pixels is less than or equal to the ratio of changed pixels threshold, a cancel cropping count is performed. The cancel cropping count must be continuously triggered to keep increasing. When the trigger is interrupted, it is cleared and a cancel count threshold is set. When the cancel cropping count exceeds the cancel count threshold, the image edge is no longer cropped, and the image edge cropping is performed normally to reduce the computational burden of image processing.

7. The method for collecting and processing images of a streaming media rearview mirror according to claim 6, characterized in that: The cropping area retention strategy includes setting a threshold value for the number of changed pixels, dividing the area to be cropped into an upper area, a lower area, a left area, and a right area, obtaining the corresponding area with the least number of changed pixels, and comparing the total number of changed pixels in the area with the threshold value for the number of changed pixels. If the total number of changed pixels in the area exceeds the threshold value for the number of changed pixels, the area to be cropped is not retained, and cropping of the image edge is performed normally. If the total number of changed pixels in the area does not exceed the threshold value for the number of changed pixels, when cropping the image edge, cropping of the area is canceled, that is, the area is retained for normal image processing and output on the streaming media rearview mirror.

8. The method for collecting and processing images of a streaming media rearview mirror according to claim 7, characterized in that: Obtain the corresponding area with the second smallest number of changed pixels, sum the total number of changed pixels in the area with the total number of changed pixels in the previously retained area to obtain a cumulative number of changed pixels, compare the cumulative number of changed pixels with a threshold number of changed pixels, and if the cumulative number of changed pixels exceeds the threshold number of changed pixels, do not retain the area; If the cumulative number of changed pixels does not exceed the threshold value of the number of changed pixels, the cropping of the area is canceled. Similarly, the corresponding area with the third smallest number of changed pixels and the corresponding area with the largest number of changed pixels are continued to be obtained, and the cumulative number of changed pixels in the corresponding area is compared with the threshold value of the number of changed pixels, and corresponding judgments are made.

9. A streaming media rearview mirror image acquisition and processing system, characterized in that: The method for processing collected images according to any one of claims 1 to 8, comprising: An image acquisition module, which includes using a camera at the rear of the vehicle to shoot and capture images from the rear of the vehicle, marking the real-time captured images as real-time frame images, and obtaining images before a preset number of frames of the real-time frame image and marking them as preset frame images; A pixel comparison module includes calculating the SSIM of the real-time frame image and the preset frame image, and obtaining the similarity between the real-time frame image and the preset frame image according to the SSIM result, and marking the similarity as the similarity of the two frame images; The data processing module includes setting a similarity threshold, comparing the similarity of the two frames of images with the similarity threshold, and performing geometric correction on the images according to the calibration parameters if the similarity of the two frames of images is greater than or equal to the similarity threshold; if the similarity of the two frames of images is less than the similarity threshold, executing the vibration judgment strategy; The strategy execution module includes executing the vibration judgment strategy, including obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vectors of the pixels on all continuous image frames. When the pixels on all continuous image frames have a consistent motion direction, the image edges are cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the image acquisition processing method of a streaming media rearview mirror as described in any one of claims 1 to 8.

Images