Image processing method and system for collecting images of a streaming rearview mirror

Abstract

The application relates to the technical field of streaming rearview mirror image processing, and discloses a streaming rearview mirror image collection processing method and system, which comprises the following steps: a camera at the tail of a vehicle body is used to shoot the rear of the vehicle and collect images; the real-time collected images are marked as real-time frame images; and images before a preset number of real-time frame images are acquired and marked as preset frame images. The application calculates the similarity between the real-time frame images and the preset frame images by using SSIM, can quickly distinguish abnormal changes caused by vehicle vibration from slight changes in normal scenes, and when large-area motion blur is caused by vehicle vibration, the image edges are trimmed, so that subsequent image processing algorithms such as geometric correction and feature extraction process smaller size images, and the real-time processing performance is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of streaming rearview mirror image processing, in particular to a streaming rearview mirror image acquisition processing method and system. BACKGROUND

[0002] The streaming rearview mirror is a kind of automobile equipment that realizes the rearview function by using digital technology, which is used to replace the traditional glass rearview mirror. Its main function is to install a camera at the rear of the vehicle to collect images in real time, and convert the analog signal into digital signal. The video data collected by the camera is transmitted to the display system through the stable data network inside the vehicle. This transmission process usually requires low delay and high stability to ensure that the driver can obtain the latest picture in real time.

[0003] Since the camera, especially the wide-angle camera, is prone to barrel distortion, it is necessary to correct the distortion of the edge part by using camera calibration parameters to obtain an image with true proportions. However, in the actual lens distortion correction process, the vehicle may be affected by severe vibration during driving, making the image edge distortion more serious, and the correction of the image edge will require more calculation steps and introduce additional delay. The streaming rearview mirror requires that the image processing must be completed in a very short time, and the additional calculation burden will affect the timeliness of real-time display and driving safety. SUMMARY

[0004] In view of the problems existing in the prior art, the purpose of the present application is to provide a streaming rearview mirror image acquisition processing method and system, which can adaptively adjust the correction processing method of severely distorted images during image acquisition of the streaming rearview mirror, to ensure the timeliness of the image displayed to the driver.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme: a streaming rearview mirror image acquisition processing method, comprising the following steps:

[0006] The camera at the tail of the vehicle body is used to take pictures and collect images of the rear of the vehicle. The real-time collected images are marked as real-time frame images, and the images before a preset number of frames of the real-time frame images are obtained and marked as preset frame images;

[0007] The SSIM of the real-time frame image and the preset frame image is calculated, and the similarity of the real-time frame image and the preset frame image is obtained according to the SSIM result, and the similarity is marked as the similarity of the two frame images;

[0008] A similarity threshold is set, and the similarity of the two frames of images is compared with the similarity threshold. If the similarity of the two frames of images is greater than or equal to the similarity threshold, the images are normally geometrically corrected according to the calibration parameters. If the similarity of the two frames of images is less than the similarity threshold, a vibration judgment strategy is executed.

[0009] The vibration judgment strategy includes obtaining all continuous image frames between the real-time frame image and the preset frame image, and using an optical flow algorithm to calculate the motion vectors of the pixels on all the continuous image frames. When the pixels on all the continuous image frames have a consistent motion direction, the edges of the images are cropped to reduce the generated size of the images, and the generated images are output on the streaming rearview mirror.

[0010] In some embodiments, a motion pixel threshold is set. When the pixels on all the continuous image frames have a consistent motion direction, the proportion of the pixels having the consistent motion direction is obtained. The specific way of obtaining the consistent motion pixel proportion is: obtaining the total number of pixels having the consistent motion direction Rn and the total number of image pixels Tn, dividing the two to obtain the consistent motion pixel proportion Rb = Rn ÷ Tn, and comparing the consistent motion pixel proportion with the motion pixel threshold, and making a corresponding response according to the comparison result.

[0011] In some embodiments, when the consistent motion pixel proportion exceeds the motion pixel threshold, it is determined that the camera is shaking due to a large vibration of the vehicle, and subsequent processing steps are executed. When the consistent motion pixel proportion does not exceed the motion pixel threshold, it is determined that the vehicle does not produce a large vibration, and the captured image is normally corrected and output to the streaming rearview mirror.

[0012] In some embodiments, when the consistent motion pixel proportion exceeds the motion pixel threshold, the distribution and number of pixels not having a consistent motion direction on the continuous image frames are obtained. The pixels not having a consistent motion direction on the continuous image frames are marked as change pixels. The total number of change pixels existing in the to-be-cropped region is obtained. The proportion of change pixels in the to-be-cropped region can be obtained by dividing the total number of change pixels existing in the to-be-cropped region by the total pixels of the to-be-cropped region, and the proportion is marked as the change pixel proportion. A change pixel proportion threshold is set, and the change pixel proportion is compared with the change pixel proportion threshold. A corresponding response is made according to the comparison result.

[0013] In some embodiments, when the change pixel proportion is greater than the change pixel proportion threshold, a cropping region retention strategy is executed. When the change pixel proportion is less than or equal to the change pixel proportion threshold, the edge cropping of the image is cancelled, the captured image is normally corrected, and the corrected image is output to the streaming rearview mirror.

[0014] In some embodiments, during the process of processing subsequent continuous image frames, each time the edge clipping of the image is cancelled due to the condition that the proportion of changed pixels is less than or equal to the threshold of the proportion of changed pixels, the cancellation clipping count is performed once, the cancellation clipping count must be kept increasing under the continuity of triggering, and is cleared when the triggering is interrupted, and the cancellation times threshold is set, and when the cancellation clipping count exceeds the cancellation times threshold, the clipping of the image edge is no longer cancelled, and the image edge clipping is normally performed to reduce the calculation burden of image processing.

[0015] In some embodiments, the clipping region reservation strategy includes setting a threshold of the number of changed pixels, dividing the to-be-clipped region into an upper edge region, a lower edge region, a left edge region and a right edge region, obtaining a corresponding region with the least number of changed pixels, comparing the total number of changed pixels in the region with the threshold of the number of changed pixels, if the total number of changed pixels in the region exceeds the threshold of the number of changed pixels, the to-be-clipped region is not reserved, and the clipping of the image edge is normally performed; if the total number of changed pixels in the region does not exceed the threshold of the number of changed pixels, the clipping of the region is cancelled when the image edge is clipped, that is, the region is reserved for normal image processing and output on the streaming rearview mirror.

[0016] In some embodiments, a corresponding region with the second least number of changed pixels is obtained, the total number of changed pixels in the region is summed with the total number of changed pixels in the previously reserved region to obtain a cumulative number of changed pixels, the cumulative number of changed pixels is compared with the threshold of the number of changed pixels, if the cumulative number of changed pixels exceeds the threshold of the number of changed pixels, the region is not reserved; if the cumulative number of changed pixels does not exceed the threshold of the number of changed pixels, the clipping of the region is cancelled, and in this way, a corresponding region with the third least number of changed pixels and a corresponding region with the most number of changed pixels are continuously obtained, the cumulative number of changed pixels in the corresponding region is compared with the threshold of the number of changed pixels, and a corresponding judgment is made.

[0017] The application also provides the following technical scheme: a collection image processing system of a streaming rearview mirror, comprising:

[0018] An image collection module, which comprises a camera at the tail of a vehicle body to shoot and collect images of the rear of the vehicle, marks real-time collected images as real-time frame images, and obtains images before a preset number of frame images, and marks the images as preset frame images;

[0019] A pixel comparison module, which comprises calculating the SSIM of the real-time frame images and the preset frame images, and obtaining the similarity of the real-time frame images and the preset frame images according to the SSIM result, and marking the similarity as the similarity of the two frame images;

[0020] The data processing module includes setting a similarity threshold, comparing the similarity of two image frames with the similarity threshold, if the similarity of two image frames is greater than or equal to the similarity threshold, then the image is geometrically corrected according to the calibration parameters; if the similarity of two image frames is less than the similarity threshold, then the vibration judgment strategy is executed;

[0021] The strategy execution module includes executing the vibration judgment strategy, including obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm to calculate the motion vector of the pixels on all continuous image frames, when the pixels on all continuous image frames have consistent motion direction, then the image edge is cropped, thereby reducing the image generation size, and the generated image is output on the streaming rearview mirror.

[0022] The application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the image processing method of the streaming rearview mirror.

[0023] The technical scheme provided by the application has the following beneficial effects compared with the prior art:

[0024] Firstly, the application calculates the similarity between the real-time frame image and the preset frame image by using SSIM, which can quickly distinguish abnormal changes caused by vehicle vibration from small changes in normal scenes, and when large-area motion blur is caused by vehicle vibration, the image edge is cropped, so that subsequent image processing algorithms such as geometric correction and feature extraction process smaller images, improve real-time processing performance, and avoid the delay problem of the streaming rearview mirror caused by image processing burden.

[0025] Secondly, the application counts the change degree of pixels in the region to be cropped, when the change pixel ratio is small, it indicates that the calculation burden caused by cropping is limited, and retaining this region can expand the driver's field of view as much as possible, and the cropping cancellation mechanism triggered by continuous image frames can restore the edge cropping in time when continuous shaking causes the system to be continuously high in load, thereby realizing the burden balance of the overall image processing of the system, improving the real-time response capability and image stability.

[0026] Thirdly, the application divides the region to be cropped into four independent regions of upper, lower, left and right, and can make targeted retention or cropping decisions based on the number of change pixels in each region, which avoids the one-size-fits-all global cropping, prevents the system from being in high calculation load for a long time when the camera shakes greatly, and retains as much additional image information as possible in a relatively stable scene. BRIEF DESCRIPTION OF DRAWINGS

[0027] Figure 1A flowchart of a method for processing collected images of a streaming rearview mirror according to the present application is shown in the figure.

[0028] Figure 2 A module diagram of a system for processing collected images of a streaming rearview mirror according to the present application is shown in the figure.

[0029] Figure 3 A diagram of region division of a region to be cropped according to the present application is shown in the figure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0031] It can be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of one element can be one, and in another embodiment, the number of the element can be multiple, and the term "one" cannot be understood as a limitation on the number.

[0032] The present application provides a method for processing collected images of a streaming rearview mirror, as shown in the figure, the method comprises the following steps: Figure 1

[0033] Step one, using the camera at the tail of the vehicle body to take pictures and collect images of the rear of the vehicle, marking the real-time collected images as real-time frame images, and obtaining images before a preset number of frames of the real-time frame images, marking them as preset frame images, the preset frame can be 5 to 10 frames, for example, in the case of taking 10 frames, that is, the preset frame image is the image 10 frames before the real-time frame image;

[0034] Step two, calculating the SSIM (Structural Similarity Index) of the real-time frame image and the preset frame image, the obtained index value ranges between 0 and 1, and obtaining the similarity of the real-time frame image and the preset frame image according to the SSIM result, marking the similarity as the similarity of the two frame images, for example, SSIM is 0.9, that is, the similarity of the two frame images is 90%;

[0035] ​Step three, set the similarity threshold, compare the similarity of two frames of images with the similarity threshold, if the similarity of two frames of images is greater than or equal to the similarity threshold, it indicates that the real-time frame image has low change amplitude compared with the preset frame image, and only reasonable changes exist between the two images, then the image is corrected according to the calibration parameters, and the corrected image is displayed on the streaming rearview mirror for the driver to watch and use; if the similarity of two frames of images is less than the similarity threshold, it indicates that the real-time frame image has high change amplitude compared with the preset frame image, and the two images change obviously after a short frame number change, then the vibration judgment strategy is executed;

[0036] Step four, the vibration judgment strategy includes obtaining all continuous image frames between the real-time frame image and the preset frame image, and using the optical flow algorithm (such as Lucas-Kanade) to calculate the motion vector of the pixels on all continuous image frames. When the pixels on all continuous image frames have consistent motion direction, it is judged that the vehicle has large vibration and causes camera shaking. At this time, the motion blur produced will cause the details in the image to be lost, and the calculation pressure of real-time correction processing of the image will increase sharply. Therefore, the image edge is cropped to reduce the image generation size, and the generated image is output on the streaming rearview mirror. The range of edge cropping can be 5%-20%. After edge cropping, the image size is reduced, the processing area is narrowed, the calculation amount of subsequent geometric correction, feature extraction and other real-time image processing algorithms can be reduced, thereby improving the real-time processing performance of the system and reducing the problem of sudden increase in calculation pressure caused by vibration. In the case of large vehicle vibration and camera shaking, image edges are more prone to dynamic blur or motion distortion. In particular, the wide-angle camera configured at the tail of the vehicle, after the image it captures produces barrel distortion, the deformation distortion at the edge is the most serious. By cropping the edge, the area most seriously affected by vibration and blur can be removed, making the geometric corrected image more stable and more consistent with the actual scene, avoiding the loss of details that interfere with the driver's visual experience, and improving the timeliness and reliability of the final output image of the streaming rearview mirror.

[0037] The motion pixel threshold is set, when the pixels on all the continuous image frames have consistent motion direction, the proportion of the pixels with consistent motion direction is obtained, which is marked as consistent motion pixel ratio, and the consistent motion pixel ratio Rb=Rn÷Tn is obtained through the total number of pixels with consistent motion direction Rn and the total number of image pixels Tn, the consistent motion pixel ratio Rb is compared with the motion pixel threshold, when the consistent motion pixel ratio exceeds the motion pixel threshold, it indicates that most of the pixels on all the continuous image frames have consistent motion, which is a strong basis for the whole image shaking caused by the camera lens shaking, then the judgment of the vehicle having large vibration causing the camera shaking is maintained, and the subsequent processing steps are performed; when the consistent motion pixel ratio does not exceed the motion pixel threshold, it indicates that the proportion of the pixels with consistent motion on all the continuous image frames is not high, which also means that the pixels on the continuous image frames do not have consistent motion in the whole direction, and the pixels with consistent motion direction on the continuous image frames may be due to the vehicle, roadside objects and the like entering the camera shooting range, which is the normal shooting method, the lens of the camera does not shake due to the vibration of the vehicle, and the image does not have dynamic blur and distortion, so it is judged that the vehicle does not have large vibration, and the shooting image is normally corrected and output to the streaming rearview mirror.

[0038] For example, it is assumed that a frame of image shot by the rear camera has 2.5 million pixels, the preset frame image adopts 10 frames of images before the real-time frame image, the similarity of two frames of images is set to 0.7, and the similarity threshold is set to 0.8. Since the similarity of two frames of images is less than the similarity threshold, it indicates that the change amplitude between the real-time frame image and the preset frame image is large, the vibration judgment strategy is executed, and it is assumed that there are 1 million pixels with consistent motion direction on the continuous image frames between the preset frame image and the real-time frame image, and the consistent motion pixel ratio is 0.4. Since the consistent motion pixel ratio does not exceed the motion pixel threshold, it is judged that the vehicle does not have large vibration, and the image edge is not cropped, and the shooting image is normally corrected and output to the streaming rearview mirror.

[0039] And, after executing the above steps, when it is judged that the image edge needs to be cropped, the distribution and number of pixels that do not produce consistent motion direction on consecutive image frames are obtained, the pixels that do not produce consistent motion direction on consecutive image frames are marked as change pixels, and the total number of change pixels existing in the to-be-cropped region is obtained, the to-be-cropped region refers to the part that is cropped when the image is cropped, and since the image edge is cropped, the to-be-cropped region is a frame-shaped region. The proportion of change pixels in the to-be-cropped region can be obtained by dividing the total number of change pixels existing in the to-be-cropped region by the total number of pixels in the to-be-cropped region, and the proportion is marked as the change pixel proportion amount. A change pixel proportion threshold is set, and the change pixel proportion amount is compared with the change pixel proportion threshold. When the change pixel proportion is greater than the change pixel proportion threshold, it indicates that a large number of pixels that do not produce consistent motion direction on consecutive image frames are distributed in the to-be-cropped region. Although the image edge can be cropped to effectively reduce the computational burden of the system on image processing, the cropped region of the image should be preserved as much as possible to improve the field of view of the streaming rearview mirror presentation image. Therefore, the cropping region preservation strategy is executed. When the change pixel proportion is less than or equal to the change pixel proportion threshold, it indicates that a small number of pixels that do not produce consistent motion direction on consecutive image frames are distributed in the to-be-cropped region, and more are distributed in the image center region, i.e., the region that will not be cropped and removed. This also means that the image edge cropping reduces the image processing computational burden to a limited extent. Therefore, the image edge cropping is cancelled, the captured image is normally corrected and processed, and output to the streaming rearview mirror. It should be noted that during the processing of subsequent consecutive image frames, each time the condition that the change pixel proportion is less than or equal to the change pixel proportion threshold is triggered to cancel the image edge cropping, a cancel cropping count is performed. The cancel cropping count must be continuously triggered to increase. When the trigger is interrupted, it is cleared. A cancel count threshold is set. When the cancel cropping count exceeds the cancel count threshold, the image edge cropping is no longer cancelled, and the image edge cropping is normally executed to reduce the computational burden of image processing. This is because the image blur distortion caused by continuous camera shaking will cause the system image processing power to be consistently at a high load level. To ensure the overall image processing speed and stability, after the to-be-cropped region is continuously retained for a plurality of times, the edge cropping is restored.

[0040] The cropping region preservation strategy includes setting a change pixel number threshold, dividing the to-be-cropped region into an upper edge region, a lower edge region, a left edge region, and a right edge region (see Figure 3), the corresponding region with the least number of changed pixels is obtained, the total number of changed pixels in the region is compared with the threshold of the number of changed pixels, if the total number of changed pixels in the region exceeds the threshold of the number of changed pixels, the region to be cut is not reserved, and the cutting of the image edge is normally performed, if the total number of changed pixels in the region does not exceed the threshold of the number of changed pixels, when the image edge is cut, the cutting of the region is cancelled, that is, the region is reserved, and the image processing is normally performed on the region and output on the streaming rearview mirror;

[0041] Furthermore, the corresponding region with the second least number of changed pixels is continuously obtained, the total number of changed pixels in the region is summed with the total number of changed pixels in the previously reserved region to obtain a cumulative number of changed pixels, the cumulative number of changed pixels is compared with the threshold of the number of changed pixels, if the cumulative number of changed pixels exceeds the threshold of the number of changed pixels, the region is not reserved, if the cumulative number of changed pixels does not exceed the threshold of the number of changed pixels, the cutting of the region is cancelled, in this way, the corresponding region with the third least number of changed pixels and the corresponding region with the most number of changed pixels are continuously obtained, the cumulative number of changed pixels in the corresponding region is compared with the threshold of the number of changed pixels, and corresponding judgment is made.

[0042] In summary, the present application aims to design a streaming rearview mirror image acquisition processing method and system, aiming at the problem that the image captured by the camera is blurred and distorted due to vehicle vibration, so that the output will be delayed due to image correction processing when the image is output to the streaming rearview mirror, the present application uses SSIM to calculate the similarity between the real-time frame image and the preset frame image, which can quickly distinguish abnormal changes caused by vehicle vibration from slight changes in normal scenes, when the vehicle vibration causes large-area motion blur, the image edge is cut to make the subsequent geometric correction, feature extraction and other image processing algorithms process smaller images, improve real-time processing performance, edge cutting not only reduces the calculation amount, but also can focus on preserving the center area of the image (which usually contains more important information), reducing the interference caused by blur distortion. By statistically analyzing the pixel change degree of the region to be cut, when the change pixel ratio is small, it indicates that the calculation burden brought by cutting is reduced to a limited extent, and the region is reserved to maximize the driver's field of view, and the cutting cancellation mechanism triggered by continuous image frames can timely restore the edge cutting when continuous shaking causes the system to be continuously high in load, thereby realizing the balanced burden of the whole system image processing, improving the real-time response capability and image stability. By dividing the region to be cut into four independent regions, that is, upper, lower, left and right, targeted reservation or cutting decisions can be made based on the number of changed pixels in each region, this local selection method avoids the one-size-fits-all global cutting, which can prevent the system from being in high calculation load for a long time when the camera shakes greatly, and can maximize the preservation of additional image information in relatively stable scenes.

[0043] The application provides a collection image processing system of a streaming rearview mirror, which comprises Figure 2 as shown, comprising:

[0044] An image collection module, which comprises a camera at the tail of a vehicle body to shoot the rear of the vehicle and collect images, marks the real-time collected images as real-time frame images, and acquires images before a preset number of frame images of the real-time frame images, and marks the images as preset frame images;

[0045] A pixel comparison module, which comprises calculating the SSIM of the real-time frame images and the preset frame images, and obtaining the similarity of the real-time frame images and the preset frame images according to the SSIM result, and marking the similarity as the similarity of the two frame images;

[0046] A data processing module, which comprises setting a similarity threshold, comparing the similarity of the two frame images with the similarity threshold, if the similarity of the two frame images is greater than or equal to the similarity threshold, then performing geometric correction on the images according to the calibration parameters; if the similarity of the two frame images is less than the similarity threshold, then executing a vibration judgment strategy;

[0047] A strategy execution module, which comprises executing the vibration judgment strategy, including acquiring all continuous image frames between the real-time frame images and the preset frame images, and calculating the motion vector of the pixels on all the continuous image frames using an optical flow algorithm, when the pixels on all the continuous image frames have a consistent motion direction, then performing clipping on the edges of the images, thereby reducing the generated size of the images, and outputting the generated images on the streaming rearview mirror.

[0048] The processes described above with reference to the flowcharts can be implemented as computer software programs in accordance with embodiments of the present disclosure. Embodiments of the present disclosure include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising program code for performing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication section, and / or installed from a detachable medium. When the computer program is executed by a central processing unit, the above-described functions defined in the methods of the present application are performed. It should be noted that the computer readable medium of the present application can be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium may, for example, but not limited to, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus or device. In the present application, the computer readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, in which a computer readable program code is carried. Such a propagated data signal can take many forms, including but not limited to, an electromagnetic signal, an optical signal or any suitable combination of the above. The computer readable signal medium can also be any computer readable medium that can send, propagate or transfer a program for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the computer readable medium can be transmitted by any suitable medium, including but not limited to, wireless, wire, optical cable, RF or any suitable combination of the above.

[0049] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0050] Those skilled in the art should understand that the above description is only a specific embodiment of this application, but the protection scope of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the protection scope of this application.

Claims

1. A method for processing captured images of a streaming rearview mirror, characterized in that, The method comprises the following steps: The camera at the tail of the vehicle body is used to shoot the rear of the vehicle and collect images, the real-time collected images are marked as real-time frame images, and the images before a preset number of frame images are obtained and marked as preset frame images; The SSIM of the real-time frame images and the preset frame images is calculated, and the similarity of the real-time frame images and the preset frame images is obtained according to the SSIM result, and the similarity is marked as the similarity of the two frame images; The similarity threshold is set, and the similarity of the two frame images is compared with the similarity threshold, if the similarity of the two frame images is greater than or equal to the similarity threshold, the image is geometrically corrected according to the calibration parameters; if the similarity of the two frame images is less than the similarity threshold, the vibration judgment strategy is executed; The vibration judgment strategy comprises obtaining all the continuous image frames between the real-time frame images and the preset frame images, and using the optical flow algorithm to calculate the motion vector of the pixels on all the continuous image frames, when the pixels on all the continuous image frames have a consistent motion direction, the image edge is cropped to reduce the generated size of the image, and the generated image is output on the streaming media rearview mirror.

2. The method of claim 1, wherein the method further comprises: The motion pixel threshold is set, when the pixels on all the continuous image frames have a consistent motion direction, the proportion of the pixels with a consistent motion direction is obtained, and the specific way of obtaining the consistent motion pixel ratio is: obtaining the total number of pixels with a consistent motion direction Rn and the total number of image pixels Tn, dividing the two to obtain the consistent motion pixel ratio Rb=Rn÷Tn, and comparing the consistent motion pixel ratio with the motion pixel threshold, and making a corresponding response according to the comparison result.

3. The method of claim 2, wherein the method further comprises: When the consistent motion pixel ratio exceeds the motion pixel threshold, it is judged that the camera shakes due to the large vibration of the vehicle, and the subsequent processing steps are executed; when the consistent motion pixel ratio does not exceed the motion pixel threshold, it is judged that the vehicle does not produce large vibration, and the photographed image is normally corrected and output to the streaming media rearview mirror.

4. The method of claim 3, wherein the method further comprises: When the consistent motion pixel ratio exceeds the motion pixel threshold, the pixel distribution and quantity that do not have a consistent motion direction on the continuous image frames are obtained, the pixels that do not have a consistent motion direction on the continuous image frames are marked as change pixels, the total number of change pixels existing in the to-be-cropped region is obtained, the proportion of change pixels in the to-be-cropped region is obtained by dividing the total number of change pixels existing in the to-be-cropped region by the total pixels of the to-be-cropped region, and the proportion is marked as the change pixel proportion, the change pixel proportion threshold is set, and the change pixel proportion is compared with the change pixel proportion threshold, and a corresponding response is made according to the comparison result.

5. The method of claim 4, wherein the method further comprises: When the change pixel proportion is greater than the change pixel proportion threshold, the cropping region retention strategy is executed, and the cropping region retention strategy is used to retain part of the to-be-cropped region to reduce the area of the cropped image region; When the change pixel proportion is less than or equal to the change pixel proportion threshold, the edge of the image is not cropped, the photographed image is normally corrected and output to the streaming media rearview mirror.

6. The method of claim 5, wherein the method further comprises: In the process of processing subsequent continuous image frames, each time the edge clipping of the image is cancelled due to the condition that the proportion of changed pixels is less than or equal to the threshold of the proportion of changed pixels, a clipping cancellation count is performed, the clipping cancellation count must be increased continuously, and is cleared when the trigger is interrupted, and a cancellation times threshold is set, and when the clipping cancellation count exceeds the cancellation times threshold, the clipping of the image edge is no longer cancelled, and the image edge clipping is normally executed to reduce the calculation burden of image processing.

7. The method of claim 6, wherein the method further comprises: The clipping region retention strategy includes setting a changed pixel number threshold, dividing the to-be-clipped region into an upper edge region, a lower edge region, a left edge region and a right edge region, obtaining a corresponding region with the least number of changed pixels, comparing the total number of changed pixels in the region with the changed pixel number threshold, if the total number of changed pixels in the region exceeds the changed pixel number threshold, the to-be-clipped region is not retained, and the clipping of the image edge is normally executed; if the total number of changed pixels in the region does not exceed the changed pixel number threshold, the clipping of the region is cancelled when the image edge is clipped, that is, the region is retained and normal image processing is performed on the region and output on the streaming rearview mirror.

8. The method of claim 7, wherein the method further comprises: A corresponding region with the second least number of changed pixels is obtained, the total number of changed pixels in the region is summed with the total number of changed pixels in the previously retained region to obtain a changed pixel cumulative number, the changed pixel cumulative number is compared with the changed pixel number threshold, and if the changed pixel cumulative number exceeds the changed pixel number threshold, the region is not retained; If the changed pixel cumulative number does not exceed the changed pixel number threshold, the clipping of the region is cancelled, and in this way, a corresponding region with the third least number of changed pixels and a corresponding region with the most number of changed pixels are continuously obtained, the changed pixel cumulative number in the corresponding region is compared with the changed pixel number threshold, and a corresponding judgment is made.

9. An image processing system for a streaming rearview mirror, characterized by The system includes an image acquisition module, a pixel comparison module, a data processing module and a strategy execution module, and the above modules are used to execute and implement the acquisition image processing method in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the acquisition image processing method of the streaming rearview mirror in any one of claims 1-8.

Images