An image processing method, device and system

Abstract

The embodiment of the present application provides an image processing method, device and system, and relates to the technical field of computers, and comprises the following steps: obtaining an image to be processed collected by an image sensor, and obtaining motion data of a target time period collected by a motion sensor, wherein the target time period is a time period during which the image sensor collects the image to be processed; determining a plurality of target row regions from the image to be processed; for each target row region, correcting the target row region by using a correction mode corresponding to the target row region to obtain a processed image, wherein the correction mode corresponding to each target row region is determined based on time domain motion data of the motion data collected during the target row region, and the correction mode corresponding to different target row regions is different in the case that the time domain motion data corresponding to different target row regions is different. The image processing method provided by the embodiment of the present application can improve the image quality.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an image processing method, device and system. Background Technology

[0002] When a camera captures an image, camera shake can easily cause the captured image to be blurry. For example, assuming the camera mentioned above uses a rolling shutter, after the initial exposure, the camera exposes line by line until all pixels are exposed. If the camera shakes during this exposure process, the exposure result may show tilting, swaying, or partial exposure, thus producing a rolling shutter effect and causing the captured image to be blurry.

[0003] Therefore, there is an urgent need for an image processing solution to overcome the rolling shutter effect and improve image quality. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, device, and system to improve image quality. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of this application provide an image processing method, the method comprising:

[0006] The image to be processed is acquired by an image sensor, and motion data for a target time period is acquired by a motion sensor, wherein the target time period is the time period during which the image sensor acquires the image to be processed.

[0007] Multiple target row regions are determined from the image to be processed, wherein the pixels included in different row regions belong to different row pixels;

[0008] For each target row region, the target row region is corrected using the correction method corresponding to that target row region to obtain the processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of that target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

[0009] In one embodiment of this application, the step of correcting each target row region using the correction method corresponding to that target row region to obtain a processed image includes:

[0010] The jitter information of the image sensor in the target time period is determined based on the motion data;

[0011] For each target row region, based on the temporal jitter information of the image sensor during the acquisition of the target row region from the jitter information, the correction method corresponding to the target row region is determined, and the target row region is corrected using the determined correction method to obtain the processed image.

[0012] In one embodiment of this application, the step of determining a correction method corresponding to each target row region based on the temporal jitter information of the image sensor during the acquisition of the target row region from the jitter information, and correcting the target row region using the determined correction method to obtain a processed image includes:

[0013] Obtain the regional acquisition time for each target row region;

[0014] For each target row region, the temporal jitter information of the image sensor during the regional acquisition time of the target row region is determined from the jitter information. The target mapping relationship corresponding to the temporal jitter information is found from the first correspondence between jitter information and mapping relationship, which serves as the correction method for the target row region. Each pixel in the target row region is mapped according to the target mapping relationship to achieve correction of the target row region and obtain the processed image. The mapping relationship is the positional correspondence between the pixels in each row region of the image to be processed and the pixels in each row region of the processed image.

[0015] In one embodiment of this application, determining multiple target row regions from the image to be processed includes:

[0016] Based on the temporal motion data collected during the period of each candidate row region in the motion data, offset information of the image content shift in each candidate row region is obtained;

[0017] For each candidate row region, the target start position and target end position are determined by using the offset information of the image content shift in the candidate row region and the preset target region length. The region between the target start position and target end position is determined from the candidate row region as the target row region.

[0018] In one embodiment of this application, obtaining offset information indicating the shift in image content within each candidate row region based on temporal motion data collected during the acquisition of each candidate row region from the motion data includes:

[0019] The jitter information of the image sensor in the target time period is determined based on the motion data;

[0020] Based on the temporal jitter information of the image sensor during the acquisition of each candidate row region from the jitter information, offset information of the image content shifting in each candidate row region is obtained.

[0021] In one embodiment of this application, obtaining offset information indicating that the image content in each candidate row region has shifted based on the temporal jitter information of the image sensor during the acquisition of each candidate row region from the jitter information includes:

[0022] For each candidate row region, the region acquisition time of the image sensor in acquiring the candidate row region is obtained. The temporal jitter information of the image sensor during the region acquisition time is obtained from the jitter information. The offset information corresponding to the temporal jitter information is found from the second correspondence between jitter information and offset information, and is used as the offset information of the image content shift in the candidate row region.

[0023] In one embodiment of this application, the step of correcting each target row region using the correction method corresponding to that target row region to obtain a processed image includes:

[0024] For each target row region, based on the offset information corresponding to that target row region, a correction method is determined to align that target row region with other target row regions. The determined correction method is then used to correct the target row region, resulting in the processed image.

[0025] In one embodiment of this application, after the step of determining multiple target row regions from the image to be processed, the method further includes:

[0026] Based on the temporal motion data collected during the period of collecting each target row region from the motion data, target row regions whose temporal motion data during the collection period meets the preset conditions of intense motion are filtered.

[0027] Secondly, embodiments of this application provide an image acquisition device, which includes an image sensor, a motion sensor, and an image processor, wherein:

[0028] The image sensor is used to: acquire an image to be processed and send the image to be processed to the image processor;

[0029] The motion sensor is used to: collect motion data over a target time period and send the motion data to the image processor, wherein the target time period is the time period during which the image sensor collects the image to be processed;

[0030] The image processor is configured to: acquire an image to be processed from an image sensor and acquire motion data for a target time period from a motion sensor; determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data acquired during the period of acquisition of the target row region in the motion data, and the correction methods corresponding to different target row regions are different when the temporal motion data corresponding to different target row regions are different.

[0031] Thirdly, embodiments of this application provide an image processing system, the system comprising an image sensor, a motion sensor, and a processor, wherein:

[0032] The image sensor is used to: acquire an image to be processed and send the image to be processed to the processor;

[0033] The motion sensor is used to: collect motion data over a target time period and send the motion data to the processor, wherein the target time period is the time period during which the image sensor collects the image to be processed;

[0034] The processor is configured to: obtain an image to be processed acquired by an image sensor and obtain motion data for a target time period acquired by a motion sensor; determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data acquired during the period of the target row region in the motion data, and the correction method corresponding to different target row regions is different when the temporal motion data corresponding to different target row regions is different.

[0035] Fourthly, embodiments of this application provide an image processing apparatus, the apparatus comprising:

[0036] The data acquisition module is used to acquire the image to be processed collected by the image sensor and to acquire motion data of a target time period collected by the motion sensor, wherein the target time period is the time period during which the image sensor collects the image to be processed;

[0037] The region determination module is used to determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels;

[0038] The image processing module is used to correct each target row region using the correction method corresponding to that target row region to obtain a processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of that target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

[0039] Fifthly, embodiments of this application provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0040] Memory, used to store computer programs;

[0041] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0042] In a sixth aspect, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0043] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the image processing methods described above.

[0044] Beneficial effects of the embodiments in this application:

[0045] The image processing method provided in this application embodiment can obtain an image to be processed acquired by an image sensor and motion data of a target time period acquired by a motion sensor. The target time period is the time during which the image sensor acquires the image to be processed. Multiple target row regions are determined from the image to be processed, where the pixels included in different row regions belong to different rows. For each target row region, a correction method corresponding to that target row region is used to correct it, resulting in a processed image. The correction method for each target row region is determined based on the temporal motion data acquired during the acquisition of that target row region. Different correction methods correspond to different target row regions when the temporal motion data is different. Since image sensors typically use a rolling shutter for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the tilt, sway, or partial exposure of pixels in different rows are different. The motion data collected by the motion sensor reflects the motion of the image sensor. Therefore, different target row regions are corrected separately based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for correcting different target row regions are also different. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image being processed. Therefore, it is evident that the image processing scheme provided in this application can improve image quality. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0047] Figure 1 A schematic flowchart of an image processing method provided in an embodiment of this application;

[0048] Figure 2 A schematic diagram of a target row region provided in an embodiment of this application;

[0049] Figure 3 A schematic diagram of a processed image provided in an embodiment of this application;

[0050] Figure 4 A flowchart illustrating another image processing method provided in an embodiment of this application;

[0051] Figure 5A schematic diagram illustrating an image processing procedure provided in an embodiment of this application;

[0052] Figure 6 This is a schematic diagram of the structure of an image acquisition device provided in an embodiment of this application;

[0053] Figure 7 This is a schematic diagram of the structure of an image processing system provided in an embodiment of this application;

[0054] Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0055] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art based on this application are within the scope of protection of this application.

[0057] To improve image quality, embodiments of this application provide an image processing method, device, and system, which will be described in detail below.

[0058] This application provides an image processing method that can be applied to electronic devices such as image acquisition devices, computers, and servers. The method includes:

[0059] The image to be processed is acquired by the image sensor, and the motion data of the target time period is acquired by the motion sensor. The target time period is the time period during which the image to be processed is acquired by the image sensor.

[0060] Multiple target row regions are determined from the image to be processed, wherein the pixels included in different row regions belong to different row pixels;

[0061] For each target row region, the target row region is corrected using the correction method corresponding to that target row region to obtain the processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of that target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

[0062] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0063] The above image processing methods will be described in detail below.

[0064] See Figure 1 , Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. The method includes the following steps S101-S103:

[0065] S101, obtain the image to be processed acquired by the image sensor, and obtain the motion data for the target time period acquired by the motion sensor.

[0066] The aforementioned motion data may include linear acceleration, velocity, angular velocity, angular acceleration, etc.

[0067] The motion data collected by motion sensors can reflect the motion of image sensors. These motion sensors can be gyroscopes, etc., which can be used to collect motion data such as linear acceleration and angular velocity.

[0068] The target time period is the time during which the image sensor acquires the image to be processed, specifically the time from the start of exposure to the end of exposure when the image sensor acquires the image. For example, assuming that the image sensor starts exposure at 5 milliseconds and ends exposure at 35 milliseconds when acquiring the image, then the time period from 5 to 35 milliseconds can be used as the target time period.

[0069] Specifically, an image acquired by an image sensor can be obtained as an image to be processed; and the time period during which the image sensor acquired the image to be processed can be determined as a target time period, and motion data within the target time period can be selected from the motion data acquired by a motion sensor.

[0070] In one embodiment of this application, when obtaining an image to be processed, the image signal collected by the image sensor can be obtained first, and the image signal can be analyzed to obtain the analyzed image as the image to be processed.

[0071] In one embodiment of this application, after obtaining the image to be processed, the image to be processed can be preprocessed, for example, the preprocessing may include noise reduction processing, image enhancement processing, color restoration processing, etc.

[0072] In one embodiment of this application, after obtaining motion data, the motion data can be preprocessed. For example, the preprocessing may include anti-interference processing, noise reduction processing, etc.

[0073] In one embodiment of this application, the relative positions between the motion sensor and the image sensor are fixed. Specifically, the motion sensor and the image sensor can be directly fixedly connected, or they can be fixedly connected through connecting devices such as brackets, bases, hinges, and connecting rods. This ensures that the motion sensor and the image sensor will not move relative to each other, and that the motion data collected by the motion sensor can directly reflect the motion of the image sensor.

[0074] S102, determine multiple target row regions from the image to be processed.

[0075] In this context, the pixels included in different row regions belong to different row pixels, which refers to pixels located in the same row.

[0076] Specifically, image sensors typically use a rolling shutter, meaning that the image sensor exposes pixels row by row. The exposure time interval between pixels in the same row is short. Since pixels in the same row are considered to have the same tilt, sway, or partial exposure due to image sensor jitter, multiple row regions can be determined from the image to be processed, based on row pixels, as the target row region.

[0077] The height of each target row region can be 1 pixel, 3 pixels, 8 pixels, etc. The heights of different target row regions can be the same or different, and this application embodiment does not limit this.

[0078] The length of each target row region can be a preset length, which is less than or equal to the width of the image to be processed. For example, assuming the size of the image to be processed is 1920*1080, it means that the width of the image to be processed is 1920 pixels, then the above-mentioned preset length is less than or equal to 1920.

[0079] In one embodiment of this application, after determining the row region, the target row region can be filtered based on the temporal motion data collected during the collection of each target row region in the motion data, and the target row region whose temporal motion data during the collection period meets the preset intense motion conditions.

[0080] The aforementioned severe motion condition is used to determine whether the image sensor is jittering severely based on motion data. For example, the severe motion condition may include one or more of the following conditions: acceleration greater than a preset acceleration threshold, angular acceleration greater than a preset angular acceleration threshold, angular velocity greater than a preset angular velocity threshold, etc.

[0081] Specifically, after determining each target row region, the time period during which the image sensor collects each target row region can also be obtained. Then, motion data collected within the above time period can be found from the above motion data as temporal motion data. Then, it is determined whether the temporal motion data meets the above-mentioned severe motion condition. If it does, it means that the image sensor jittered severely during the collection of the target row region, and the image quality of the target row region may be poor. Therefore, the target row region can be directly filtered out.

[0082] If no, it means that the image sensor did not shake violently during the acquisition of the target row area, and the image quality of the target row area may be high, so the target row area can be preserved.

[0083] The above scheme can filter out target row regions with poor imaging quality in the image to be processed, ensuring that the quality of the row regions retained in the processed image is high, thereby improving the quality of the processed image.

[0084] S103, for each target row region, the target row region is corrected using the correction method corresponding to that target row region to obtain the processed image.

[0085] The correction method for each target row region is determined based on the temporal motion data collected during the period of collecting the target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

[0086] The temporal motion data corresponding to each target row region is: the data collected by the motion sensor during the collection period of that target row region.

[0087] Specifically, for each target row region, the time period during which the image sensor acquired that target row region can be obtained. Then, motion data acquired within that time period is retrieved from the aforementioned motion data and used as temporal motion data. This temporal motion data reflects the jitter of the image sensor during the acquisition of that target row region. Based on the jitter of the image sensor, it is possible to determine whether the acquired image exhibits tilt, swaying, or partial exposure. Therefore, based on the aforementioned temporal motion data, a correction method for correcting that target row region can be determined. The target row region can then be corrected according to the determined correction method to overcome the rolling shutter effect produced by that target row region. For different target row regions with different temporal motion data, the correction methods will also differ due to the varying jitter between these target row regions. After correcting each target row region, a processed image can be obtained.

[0088] The aforementioned methods for correcting the target row region include displacement correction and / or rotation correction. Displacement correction involves translating the target row region based on temporal motion data to overcome the rolling shutter effect caused by image sensor translational jitter, thus correcting the target row region. Rotation correction involves rotating the target row region based on temporal motion data to overcome the rolling shutter effect caused by image sensor rotational jitter, thus correcting the target row region. These methods for correcting the target row region will be described in detail later.

[0089] In one embodiment of this application, after obtaining the processed image, the processed image may be stretched and / or cropped to obtain an image that meets a preset resolution.

[0090] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0091] In one embodiment of this application, when correcting the target row region in step S103, the jitter information of the image sensor in the target time period can be determined based on motion data; for each target row region, the correction method corresponding to the target row region is determined based on the temporal jitter information of the image sensor collected during the target row region in the jitter information, and the target row region is corrected using the determined correction method to obtain the processed image.

[0092] The aforementioned jitter information may include at least one of the following: jitter direction, jitter distance, rotation angle, jitter speed, etc.

[0093] Specifically, based on the motion data of the target time period, the jitter information of the motion sensor during the target time period can be calculated, and the jitter information of the motion sensor can reflect the jitter of the image sensor.

[0094] For each target row region, the time period during which the image sensor acquired that target row region can be obtained. Then, the jitter information acquired within the aforementioned time period is searched from the jitter information and used as temporal jitter information. This temporal jitter information reflects the jitter situation of the image sensor during the acquisition of the target row region. Based on the jitter situation of the image sensor, the tilt, sway, or partial exposure of the acquired image can be determined. Therefore, using the aforementioned temporal jitter information, the corresponding correction method for the target row region can be determined. Then, the determined correction method is used to correct the target row region, thereby overcoming the rolling shutter effect produced by the target row region. After correcting each target row region, the processed image can be obtained.

[0095] In one embodiment of this application, when determining jitter information based on motion data, the motion data can be integrated, and the integrated results can be accumulated to obtain the jitter information. For example, assuming the motion data is acceleration, integrating the acceleration can yield the distance of motion, which can be used as jitter information.

[0096] In one embodiment of this application, the regional acquisition time of each target row region can be obtained; for each target row region, the temporal jitter information of the image sensor during the regional acquisition time of the target row region is determined from the jitter information; the target mapping relationship corresponding to the temporal jitter information is found from the first correspondence between the jitter information and the mapping relationship, which is used as the correction method corresponding to the target row region; each pixel in the target row region is mapped according to the target mapping relationship to realize the correction of the target row region and obtain the processed image.

[0097] The mapping relationship is the positional correspondence between the pixels in each row of the image to be processed and the pixels in each row of the processed image.

[0098] Specifically, for each target row region, the acquisition time of the target row region can be collected based on the image sensor. Then, the jitter information whose acquisition time is located within the acquisition time of the target row region can be found from the jitter information mentioned above. This jitter information is used as temporal jitter information. This temporal jitter information can reflect the jitter situation of the image sensor during the acquisition of the target row region. Then, the target mapping relationship corresponding to the temporal jitter information can be found from the preset first correspondence relationship. This is used as the correction method corresponding to the target row region. This target mapping relationship can reflect the positional correspondence between the pixels of the target row region in the image to be processed and the pixels of the row region in the processed image. According to this target mapping relationship, each pixel in the target row region can be mapped to the processed image to achieve the correction of the target row region.

[0099] In one embodiment of this application, for step S102 above, when determining multiple target row regions from the image to be processed, the offset information of the image content shift in each candidate row region can be obtained based on the temporal motion data collected during the period of each candidate row region in the motion data; for each candidate row region, the target start position and target end position are determined by using the offset information of the image content shift in the candidate row region and the preset target region length, and the region between the target start position and the target end position is determined from the candidate row region as the target row region.

[0100] In this model, the pixels included in different candidate row regions belong to different row pixels. The height of each candidate row region is the same as the height of the target row region, and its length can be the length of the image to be processed. Each candidate row region can be a pre-defined row region.

[0101] The offset information mentioned above reflects the offset of the boundary of the target row region relative to the boundary of the image to be processed.

[0102] The length of the target region is less than the length of the image to be processed. For example, assuming the resolution of the image to be processed is 1920×1080, the length of the image to be processed is 1920, so the length of the target region is less than 1920.

[0103] Specifically, for each candidate row region, the time period during which the image sensor acquired the target row region can be obtained. Then, motion data acquired within the aforementioned time period can be found from the motion data and used as temporal motion data. This temporal motion data can reflect the jitter of the image sensor during the acquisition of the target row region. Based on the jitter of the image sensor, the tilt, sway, or partial exposure of the acquired image can be determined. Therefore, using the aforementioned temporal motion data, offset information of the image content shift in the candidate row region can be obtained.

[0104] Using the aforementioned offset information, the offset amount of the image content in the candidate row region can be determined. Based on this offset amount and the preset target region length, the target start position and target end position of the target row region to be determined can be obtained. The region between the target start position and the target end position is determined from the candidate row region as the target row region.

[0105] See Figure 2 , Figure 2 This is a schematic diagram of a target row region provided in an embodiment of this application. The image to be processed is referred to as the original image. Based on offset information and a preset target region length, the target start position and target end position of the target row region to be determined can be obtained, thereby determining the target row region. Figure 2 It can be seen that, based on the target start position (pix-start-0, line-start-0) and the target end position (pix-stop-0, line-stop-0), the first target row region in the image to be processed can be determined, which is the 0th region of the original image; based on the target start position (pix-start-1, line-start-1) and the target end position (pix-stop-1, line-stop-1), the second target row region in the image to be processed can be determined, which is the 1st region of the original image; based on the target start position (pix-start-n, line-start-n) and the target end position (pix-stop-n, line-stop-n), the (n+1)th target row region in the image to be processed can be determined, which is the nth region of the original image.

[0106] In one embodiment of this application, the jitter information of the image sensor during the target time period can be determined based on motion data; and the temporal jitter information of the image sensor during each candidate row region can be collected based on the jitter information to obtain the offset information of the image content shifting in each candidate row region.

[0107] Specifically, for each candidate row region, the region acquisition time of the candidate row region can be collected based on the image sensor. Then, the jitter information whose acquisition time is located in the region acquisition time can be found from the jitter information mentioned above. This jitter information is used as time-domain jitter information. This time-domain jitter information can reflect the jitter situation of the image sensor during the acquisition of the target row region. Based on this jitter situation, the offset information of the image content shift in the candidate row region can be obtained.

[0108] In one embodiment of this application, for each candidate row region, the region acquisition time of the image sensor acquiring the candidate row region can be obtained, the temporal jitter information of the image sensor during the region acquisition time can be obtained from the jitter information, and the offset information corresponding to the temporal jitter information can be found from the second correspondence between the jitter information and the offset information, which is used as the offset information of the image content in the candidate row region being offset.

[0109] Specifically, for each candidate row region, after obtaining the temporal jitter information corresponding to the candidate row region, the offset information corresponding to the temporal jitter information can be found from the preset second correspondence, and used as the offset information for the image content shift in the candidate row region.

[0110] Based on the above scheme, when correcting the target row region, for each target row region, a correction method can be determined based on the offset information corresponding to the target row region to align the target row region with other target row regions. The target row region is then corrected using the determined correction method to obtain the processed image.

[0111] The offset information corresponding to each target row region is: the offset information of the image content in the candidate row region corresponding to the target row region, which is determined based on the temporal motion data.

[0112] Specifically, since the length of each of the target row regions is the same, which is the target region length, and the target row regions are all regions determined after considering the offset information of the image content, the alignment method for aligning each target row region, such as translation, can be directly determined based on the offset information as the correction method. Then, the determined correction method is used to align each of the target row regions row by row to achieve the correction of each target row region and obtain the processed image.

[0113] In one embodiment of this application, the determined target row regions can be mapped onto a target image. During the mapping process, the order of the target row regions in the vertical direction is kept unchanged, and the target row regions are aligned in the horizontal direction to obtain a target image, which can be used as a processed image.

[0114] See Figure 3 , Figure 3 This is a schematic diagram of a processed image provided in an embodiment of this application. After obtaining the above... Figure 2After defining each target row region, each target row region can be mapped onto the target image, keeping the vertical order of each target row region unchanged and keeping the horizontal alignment of each target row region unchanged. Specifically, the 0th region of the original image can be mapped to the 0th region of the target image, the 1st region of the original image can be mapped to the 1st region of the target image, and the nth region of the original image can be mapped to the nth region of the target image, finally obtaining the target image as the processed image.

[0115] See Figure 4 , Figure 4 This is a flowchart illustrating another image processing method provided in an embodiment of this application, which includes the following steps S401-S406:

[0116] S401, obtain the image to be processed acquired by the image sensor, and obtain the motion data for the target time period acquired by the motion sensor.

[0117] The motion sensor and the image sensor are fixedly connected, and the target time period is the time period during which the image sensor acquires the image to be processed.

[0118] S402, determine the jitter information of the image sensor in the target time period based on motion data.

[0119] S403, for each candidate row region, obtain the region acquisition time of the image sensor acquiring the candidate row region, obtain the temporal jitter information of the image sensor during the region acquisition time from the jitter information, and find the offset information corresponding to the temporal jitter information from the correspondence between jitter information and offset information, as the offset information of the image content shift in the candidate row region.

[0120] S404, for each candidate row region, using the offset information of the image content shift in the candidate row region and the preset target region length, determine the target start position and the target end position, and determine the region between the target start position and the target end position from the candidate row region as the target row region.

[0121] S405, map the determined target row regions onto the target image to obtain the target image as the processed image.

[0122] During the mapping process, the vertical order of each target row region is maintained, and the horizontal alignment of each target row region is also maintained. There is a correspondence between the starting position of each target row region in the image to be processed and its starting position in the processed image, and a correspondence between the ending position of each target row region in the image to be processed and its ending position in the processed image.

[0123] S406, stretch the processed image so that the resolution of the stretched image is the same as the resolution of the image to be processed.

[0124] See Figure 5 , Figure 5 This is a schematic diagram of an image processing procedure provided in an embodiment of this application. Multiple target row regions can be determined from the image to be processed, namely region 0, region 1... region 39. The above target row regions are neatly mapped onto the target image to obtain the processed image. Finally, the processed image is stretched to restore the resolution of the processed image.

[0125] In traditional image processing solutions, the rolling shutter effect caused by camera shake is usually eliminated by cropping or stretching the entire image. However, in practical applications, different rows of an image may have varying degrees of tilt, sway, or partial exposure. Taking CMOS (Complementary Metal Oxide Semiconductor) image sensors as an example, they typically use a rolling shutter exposure method, where the exposure time for each row of the image differs. When the image sensor itself shakes, this can lead to the rolling shutter effect. Especially when the device shakes at a high frequency, the shaking within a single frame may be inconsistent between the upper and lower parts. Continuing to process the entire image will not achieve a satisfactory image stabilization effect.

[0126] The inventors discovered that, to address the aforementioned issues, images can be processed by dividing them into regions, with each row region employing a different correction method. Specifically, firstly, motion data is acquired during image acquisition using a motion sensor. This motion data reflects the image sensor's jitter during the acquisition of each row region. Then, based on the exposure parameters of the CMOS image sensor, the acquisition time for each row region can be determined. Finally, by combining the acquisition time of each row region, the jitter condition of each row region is determined. Based on this jitter condition, each target row region is extracted from the image and mapped to form a new, repaired image.

[0127] In one embodiment of this application, this solution is also compatible with traditional image processing solutions. Specifically, it can be determined whether the motion data collected by the motion sensor meets the preset conditions for violent motion. If the above conditions are not met, it indicates that the rolling shutter effect of the image collected by the image sensor is not obvious. Therefore, traditional image processing solutions can be used to reduce the consumption of computing resources.

[0128] When the motion data collected by the motion sensor meets the preset conditions of intense motion, it indicates that the image collected by the image sensor has a more obvious rolling shutter effect. Therefore, the image processing scheme provided in this solution can be used to improve the image quality.

[0129] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0130] Corresponding to the image processing method described above, this application also provides an image acquisition device, which will be described in detail below.

[0131] See Figure 6 , Figure 6 This is a schematic diagram of the structure of an image acquisition device provided in an embodiment of this application. The image acquisition device includes an image sensor 601, a motion sensor 602, and an image processor 603, wherein:

[0132] Image sensor 601 is used to: acquire an image to be processed and send the image to be processed to image processor 603;

[0133] Motion sensor 602 is used to: collect motion data for a target time period and send the motion data to image processor 603, wherein the target time period is the time period during which image sensor 601 collects the image to be processed;

[0134] Image processor 603 is configured to: acquire an image to be processed collected by image sensor 601 and acquire motion data of a target time period collected by motion sensor 602; determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data collected during the period of the target row region in the motion data, and the correction method corresponding to different target row regions is different when the temporal motion data corresponding to different target row regions is different.

[0135] In one embodiment of this application, the image acquisition device may further include an image preprocessor, wherein:

[0136] Image sensors are used to: acquire images to be processed and send the images to be processed to an image preprocessor;

[0137] An image preprocessor is used to preprocess the image to be processed and then send the preprocessed image to the image processor.

[0138] In one embodiment of this application, the image acquisition device may further include a first memory, wherein:

[0139] An image preprocessor is used to send the preprocessed image to a first memory;

[0140] An image processor for retrieving an image to be processed from a first memory.

[0141] In one embodiment of this application, the image acquisition device may further include a motion data preprocessor, wherein:

[0142] Motion sensors are used to: collect motion data over a target time period and send the motion data to a motion data preprocessor;

[0143] The motion data preprocessor is used to preprocess motion data and then send the preprocessed motion data to the image processor.

[0144] In one embodiment of this application, the image acquisition device may further include a second memory, wherein:

[0145] A motion data preprocessor is used to send preprocessed motion data to a second memory;

[0146] An image processor for acquiring motion data from a second memory.

[0147] In addition, in one embodiment of this application, the motion data preprocessor can also directly send the preprocessed motion data to the image processor.

[0148] In one embodiment of this application, the image acquisition device may further include a third memory, wherein:

[0149] An image processor is used to store the processed image into the third memory.

[0150] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0151] Corresponding to the image processing method described above, this application also provides an image processing system, which will be described in detail below.

[0152] See Figure 7 , Figure 7 This is a schematic diagram of an image processing system provided in an embodiment of this application. The system includes an image sensor, a motion sensor, and a processor, wherein:

[0153] The image sensor is used to: acquire an image to be processed and send the image to be processed to the processor;

[0154] The motion sensor is used to: collect motion data over a target time period and send the motion data to the processor, wherein the target time period is the time period during which the image sensor collects the image to be processed;

[0155] The processor is configured to: obtain an image to be processed acquired by an image sensor and obtain motion data for a target time period acquired by a motion sensor; determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data acquired during the period of the target row region in the motion data, and the correction method corresponding to different target row regions is different when the temporal motion data corresponding to different target row regions is different.

[0156] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0157] Corresponding to the image processing method described above, this application also provides an image processing apparatus, which will be described in detail below.

[0158] See Figure 8 , Figure 8 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. The apparatus includes:

[0159] The data acquisition module 801 is used to acquire the image to be processed collected by the image sensor and to acquire motion data of a target time period collected by the motion sensor, wherein the target time period is the time period during which the image sensor collects the image to be processed.

[0160] The region determination module 802 is used to determine multiple target row regions from the image to be processed, wherein the pixels included in different row regions belong to different row pixels;

[0161] Image processing module 803 is used to correct each target row region using the correction method corresponding to that target row region to obtain a processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of the target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

[0162] In one embodiment of this application, the image processing module 803 includes:

[0163] The first jitter information acquisition unit is used to determine the jitter information of the image sensor in the target time period based on the motion data;

[0164] The first image processing unit is configured to, for each target row region, determine the correction method corresponding to the target row region based on the temporal jitter information of the image sensor during the acquisition of the target row region in the jitter information, and use the determined correction method to correct the target row region to obtain the processed image.

[0165] In one embodiment of this application, the first image processing unit is specifically used for:

[0166] Obtain the regional acquisition time for each target row region;

[0167] For each target row region, the temporal jitter information of the image sensor during the regional acquisition time of the target row region is determined from the jitter information. The target mapping relationship corresponding to the temporal jitter information is found from the first correspondence between jitter information and mapping relationship, which serves as the correction method for the target row region. Each pixel in the target row region is mapped according to the target mapping relationship to achieve correction of the target row region and obtain the processed image. The mapping relationship is the positional correspondence between the pixels in each row region of the image to be processed and the pixels in each row region of the processed image.

[0168] In one embodiment of this application, the region determination module 802 includes:

[0169] The offset information acquisition submodule is used to obtain offset information of the image content shifting in each candidate row region based on the temporal motion data collected during the acquisition of each candidate row region in the motion data.

[0170] The region determination submodule is used to determine the target start position and target end position for each candidate row region by using the offset information of the image content shift in the candidate row region and the preset target region length, and to determine the region between the target start position and target end position from the candidate row region as the target row region.

[0171] In one embodiment of this application, the offset information acquisition submodule includes:

[0172] The second jitter information acquisition unit is used to determine the jitter information of the image sensor in the target time period based on the motion data;

[0173] The offset information acquisition unit is used to obtain offset information of image content shifting in each candidate row region based on the temporal jitter information of the image sensor during the acquisition of each candidate row region from the jitter information.

[0174] In one embodiment of this application, the offset information obtaining unit is specifically used for:

[0175] For each candidate row region, the region acquisition time of the image sensor in acquiring the candidate row region is obtained. The temporal jitter information of the image sensor during the region acquisition time is obtained from the jitter information. The offset information corresponding to the temporal jitter information is found from the second correspondence between jitter information and offset information, and is used as the offset information of the image content shift in the candidate row region.

[0176] In one embodiment of this application, the image processing module 803 is specifically used for:

[0177] For each target row region, based on the offset information corresponding to that target row region, a correction method is determined to align that target row region with other target row regions. The determined correction method is then used to correct the target row region, resulting in the processed image.

[0178] In one embodiment of this application, the apparatus further includes:

[0179] The region filtering module is used to filter target row regions that meet preset conditions of violent motion based on the temporal motion data collected during the acquisition of each target row region in the motion data after determining multiple target row regions from the image to be processed.

[0180] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0181] This application also provides an electronic device, such as... Figure 9 As shown, it includes a processor 901, a communication interface 902, a memory 903, and a communication bus 904, wherein the processor 901, the communication interface 902, and the memory 903 communicate with each other through the communication bus 904.

[0182] Memory 903 is used to store computer programs;

[0183] The processor 901 is used to implement an image processing method when executing a program stored in the memory 903.

[0184] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0185] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0186] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0187] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0188] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described image processing methods.

[0189] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform any of the image processing methods described above.

[0190] In the solutions provided in the above embodiments, since image sensors typically use rolling shutters for exposure, the exposure times of pixels in different rows of the acquired image are different. In this case, when the image sensor's jitter frequency is high, the motion trend of the image sensor differs at different exposure times, meaning that the exposure results of pixels in different rows exhibit different tilting, swaying, or partial exposure. Motion data collected by a motion sensor can reflect the motion of the image sensor. Therefore, corrections are performed separately for different target row regions based on the motion data. Since the temporal motion data corresponding to different target row regions are different, the correction methods used for different target row regions also differ. This allows for targeted correction of pixels in different target row regions using different methods, overcoming the rolling shutter effect in the image to be processed. Therefore, it is evident that applying the image processing solution provided in the above embodiments can improve image quality.

[0191] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0192] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0193] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for image acquisition devices, systems, apparatuses, electronic devices, computer storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0194] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. An image processing method, characterized in that, The method includes: The image to be processed is acquired by an image sensor, and motion data for a target time period is acquired by a motion sensor, wherein the target time period is the time period during which the image sensor acquires the image to be processed. The jitter information of the image sensor in the target time period is determined based on the motion data; Based on the temporal jitter information of the image sensor during the acquisition of each candidate row region from the jitter information, offset information of the image content shifting in each candidate row region is obtained; For each candidate row region, the target start position and target end position are determined using the offset information of the image content shift in the candidate row region and the preset target region length. The region between the target start position and the target end position is determined from the candidate row region as the target row region. The pixels included in different row regions belong to different row pixels. For each target row region, the target row region is corrected using the correction method corresponding to that target row region to obtain the processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of that target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

2. The method according to claim 1, characterized in that, For each target row region, the corresponding correction method is used to correct the target row region to obtain the processed image, including: The jitter information of the image sensor in the target time period is determined based on the motion data; For each target row region, based on the temporal jitter information of the image sensor during the acquisition of the target row region from the jitter information, the correction method corresponding to the target row region is determined, and the target row region is corrected using the determined correction method to obtain the processed image.

3. The method according to claim 2, characterized in that, For each target row region, based on the temporal jitter information of the image sensor during the acquisition of that target row region from the jitter information, a correction method corresponding to that target row region is determined. The determined correction method is then used to correct the target row region to obtain a processed image, including: Obtain the regional acquisition time for each target row region; For each target row region, the temporal jitter information of the image sensor during the regional acquisition time of the target row region is determined from the jitter information. The target mapping relationship corresponding to the temporal jitter information is found from the first correspondence between jitter information and mapping relationship, which serves as the correction method for the target row region. Each pixel in the target row region is mapped according to the target mapping relationship to achieve correction of the target row region and obtain the processed image. The mapping relationship is the positional correspondence between the pixels in each row region of the image to be processed and the pixels in each row region of the processed image.

4. The method according to claim 1, characterized in that, The step of obtaining offset information of image content shift in each candidate row region based on the temporal jitter information of the image sensor during the acquisition of each candidate row region from the jitter information includes: For each candidate row region, the region acquisition time of the image sensor in acquiring the candidate row region is obtained. The temporal jitter information of the image sensor during the region acquisition time is obtained from the jitter information. The offset information corresponding to the temporal jitter information is found from the second correspondence between jitter information and offset information, and is used as the offset information of the image content shift in the candidate row region.

5. The method according to claim 1, characterized in that, For each target row region, the corresponding correction method is used to correct the target row region to obtain the processed image, including: For each target row region, based on the offset information corresponding to that target row region, a correction method is determined to align that target row region with other target row regions. The determined correction method is then used to correct the target row region, resulting in the processed image.

6. The method according to any one of claims 1-5, characterized in that, After determining the target start position and target end position for each candidate row region using offset information of image content shift within that candidate row region and a preset target region length, and determining the region between the target start position and target end position from the candidate row region as the target row region, the method further includes: Based on the temporal motion data collected during the collection of each target row region from the motion data, target row regions whose temporal motion data during the collection period meets the preset conditions of intense motion are filtered.

7. An image acquisition device, characterized in that, The image acquisition device includes an image sensor, a motion sensor, and an image processor, wherein: The image sensor is used to: acquire an image to be processed and send the image to be processed to the image processor; The motion sensor is used to: collect motion data over a target time period and send the motion data to the image processor, wherein the target time period is the time period during which the image sensor collects the image to be processed; The image processor is configured to: acquire an image to be processed from an image sensor and acquire motion data for a target time period from a motion sensor; determine jitter information of the image sensor during the target time period based on the motion data; acquire offset information of image content shift in each candidate row region based on the temporal jitter information of the image sensor acquired during each candidate row region from the jitter information; for each candidate row region, determine a target start position and a target end position using the offset information of image content shift in the candidate row region and a preset target region length, and determine the region between the target start position and the target end position from the candidate row region as a target row region; wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data acquired during the target row region from the motion data, and the correction method corresponding to different target row regions is different when the temporal motion data corresponding to different target row regions is different.

8. An image processing system, characterized in that, The system includes an image sensor, a motion sensor, and a processor, wherein: The image sensor is used to: acquire an image to be processed and send the image to be processed to the processor; The motion sensor is used to: collect motion data over a target time period and send the motion data to the processor, wherein the target time period is the time period during which the image sensor collects the image to be processed; The processor is configured to: acquire an image to be processed from an image sensor and acquire motion data for a target time period from a motion sensor; determine jitter information of the image sensor during the target time period based on the motion data; acquire offset information of image content shift in each candidate row region based on the temporal jitter information of the image sensor acquired during each candidate row region from the jitter information; for each candidate row region, determine a target start position and a target end position using the offset information of image content shift in the candidate row region and a preset target region length, and determine the region between the target start position and the target end position from the candidate row region as a target row region; wherein the pixels included in different row regions belong to different row pixels; for each target row region, correct the target row region using the correction method corresponding to the target row region to obtain a processed image, wherein the correction method corresponding to each target row region is determined based on the temporal motion data acquired during the target row region from the motion data, and the correction method corresponding to different target row regions is different when the temporal motion data corresponding to different target row regions is different.

9. An image processing apparatus, characterized in that, The device includes: The data acquisition module is used to acquire the image to be processed collected by the image sensor and to acquire motion data of a target time period collected by the motion sensor, wherein the target time period is the time period during which the image sensor collects the image to be processed; The region determination module is used to determine the jitter information of the image sensor during the target time period based on the motion data; based on the jitter information, it collects the temporal jitter information of the image sensor during each candidate row region to obtain the offset information of the image content shift in each candidate row region; for each candidate row region, it uses the offset information of the image content shift in the candidate row region and the preset target region length to determine the target start position and the target end position, and determines the region between the target start position and the target end position from the candidate row region as the target row region; wherein, the pixels included in different row regions belong to different row pixels; The image processing module is used to correct each target row region using the correction method corresponding to that target row region to obtain the processed image. The correction method corresponding to each target row region is determined based on the temporal motion data collected during the acquisition of that target row region in the motion data. When the temporal motion data corresponding to different target row regions are different, the correction methods corresponding to different target row regions are different.

10. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images