Image processing method and device, electronic equipment and storage medium
By acquiring multiple frames of reference images from different angles during image processing and replacing the moiré pattern sub-regions, the moiré pattern problem was solved, achieving a simple and efficient image quality improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING XIAOMI MOBILE SOFTWARE CO LTD
- Filing Date
- 2021-05-31
- Publication Date
- 2026-06-16
AI Technical Summary
There is a lack of effective methods in the current technology to improve or remove moiré patterns, which leads to a decrease in image quality.
By acquiring a preset sub-region in the initial image, multiple frames of reference images from different angles are collected, and the algorithm is used to replace the moiré sub-region in the initial image with the reference sub-region that meets the conditions, thus synthesizing the target image.
It can improve or eliminate moiré patterns in a simple and efficient way, improve image quality, and avoid the limitations and operational complexity of traditional methods.
Smart Images
Figure CN115423692B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing, and more particularly to an image processing method, apparatus, electronic device, and storage medium. Background Technology
[0002] With technological advancements and improved living standards, people have increasingly higher demands for the image quality captured by electronic devices. During the shooting process, if the object being photographed has fine textures, or if the spatial frequency of the object is close to that of the image sensor, causing high-frequency interference, the resulting image will exhibit water-ripple-like colored stripes, known as moiré patterns. Moiré patterns lack a distinct shape or regularity, and their presence severely impacts image quality.
[0003] There is still a lack of effective methods to improve or remove moiré patterns in related technologies. Summary of the Invention
[0004] To overcome the problems existing in related technologies, this disclosure provides an image processing method, apparatus, electronic device, and storage medium.
[0005] According to a first aspect of the present disclosure, an image processing method is proposed, the method comprising:
[0006] Acquire the initial image of the subject;
[0007] Determine a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions obtained by dividing the initial image, and the preset sub-region is a sub-region in the multiple sub-regions that has moiré patterns;
[0008] Multiple reference images of the subject at different angles are acquired from the same shooting direction as the initial image.
[0009] Based on the reference sub-region in the reference image, the preset sub-region in the initial image is processed to obtain the target image; wherein, the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions.
[0010] In some embodiments, acquiring an initial image of the subject includes:
[0011] Acquire motion data collected by preset sensors;
[0012] In response to the motion data being within a threshold range, it is determined that the electronic device is in a stationary state;
[0013] In the static state, the initial image is acquired.
[0014] In some embodiments, acquiring multiple frames of reference images of the subject at different angles from the same shooting direction as the initial image includes:
[0015] The image sensor of the control electronic device rotates within a target angle range around the central axis of the image sensor;
[0016] During the rotation of the image sensor, multiple frames of reference images of the subject at different angles are acquired in the same shooting direction as the initial image.
[0017] In some embodiments, acquiring multiple frames of reference images of the subject from different angles includes:
[0018] A reference image is captured at each set angle interval; or, a reference image is captured at each set time interval.
[0019] In some embodiments, acquiring multiple frames of reference images of the subject from different angles includes:
[0020] The target angle range is determined based on the distribution area of the preset sub-regions in the initial image;
[0021] Within the reference angle range, a reference image is acquired at each set angle interval.
[0022] In some embodiments, processing the preset sub-region in the initial image based on the reference sub-region in the reference image to obtain the target image includes:
[0023] Determine at least one frame of the reference images containing the reference sub-region among the multiple frames of the reference images;
[0024] In the reference image where the reference sub-region exists, the similarity between the reference sub-region of each frame of the reference image and the preset sub-region of the initial image is determined respectively;
[0025] A reference image whose similarity is within the reference range is determined as the target reference image. A preset sub-region of the initial image is processed using a reference sub-region of at least one frame of the target reference image to obtain the target image.
[0026] In some embodiments, processing a preset sub-region of the initial image with a reference sub-region of at least one frame of the target reference image to obtain the target image includes:
[0027] Based on the image angle of the preset sub-region, the registered reference sub-region is obtained in at least one frame of the target reference image;
[0028] The reference sub-region after registration is used to replace the corresponding preset sub-region in the initial image to synthesize the target image.
[0029] According to a second aspect of the present disclosure, an image processing apparatus is provided, the apparatus comprising:
[0030] The acquisition module is used to acquire the initial image of the subject.
[0031] A determining module is used to determine a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions obtained by dividing the initial image, and the preset sub-region is a sub-region in the multiple sub-regions that has moiré patterns;
[0032] The acquisition module is used to acquire multiple frames of reference images of the subject at different angles from the same shooting direction as the initial image;
[0033] The processing module is used to process the preset sub-region in the initial image according to the reference sub-region in the reference image to obtain the target image; wherein the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions.
[0034] In some embodiments, the acquisition module is further configured to:
[0035] Acquire motion data collected by preset sensors;
[0036] In response to the motion data being within a threshold range, it is determined that the electronic device is in a stationary state;
[0037] In the static state, the initial image is acquired.
[0038] In some embodiments, the acquisition module is used for:
[0039] The image sensor of the control electronic device rotates within a target angle range around the central axis of the image sensor;
[0040] During the rotation of the image sensor, multiple frames of reference images of the subject at different angles are acquired in the same shooting direction as the initial image.
[0041] In some embodiments, the acquisition module is further configured to:
[0042] A reference image is captured at each set angle interval; or, a reference image is captured at each set time interval.
[0043] In some embodiments, the acquisition module is further configured to:
[0044] The target angle range is determined based on the distribution area of the preset sub-regions in the initial image;
[0045] Within the target angle range, a reference image is acquired at each set angle interval.
[0046] In some embodiments, the processing module is further configured to:
[0047] Determine at least one frame of the reference images in which the reference sub-region exists among the multiple frames of the reference images;
[0048] In the reference image where the reference sub-region exists, the similarity between the reference sub-region of each frame of the reference image and the preset sub-region of the initial image is determined respectively;
[0049] A reference image whose similarity is within the reference range is determined as the target reference image. A preset sub-region of the initial image is processed using a reference sub-region of at least one frame of the target reference image to obtain the target image.
[0050] In some embodiments, the processing module is configured to:
[0051] Based on the image angle of the preset sub-region, the registered reference sub-region is obtained in at least one frame of the target reference image;
[0052] The reference sub-region after registration is used to replace the corresponding preset sub-region in the initial image to synthesize the target image.
[0053] According to a third aspect of the present disclosure, an electronic device is provided, comprising:
[0054] processor;
[0055] Memory used to store the processor's executable instructions;
[0056] The processor is configured to perform the image processing method as described in any of the preceding claims.
[0057] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, which, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the image processing method as described in any of the preceding claims.
[0058] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: Using the method of this disclosure, when moiré patterns exist, multiple frames of reference images can be acquired physically within an angular range. Combined with an algorithm, the preset sub-regions with moiré patterns in the reference images are replaced by reference sub-regions that meet certain conditions, and finally, a target image with improved moiré patterns is synthesized. This method for improving or eliminating moiré patterns is simpler and more convenient.
[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0060] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0061] Figure 1 This is a flowchart illustrating a method according to an exemplary embodiment.
[0062] Figure 2 This is a flowchart illustrating a method according to an exemplary embodiment.
[0063] Figure 3 This is a flowchart illustrating a method according to an exemplary embodiment.
[0064] Figure 4 This is a flowchart illustrating a method according to an exemplary embodiment.
[0065] Figure 5 This is a flowchart illustrating a method according to an exemplary embodiment.
[0066] Figure 6 This is a schematic diagram of an initial image shown according to an exemplary embodiment.
[0067] Figure 7 This is a schematic diagram of a reference image shown according to an exemplary embodiment.
[0068] Figure 8 This is a schematic diagram of the moiré pattern of the initial image shown according to an example embodiment.
[0069] Figure 9 This is a block diagram of an apparatus according to an exemplary embodiment.
[0070] Figure 10 This is a block diagram of an electronic device according to an exemplary embodiment. Detailed Implementation
[0071] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.
[0072] With technological advancements and improved living standards, people have increasingly higher demands for the image quality captured by electronic devices. During the shooting process, if the object being photographed has fine textures, or if the spatial frequency of the object is close to that of the image sensor, causing high-frequency interference, the resulting image will exhibit water-ripple-like colored stripes, known as moiré patterns. Moiré patterns lack a distinct shape or regularity, and their presence severely impacts image quality.
[0073] To eliminate moiré patterns, common techniques include: First, installing a low-pass filter on the lens to capture an image free of moiré patterns. Second, using software like Photoshop to remove moiré patterns in post-processing. Of these methods, the first reduces image sharpness and has limited applicability, suitable for cameras but not for common devices like mobile phones. The second method is more cumbersome and requires higher technical skills.
[0074] There is still a lack of effective methods to improve the removal of moiré patterns in related technologies.
[0075] This disclosure proposes an image processing method, comprising: acquiring an initial image of a subject; determining a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions divided from the initial image, and the preset sub-region is a sub-region containing moiré patterns among the multiple sub-regions; acquiring multiple frames of reference images of the subject at different angles in the same shooting direction of the initial image; processing the preset sub-region in the initial image according to the reference sub-region in the reference image to obtain a target image; wherein the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions. Using the method of this disclosure, when moiré patterns exist, multiple frames of reference images can be acquired physically within an angular range. Combined with an algorithm, the preset sub-region with moiré patterns is replaced by the reference sub-region in the reference image that meets the conditions, and finally a target image with improved moiré patterns is synthesized, making the method of improving or eliminating moiré patterns simpler and more convenient.
[0076] In one exemplary embodiment, the image processing method of this embodiment is applied to an electronic device. The electronic device may be, for example, a mobile phone, tablet computer, laptop computer, smart wearable device, camera, or similar device.
[0077] like Figure 1 As shown, the method in this embodiment may include the following steps:
[0078] S110. Obtain the initial image of the subject.
[0079] S120. Determine the preset sub-region in the initial image.
[0080] S130. Acquire multiple reference images of the subject from different angles in the same shooting direction as the initial image.
[0081] S140. Based on the reference sub-region in the reference image, process the preset sub-region in the initial image to obtain the target image.
[0082] In this embodiment, the method is applied to the camera application of an electronic device. Therefore, before executing this embodiment, the electronic device should have already turned on the camera and executed the method in the camera's application interface.
[0083] In step S110, based on the user's operation command (such as clicking the shutter button), an initial image of the subject is acquired. The acquired initial image is available for preview by the user in the camera's application interface.
[0084] In step S120, the preset operation command may be, for example, a click command on the "remove moiré" option in the application interface, or a voice command for "remove moiré".
[0085] In this step, the initial image includes multiple sub-regions obtained by dividing the initial image, for example, including multiple evenly spaced sub-regions. There are various ways to divide the sub-regions, such as dividing the initial image into m×n sub-regions based on user selection instructions or program default settings. During the division process, the amount of computation or matching work should be reduced, such as... Figure 6 or Figure 8 The example shown has an initial image consisting of 4×4 sub-regions.
[0086] In this step, it is determined whether there are moiré patterns in each sub-region of the initial image. Sub-regions with moiré patterns are recorded as preset sub-regions.
[0087] In one example, a deep learning neural network algorithm is used to determine whether moiré patterns exist in each sub-region of an initial image. For instance, a pre-trained network model stored in the AI chip of an electronic device can be used as input to the initial image and output information about the regions in the initial image where moiré patterns exist.
[0088] In another example, the number of edge pixels in each sub-region of the initial image is counted separately and compared with the corresponding edge pixel threshold for that sub-region. When the number of edge pixels exceeds the threshold, it indicates the presence of edge pixels in the image that do not belong to the edge of the subject, and is considered to indicate moiré patterns.
[0089] In step S130, each reference image corresponds to the initial image. For example, each reference image and the initial image have the same shooting area of the subject, the same shooting distance, and the image content is within the error range. Each reference image also includes multiple evenly distributed sub-regions, and the sub-regions contained therein correspond one-to-one with the sub-regions contained in the initial image. For example, in... Figure 6 or Figure 8 In the example shown, the initial image comprises 4×4 sub-regions, then refer to Figure 7 As shown, each frame of the reference image includes 4×4 sub-regions, and the image content of the reference image sub-regions should be related to the image content of the corresponding sub-regions of the initial image.
[0090] Understandably, the subject can remain stationary while acquiring reference images of the subject within the target angle range. During the acquisition of these reference images, algorithms such as cropping and fusion can be used to process images that are clearly detached from the subject, ensuring that the content of each frame of the acquired reference image is within the error range compared to the initial image.
[0091] In step S140, the reference sub-region is a sub-region in the reference image that meets preset conditions and corresponds to a preset sub-region, such as the image content of the reference sub-region corresponding to the image content of the preset sub-region. The preset conditions are used to characterize that the corresponding sub-region does not contain moiré patterns or the moiré patterns are not obvious, such as the moiré effect being below a threshold, or the moiré effect being lower than the moiré effect of the initial image. That is, in this step, the sub-region in the reference image that corresponds to the preset sub-region (the sub-region in the initial image that contains moiré patterns) and meets the preset conditions is recorded as the reference sub-region.
[0092] In this step, the initial image may contain moiré patterns in one or more preset sub-regions. In the multi-frame reference images acquired in step S130, reference sub-regions may exist in one or more frames. The processor can control the reference sub-regions in any frame to replace the preset sub-regions one-to-one, synthesizing a target image without moiré patterns or with indistinct moiré patterns. Alternatively, it can control the selection of different reference sub-regions in the multi-frame reference images, replacing the preset sub-regions one-to-one, to synthesize a target image without moiré patterns or with indistinct moiré patterns.
[0093] In one exemplary embodiment, such as Figure 2 As shown, step S110 of this embodiment includes the following steps:
[0094] S1101. Acquire motion data collected by preset sensors.
[0095] S1102. In response to motion data being within a threshold range, determine that the electronic device is in a stationary state.
[0096] S1103. In a static state, acquire the initial image.
[0097] In step S1101, the preset sensor is, for example, an accelerometer or a gyroscope. Taking a gyroscope as the preset sensor, the motion data is, for example, the average angular acceleration data of the electronic device. The electronic device has angular acceleration in the X, Y, and Z directions. The average data is calculated based on the angular acceleration in the three directions as the motion data.
[0098] In step S1102, the threshold range can be reference data pre-stored in the electronic device to characterize when the electronic device is in a stationary state. The processor compares the acquired motion data with the threshold range; when the motion data is within the threshold range, the processor determines that the electronic device is in a stationary state.
[0099] In step S1103, the static state can refer to the tripod mode of the electronic device, that is, the electronic device is not moving, is in a stable state, or is stationary. In the static state, the electronic device and the subject can always maintain the same angle of opposition, for example, the electronic device is always facing side A of the subject. Therefore, in step S130, during the acquisition of multiple reference images, multiple frames of images of side A of the subject are acquired from different angles. This reduces the amount of structure of side B of the subject being acquired, reduces interference factors in the reference images, and reduces the difficulty and time consumption of image processing.
[0100] In this step, the processor of the electronic device can combine the motion data detected by the sensor to determine whether the electronic device is stationary. If so, after acquiring the initial image, step S120 of this embodiment continues; if not, normal shooting is performed, and the current shooting can be ended after acquiring the initial image (no need to remove moiré patterns). It is understood that the shooting method for removing moiré patterns in this embodiment takes longer than the normal shooting method.
[0101] In one exemplary embodiment, such as Figure 3 As shown, step S130 in this embodiment may include the following steps:
[0102] S1301, Controlling the image sensor of the electronic device, rotating the target angle range around the central axis of the image sensor.
[0103] S1302. During the rotation of the image sensor, multiple reference images of the subject at different angles are acquired in the same shooting direction as the initial image.
[0104] In step S1301, the camera component of the electronic device includes, for example, a lens group, a color filter, and an image sensor. In this imaging system, the lens group can be considered the object side, and the image sensor can be considered the image side or imaging surface. The image sensor is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) sensor.
[0105] The central axis of the image sensor refers to, for example, the axis passing through the centroid of the image sensor, which is parallel to or coincides with the optical axis of the camera assembly. The target angle range is, for example, 0°-180° or 0°-360°. The processor controls the drive assembly to drive the image sensor to rotate around the central axis, and the time for the image sensor to rotate within the target angle range can be set according to requirements.
[0106] In step S1302, during the rotation of the image sensor, the processor can issue control commands for acquiring reference images at different angles or times, controlling the camera assembly to acquire multiple frames of reference images from different angles.
[0107] In one example, a reference image is captured at intervals with a set angle.
[0108] In this example, during the rotation, multiple reference images are acquired at set angle intervals within the total angle range. For example, if the image sensor acquires one reference image every 30° of rotation, then six reference images can be acquired after a 180° rotation; and twelve reference images can be acquired after a 360° rotation.
[0109] In another example, a reference image is captured at set intervals.
[0110] In this example, during the rotation, multiple reference images are captured at set intervals. For instance, starting from the initial moment of rotation, the image sensor acquires one reference image every 1 second until a preset number of reference images are acquired. The preset number is, for example, 10.
[0111] In another example, it could also include the following steps:
[0112] S1302-1. Determine the target angle range based on the distribution area of the preset sub-region in the initial image.
[0113] In this step, for example, if the initial image is a human image, and the preset sub-region (the area with moiré patterns) is the face, then this step can further narrow down the rotation range within the target angle range to determine the target angle range that yields the best image capture effect for the face portion.
[0114] Alternatively, determine the distribution area of a preset sub-region (the area where moiré patterns exist) in the initial image, and determine the angle information of the edge line of the moiré pattern relative to the reference direction within the preset sub-region to determine the target angle range.
[0115] S1302-2. Within the target angle range, acquire one reference image frame at set angle intervals. In this step, the rotation range of the image sensor becomes the reference angle range, and the interval angle can be set to, for example, 1°. Reducing the rotation range helps improve the efficiency of acquiring reference images.
[0116] In one exemplary embodiment, such as Figure 4 As shown, step S140 in this embodiment may include the following steps:
[0117] S1401. Determine at least one reference image among multiple reference images that contains a reference sub-region.
[0118] S1402. In the reference image where there is a reference sub-region, determine the similarity between the reference sub-region of each frame of the reference image and the preset sub-region of the initial image.
[0119] S1403. Determine a reference image whose similarity is within the reference range as the target reference image, and process the preset sub-region of the initial image with the reference sub-region of at least one frame of the target reference image to obtain the target image.
[0120] In step S1401, as described in the aforementioned embodiment, each sub-region in the reference image corresponds one-to-one with a sub-region in the initial image. In the acquired multiple reference images, it is determined whether the sub-region corresponding to the preset sub-region in each reference image meets a preset condition. The sub-region corresponding to the preset sub-region and meeting the preset condition is the reference sub-region. In other words, this step determines whether a reference sub-region exists in each reference image.
[0121] For example, combining Figure 6 or Figure 8 In the example shown, the initial image has 4×4 sub-regions, with the preset sub-regions being P5 to P8. Then, in a multi-frame reference image, it is determined whether the P5' to P8' sub-regions of each frame of the reference image meet the preset conditions. If at least one sub-region among the P5' to P8' sub-regions of each frame of the reference image meets the preset conditions, then there is at least one reference sub-region in that reference image.
[0122] In step S1402, the sub-region that corresponds to the preset sub-region and meets the preset conditions is the reference sub-region, and the reference image corresponding to the reference sub-region is the reference image that contains the reference sub-region.
[0123] For example, still refer to Figure 6In the example shown, there are multiple preset sub-regions. Therefore, for each frame of the reference image, at least one reference sub-region needs to exist. For example, in the first frame of the reference image, sub-regions P5' to P7' satisfy the preset condition, meaning they contain all three reference sub-regions P5' to P7'. In the second frame of the reference image, sub-region P8' satisfies the preset condition, meaning it contains only one reference sub-region, P8'. In the third frame of the reference image, as... Figure 7 As shown, sub-regions P5' to P8' all meet the preset conditions, meaning they contain these four reference sub-regions. Reference sub-regions also exist in the first, second, and third frame reference images and are reserved for later use.
[0124] In this step, reference images that do not contain reference sub-regions are removed.
[0125] For reference images with reference sub-regions, each reference image is matched against the initial image. The matching process can be as follows: identify and compare the similarity between the reference sub-regions of the reference images and the preset sub-regions of the initial images.
[0126] In step S1403, the target reference image is: the reference image whose similarity to the matched images is within the reference range during the above matching process. When the similarity between the target reference image and the initial image is within the reference range, it indicates that the subject images in each sub-region of the field of view of the reference image obtained after CMOS rotation are consistent with the subject images in each sub-region of the field of view of the initial image. The target reference image obtained after rotation is a view of the subject from other angles.
[0127] The preset sub-region of the initial image is repaired using the reference sub-region of the target reference image to obtain a target image without moiré patterns.
[0128] When the similarity between the reference image and the initial image is outside the reference range, it indicates that the images in the field of view of some sub-regions in the reference image are inconsistent with the images in the field of view of some sub-regions in the initial image. For example, sub-region P16 in the initial image is the edge of the subject, while sub-region P16' in the reference image is the background image.
[0129] In this step, the target reference image is retained, and reference images that fail to match are removed.
[0130] In this embodiment, as Figure 5 As shown, step S1403 may include the following steps:
[0131] S1403-1. Based on the image angle of the preset sub-region, obtain the registered reference sub-region in at least one frame of the target reference image.
[0132] S1403-2. Replace the corresponding preset sub-region in the initial image with the reference sub-region after registration, and synthesize the target image.
[0133] In step S1403-1, the reference sub-region can be obtained by cropping the reference image, for example.
[0134] In the first example, when there is only one preset sub-region, a reference sub-region is obtained in any frame of the target reference image, and image registration is performed between the reference sub-region and the preset sub-region based on the image angle within the preset sub-region. For example, the obtained reference sub-region is flipped at an appropriate angle based on the image angle within the preset sub-region.
[0135] In the second example, when there are multiple preset sub-regions, they can be obtained in the following two ways:
[0136] The first approach involves identifying multiple registered reference sub-regions within the first target reference image.
[0137] In this method, a first target reference image also exists within the target reference image, containing multiple reference sub-regions that correspond one-to-one with multiple preset sub-regions, such as combining... Figure 6 or Figure 8 As shown, sub-regions P5 to P8 in the initial image are all preset sub-regions, such as... Figure 7 As shown, sub-regions P5' to P8' in the reference image are defined as reference sub-regions. Then, the first target reference image is cropped to obtain multiple reference sub-regions. The registration method can be found in the first example above.
[0138] The second method involves determining the reference sub-region in each frame of the target reference image to obtain multiple registered reference sub-regions.
[0139] In this method, if there is no single frame reference image in the target reference image that contains a number of reference sub-regions exactly the same as the preset number of sub-regions, then the method of cropping reference sub-regions from multiple target reference images and stitching them together can be used. The registration method can be found in the first example above.
[0140] Understandably, multiple reference sub-regions correspond one-to-one with multiple preset sub-regions. In the second method of this example, when the same preset sub-region corresponds to a reference sub-region in different target reference images, only one of the different target reference images is selected for cropping to obtain the corresponding reference sub-region; it is not necessary to repeatedly obtain the same reference sub-region.
[0141] The image processing method in this embodiment removes moiré patterns from the final synthesized image by physically acquiring multiple reference images and performing matching calibration within those reference images, ensuring the image remains undistorted. Compared to post-processing software methods for moiré patterns, this method is more efficient and improves the user experience.
[0142] In one exemplary embodiment, this disclosure also provides an image processing apparatus, such as... Figure 9 As shown, the apparatus of this embodiment includes: an acquisition module 110, a determination module 120, a collection module 130, and a processing module 140. The apparatus of this embodiment is used to implement... Figure 1 The method is illustrated below. The acquisition module 110 acquires an initial image of the subject. The determination module 120 determines a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions divided from the initial image, and the preset sub-region is a sub-region containing moiré patterns among the multiple sub-regions. The acquisition module 130 acquires multiple frames of reference images of the subject at different angles from the same shooting direction as the initial image. The processing module 140 processes the preset sub-region in the initial image based on the reference sub-region in the reference image to obtain a target image; wherein the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions.
[0143] In one exemplary embodiment, reference is still made to... Figure 9 The apparatus in this embodiment is used to achieve, for example... Figure 2 The method is illustrated. The acquisition module 110 is further configured to: acquire motion data collected by a preset sensor; and, in response to the motion data being within a threshold range, determine that the electronic device is in a stationary state. In the stationary state, an initial image is acquired.
[0144] In one exemplary embodiment, reference is still made to... Figure 9 The apparatus in this embodiment is used to achieve, for example... Figure 3 The method is illustrated. The acquisition module 130 is used to: control the image sensor of the electronic device to rotate around the central axis of the image sensor by a target angle range; during the rotation of the image sensor, acquire multiple frames of reference images of the subject at different angles in the same shooting direction as the initial image. In this embodiment, the acquisition module 130 is also used to acquire one frame of reference image at each set angle interval; or, at each set time interval, acquire one frame of reference image. Alternatively, it can determine the target angle range based on the distribution position of a preset sub-region in the initial image; within the target angle range, acquire one frame of reference image at each set angle interval.
[0145] In one exemplary embodiment, reference is still made to... Figure 9 The apparatus in this embodiment is used to achieve, for example... Figure 4The method is shown. The processing module 140 is further configured to: determine at least one reference image among multiple reference images that contains a reference sub-region; in the reference image containing the reference sub-region, determine the similarity between the reference sub-region of each reference image and a preset sub-region of the initial image; determine a reference image whose similarity is within a reference range as a target reference image; and process the preset sub-region of the initial image using the reference sub-region of at least one target reference image to obtain the target image.
[0146] In one exemplary embodiment, reference is still made to... Figure 9 The apparatus in this embodiment is used to achieve, for example... Figure 5 The method shown is as follows. The processing module 140 is used to: obtain a registered reference sub-region in at least one frame of a target reference image based on the image angle of the preset sub-region; and control the registered reference sub-region to replace the corresponding preset sub-region in the initial image to synthesize the target image.
[0147] like Figure 10 The diagram shown is a block diagram of an electronic device. This disclosure also provides an electronic device, for example, device 500 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.
[0148] Device 500 may include one or more of the following components: processing component 502, memory 504, power component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.
[0149] Processing component 502 typically controls the overall operation of device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.
[0150] Memory 504 is configured to store various types of data to support the operation of device 500. Examples of this data include instructions for any application or method operating on device 500, contact data, phonebook data, messages, pictures, videos, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.
[0151] The power supply component 506 provides power to the various components of the device 500. The power supply component 506 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 500.
[0152] Multimedia component 508 includes a screen that provides an output interface between device 500 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When device 500 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.
[0153] Audio component 510 is configured to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) configured to receive external audio signals when device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.
[0154] I / O interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.
[0155] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of device 500. For example, sensor assembly 514 may detect the on / off state of device 500, the relative positioning of components such as the display and keypad of device 500, changes in the position of device 500 or a component of device 500, the presence or absence of user contact with device 500, the orientation or acceleration / deceleration of device 500, and temperature changes of device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 514 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.
[0156] Communication component 516 is configured to facilitate wired or wireless communication between device 500 and other devices. Device 500 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0157] In an exemplary embodiment, device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.
[0158] Another exemplary embodiment of this disclosure provides a non-transitory computer-readable storage medium, such as a memory 504 including instructions that can be executed by a processor 520 of a device 500 to perform the described method. For example, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device. When the instructions in the storage medium are executed by the processor of an electronic device, the electronic device is able to perform the described method.
[0159] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.
[0160] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.
Claims
1. An image processing method, characterized in that the method include: Acquire the initial image of the subject; Determine a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions obtained by dividing the initial image, the multiple sub-regions are evenly distributed, and the preset sub-region is a sub-region in the multiple sub-regions that has moiré patterns; Multiple reference images of the subject at different angles are acquired from the same shooting direction as the initial image. Based on the reference sub-region in the reference image, the preset sub-region in the initial image is processed to obtain the target image; wherein, the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions; The step of processing the preset sub-region in the initial image based on the reference sub-region in the reference image to obtain the target image includes: Identify and compare the similarity between the reference sub-region of the reference image and the preset sub-region of the initial image; Reference images whose similarity is within the reference range are identified as target reference images; Based on the target reference image, the preset sub-region in the initial image is processed to obtain the target image.
2. The image processing method according to claim 1, characterized in that, The process of acquiring the initial image of the subject includes: Acquire motion data collected by preset sensors; In response to the motion data being within a threshold range, it is determined that the electronic device is in a stationary state; In the static state, the initial image is acquired.
3. The image processing method according to claim 1, characterized in that, The step of acquiring multiple reference images of the subject at different angles from the same shooting direction as the initial image includes: The image sensor of the control electronic device rotates within a target angle range around the central axis of the image sensor; During the rotation of the image sensor, multiple frames of reference images of the subject at different angles are acquired in the same shooting direction as the initial image.
4. The image processing method according to claim 3, characterized in that, The acquisition of multiple frames of reference images of the subject from different angles includes: A reference image is captured at each set angle interval; or, a reference image is captured at each set time interval.
5. The image processing method according to claim 3, characterized in that, The acquisition of multiple frames of reference images of the subject from different angles includes: The target angle range is determined based on the distribution area of the preset sub-regions in the initial image; Within the target angle range, a reference image is acquired at each set angle interval.
6. The image processing method according to claim 1, characterized in that, The step of processing the preset sub-region in the initial image based on the reference sub-region in the reference image to obtain the target image includes: Determine at least one frame of the reference images containing the reference sub-region among the multiple frames of the reference images; In the reference image where the reference sub-region exists, the similarity between the reference sub-region of each frame of the reference image and the preset sub-region of the initial image is determined respectively; The reference image whose similarity is within the reference range is determined as the target reference image. The preset sub-region of the initial image is processed using the reference sub-region of at least one frame of the target reference image to obtain the target image.
7. The image processing method according to claim 6, characterized in that, The step of processing a preset sub-region of the initial image with a reference sub-region of at least one frame of the target reference image to obtain the target image includes: Based on the image angle of the preset sub-region, the registered reference sub-region is obtained in at least one frame of the target reference image; The reference sub-region after registration is used to replace the corresponding preset sub-region in the initial image to synthesize the target image.
8. An image processing apparatus, characterized in that, The device includes: The acquisition module is used to acquire the initial image of the subject. A determining module is used to determine a preset sub-region in the initial image, wherein the initial image includes multiple sub-regions obtained by dividing the initial image, the multiple sub-regions are evenly distributed, and the preset sub-region is a sub-region in the multiple sub-regions that has moiré patterns; The acquisition module is used to acquire multiple frames of reference images of the subject at different angles from the same shooting direction as the initial image; The processing module is used to process the preset sub-region in the initial image according to the reference sub-region in the reference image to obtain the target image; wherein, the reference sub-region is a sub-region in the reference image that corresponds to the preset sub-region and satisfies preset conditions; The processing module is further configured to identify and compare the similarity between the reference sub-region of the reference image and the preset sub-region of the initial image; Reference images whose similarity is within the reference range are identified as target reference images; Based on the target reference image, the preset sub-region in the initial image is processed to obtain the target image.
9. The image processing apparatus according to claim 8, characterized in that, The acquisition module is also used for: Acquire motion data collected by preset sensors; In response to the motion data being within a threshold range, it is determined that the electronic device is in a stationary state; In the static state, the initial image is acquired.
10. The image processing apparatus according to claim 8, characterized in that, The acquisition module is used for: The image sensor of the control electronic device rotates within a target angle range around the central axis of the image sensor; During the rotation of the image sensor, multiple frames of reference images of the subject at different angles are acquired in the same shooting direction as the initial image.
11. The image processing apparatus according to claim 10, characterized in that, The acquisition module is also used for: A reference image is captured at each set angle interval; or, a reference image is captured at each set time interval.
12. The image processing apparatus according to claim 10, characterized in that, The acquisition module is also used for: The target angle range is determined based on the distribution area of the preset sub-regions in the initial image; Within the target angle range, a reference image is acquired at each set angle interval.
13. The image processing apparatus according to claim 8, characterized in that, The processing module is also used for: Determine at least one frame of the reference images in which the reference sub-region exists among the multiple frames of the reference images; In the reference image where the reference sub-region exists, the similarity between the reference sub-region of each frame of the reference image and the preset sub-region of the initial image is determined respectively; The reference image whose similarity is within the reference range is determined as the target reference image. The preset sub-region of the initial image is processed using the reference sub-region of at least one frame of the target reference image to obtain the target image.
14. The image processing apparatus according to claim 13, characterized in that, The processing module is also used for: Based on the image angle of the preset sub-region, the registered reference sub-region is obtained in at least one frame of the target reference image; The reference sub-region after registration is used to replace the corresponding preset sub-region in the initial image to synthesize the target image.
15. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to perform the image processing method as described in any one of claims 1 to 7.
16. A non-transitory computer-readable storage medium, characterized in that, When the instructions in the storage medium are executed by the processor of the electronic device, the electronic device is able to perform the image processing method as described in any one of claims 1 to 7.