Image processing method and device, equipment and medium
By applying the target processing strategy in image processing, generating image data with a smaller amount and inputting it into the intelligent engine, the problem of long-term image generation in the prior art is solved, and image processing efficiency and visual effects are improved.
Patent Information
- Application Number
- CN202411207909.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, it takes a long time to input blurred images in the image gallery to generate super-segment high-definition images. This is mainly due to the large input resolution, which leads to the large calculation amount and low efficiency when processing the intelligent engine.
By acquiring the original image data collected by the image acquisition device, the target processing strategy is used to enhance the image resolution, generate a smaller amount of second image data, and input it into the intelligent engine for super-resolution processing, eliminating the false after digital interpolation Pixels reduce the computing volume of the smart engine and improve processing efficiency.
It effectively reduces the computing volume of the intelligent engine, improves image processing efficiency, reduces image processing time, and ensures the smoothness of users during the acquisition process, and provides better visual effects.
Smart Images

Figure CN119006287A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image processing method, device, equipment and medium. Background Art
[0002] In the solution provided by the related art, the user inputs the blurred image in the gallery into the artificial intelligence (AI) model to generate a super-resolution high-definition image. It takes a long time to generate the image using this method. Summary of the invention
[0003] In view of this, the present disclosure provides an image processing method, apparatus, device and medium.
[0004] According to a first aspect of the present disclosure, there is provided an image processing method, comprising: if a target instruction for performing resolution enhancement on a first image file is received, obtaining second image data corresponding to the first image data contained in the first image file; inputting the second image data into an intelligent engine to obtain third image data; and outputting the third image data corresponding to the first image file in response to the target instruction; wherein the second image data is original image data acquired by an image acquisition device, and the first image data is image data acquired by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; and the data volume of the second image data is less than the data volume of the first image data.
[0005] According to an embodiment of the present disclosure, a first image file includes first image data and first description data of the first image file, and obtaining second image data corresponding to the first image file includes: reading a target processing parameter indicating a target processing strategy in the first description data; and processing the first image data based on the target processing parameter to obtain the second image data.
[0006] According to an embodiment of the present disclosure, the target processing parameters include inverse operation parameters and inverse effect parameters; processing the first image data based on the target processing parameters to obtain the second image data includes: processing the first image data based on the inverse effect parameters to obtain the fifth image data; processing the fifth image data based on the inverse operation parameters to obtain the second image data.
[0007] According to an embodiment of the present disclosure, a first image file includes first image data and first description data of the first image file, the first description data includes second image data, and obtaining second image data corresponding to the first image data contained in the first image file includes: reading the second image data in the first description data.
[0008] According to an embodiment of the present disclosure, if a target instruction for performing resolution enhancement on first image data is received, before acquiring second image data corresponding to the first image data, the method also includes: acquiring fourth image data acquired by an image acquisition device; in response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is part of the fourth image data, processing the second image data based on a target processing strategy to obtain first image data; and generating a first image file including the first image data and target processing parameters indicating the target processing strategy.
[0009] According to an embodiment of the present disclosure, if a target instruction for performing resolution enhancement on first image data is received, before acquiring second image data corresponding to the first image data, the method also includes: acquiring fourth image data acquired by an image acquisition device; in response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is part of the fourth image data, processing the second image data based on a target processing strategy to obtain the first image data; and generating a first image file containing the first image data and the second image data.
[0010] According to an embodiment of the present disclosure, the second image data is input into the intelligent engine to obtain the third image data, including: sending the second image data to the server so that the server inputs the second image data into the intelligent engine; and receiving the third image data sent by the server.
[0011] A second aspect of the present disclosure provides an image processing device, comprising: an acquisition module, for acquiring second image data corresponding to the first image data contained in the first image file if a target instruction for performing resolution enhancement on the first image file is received; a processing module, for inputting the second image data into an intelligent engine to obtain third image data; an output module, for outputting the third image data corresponding to the first image file in response to the target instruction; wherein the second image data is the original image data acquired by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; and the data volume of the second image data is less than the data volume of the first image data.
[0012] The third aspect of the present disclosure provides an electronic device, comprising: an image acquisition device, for acquiring image data; one or more processors, for obtaining second image data corresponding to the first image data contained in the first image file if a target instruction for performing resolution enhancement on the first image file is received; inputting the second image data into an intelligent engine to obtain third image data; and outputting the third image data corresponding to the first image file in response to the target instruction; wherein the second image data is the original image data acquired by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; and the data volume of the second image data is less than the data volume of the first image data.
[0013] The fourth aspect of the present disclosure further provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to execute the above-mentioned image processing method.
[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:
[0016] Figure 1 A schematic diagram schematically shows an image processing method in the related art;
[0017] Figure 2 The application scenario diagram of the image processing method, apparatus, device and medium according to the embodiment of the present disclosure is schematically shown;
[0018] Figure 3 The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown;
[0019] Figure 4 A flowchart of a method for acquiring second image data according to an embodiment of the present disclosure is schematically shown;
[0020] Figure 5 A flowchart of a method for determining second image data according to an embodiment of the present disclosure is schematically shown;
[0021] Fig. 6A A flowchart of a method for generating a first image file according to an embodiment of the present disclosure is schematically shown;
[0022] Figure 6BA schematic diagram schematically shows generation of a first image file according to an embodiment of the present disclosure;
[0023] Fig. 7A A flowchart of a method for generating a first image file according to another embodiment of the present disclosure is schematically shown;
[0024] Figure 7B A schematic diagram schematically shows generation of a first image file according to another embodiment of the present disclosure;
[0025] Figure 8 A flowchart of a method for determining third image data according to another embodiment of the present disclosure is schematically shown;
[0026] Fig. 9 The structure block diagram of the image processing device according to the embodiment of the present disclosure is schematically shown;
[0027] Fig.10 A block diagram of an electronic device suitable for implementing an image processing method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0028] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.
[0029] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.
[0030] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.
[0031] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).
[0032] The embodiments of the present disclosure provide an image processing method, apparatus, device and medium. Before introducing the technical solutions provided by the embodiments of the present disclosure, the related technologies involved in the present disclosure are first described.
[0033] In the related art, people use the zoom function of their mobile phones to take photos of distant scenes. However, since ordinary zoom is just an interpolation of digital zoom, although the picture can be enlarged, the picture quality and clarity can be obviously reduced.
[0034] Due to the rapid development of artificial intelligence, there are already some large artificial intelligence models that can generate super-resolution high-definition images with just a single image. For example, image super-resolution (SR) processing is an important application of artificial intelligence in computer vision technology, which refers to reconstructing a corresponding high-resolution image based on a low-resolution image.
[0035] In the solution provided by the related art, the user inputs the blurred image in the gallery into the artificial intelligence (AI) model to generate a super-resolution high-definition image. It takes a long time to generate the image using this method.
[0036] Figure 1 The figure schematically shows a schematic diagram of an image processing method in the related art.
[0037] For example, refer to Figure 1, the picture is divided into the stage of generating photos on the camera side and the stage of post-processing photos on the album side. In the stage of generating photos on the camera side, when the user takes a photo, assuming that the camera sensor resolution is 4000 * 4000, when the camera is not zoomed, the size of the photo taken is 4000 * 4000. When in order to shoot a certain area in the image, the user controls the camera to zoom, and the system performs zooming and cropping, that is, the real sensor pixels are actually cropped from the original pixels. For example, if the zoom is 4 times, the real pixel is only 1000 * 1000. In order to generate a photo of 4000 * 4000 in the process of related technology, digital zoom, that is, digital interpolation, is usually performed. However, since the resolution of the original physical sensor has decreased (1000 * 1000), the picture is not clear. The YUV format photo obtained after digital interpolation is encoded into a JPGE format photo (4000 * 4000) and stored in the gallery. Since it is only a data format conversion, the clarity is still very poor. Usually, in order to improve the clarity of the picture, users input the JPGE format photos in the gallery to the AI model for super-resolution calculation during the post-processing stage of the album. Since the current AI large model generates super-resolution images, the time-consuming bottleneck is related to the resolution size of the input image. Therefore, it takes a long time to input the large resolution (4000*4000) pictures stored in the gallery into the AI model for super-resolution calculation.
[0038] Before further describing the embodiments of the present disclosure in detail, the nouns and terms involved in the embodiments of the present disclosure are described. The nouns and terms involved in the embodiments of the present disclosure are subject to the following interpretations.
[0039] Super-resolution processing, also known as super-resolution processing, is to improve the resolution of the original image through hardware or software methods. The process of obtaining a high-resolution image through a series of low-resolution images is super-resolution reconstruction. The core idea of super-resolution reconstruction is to use time bands to obtain a multi-frame image sequence of the same scene in exchange for spatial resolution, thereby realizing the conversion of temporal resolution to spatial resolution.
[0040] YUV: A color encoding method. Commonly used in various video processing components, YUV allows for reduced chromaticity bandwidth when encoding photos or videos, taking into account human perception. "Y" represents brightness (Luminance or Luma), which is the grayscale value, while "U" and "V" represent chromaticity (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.
[0041] Up sampling: also known as upsampling, refers to increasing the size of an image and determining the pixel values of the pixels in the enlarged image by sampling, for example, by bicubic interpolation.
[0042] JPG (or JPEG): A widely used image compression format. The compression method of the JPG format is based on the discrete cosine transform (DCT) technology, which divides the image into small blocks and transforms and quantizes each block to achieve compression.
[0043] The embodiment of the present disclosure provides an image processing method, including: if a target instruction for performing resolution enhancement on a first image file is received, obtaining second image data corresponding to the first image data contained in the first image file; inputting the second image data into an intelligent engine to obtain third image data; in response to the target instruction, outputting the third image data corresponding to the first image file; wherein the second image data is the original image data collected by an image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy in the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data. By using the original image data collected by the image acquisition device as the input for super-resolution processing by the intelligent engine, multiple false pixels obtained by digital interpolation in the first image can be eliminated, reducing the amount of data input processed by the intelligent engine, greatly reducing the amount of calculation for super-resolution of the intelligent engine, and improving the efficiency of image processing.
[0044] The following describes an exemplary application of an electronic device of the image processing method provided by the embodiment of the present disclosure. The electronic device provided by the embodiment of the present disclosure can be implemented as various types of user terminals such as a laptop computer, a tablet computer, a desktop computer, a set-top box, a mobile electronic device (e.g., a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging electronic device, a portable gaming electronic device), and can also be implemented as a server.
[0045] Figure 2 The application scenario diagram of the image processing method, apparatus, device and medium according to the embodiments of the present disclosure is schematically shown.
[0046] like Figure 2 As shown, the application scenario 100 according to this embodiment may include a terminal device 101, a terminal device 102, a terminal device 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the server 105 and the terminal devices 101, 102, 103 respectively.
[0047] The server 105 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal devices 101, 102, and 103 may be smart phones, tablet computers, laptop computers, desktop computers, smart speakers, smart watches, etc., but are not limited thereto. The terminal devices and servers may be directly or indirectly connected via wired or wireless communications, which is not limited in the embodiments of the present disclosure.
[0048] To implement an image processing application, terminal devices 101, 102, 103 are connected to server 105 via network 104. Network 104 can be a wide area network or a local area network, or a combination of the two. Terminal devices 101, 102, 103 and server 105 are connected to each other via a wired or wireless network.
[0049] In some embodiments, the terminal devices 101, 102, and 103 capture images and send the captured images to the server 105. The server 105 performs super-resolution processing on the images through the super-resolution model to obtain super-resolution images, and returns the images to the terminal devices 101, 102, and 103 for display. In other embodiments, after the terminal devices 101, 102, and 103 capture images, they perform super-resolution processing on the images through the super-resolution models running in the terminal devices 101, 102, and 103 to obtain super-resolution images, and display them in the terminal devices 101, 102, and 103.
[0050] The super-resolution processed image may be a picture or a video frame, which is described below in conjunction with specific application scenarios.
[0051] In one application scenario, the APP in the terminal devices 101, 102, and 103 is an album application that stores multiple photos, and the user can select photos in batches in the album application. When the user's super-resolution control operation instruction is obtained, the terminal devices 101, 102, and 103 request the server 105 to call the super-resolution model (intelligent engine) to perform super-resolution processing to obtain photos, and the server 105 sends the super-resolution processed photos to the album application of the terminal devices 101, 102, and 103 for display. As an alternative, when the terminal devices 101, 102, and 103 obtain the user's super-resolution control operation instruction, the terminal devices 101, 102, and 103 call the super-resolution model to perform super-resolution processing to obtain photos, and display them in the album application of the terminal devices 101, 102, and 103.
[0052] It should be understood that Figure 2The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.
[0053] The following will be based on Figure 2 The scene described by Figure 3~Figure 8 The image method of the disclosed embodiment is described in detail.
[0054] Figure 3 The flowchart of the image processing method according to the embodiment of the present disclosure is schematically shown.
[0055] like Figure 3 As shown, the image processing method of this embodiment includes operations S210 to S230.
[0056] In operation S210, if a target instruction for performing resolution enhancement on a first image file is received, second image data corresponding to first image data included in the first image file is acquired.
[0057] Among them, the second image data is the original image data collected by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data.
[0058] Exemplarily, the second image data (original image data) may be information of a picture or video frame obtained by capturing a scene through a camera of the terminal; or information of a picture or video frame obtained from a local or other device or network.
[0059] The target processing strategy may be a method for improving the image resolution, such as an interpolation algorithm, a Gaussian algorithm, a network model, etc.
[0060] The first image data may be image information obtained by performing image resolution enhancement (eg, interpolation algorithm) on the original image data (second image data). The first image represented by the first image data and the second image represented by the second image data have the same image content, but different display effects.
[0061] The first image file may be formed when the first image data is saved as an image file. The first image file may include the first image data and the second image data, or include the first image data and associated information related to the second image data, and the second image data may be determined according to the associated information.
[0062] The first image data may be information of pixels in the first image, and the second image data may be information of pixels in the second image.
[0063] For example, a user captures a scene using a mobile phone camera to obtain a second image (low resolution). In the process of saving the captured second image to an album, an interpolation algorithm (target processing strategy) is performed on the second image to obtain a first image (high resolution). Finally, the first image is saved and displayed in the album. At the same time, the first image data corresponding to the first image and associated information associated with the second image are saved together as an image file of the first image.
[0064] It can be understood that since the first image data is obtained after the second image data is enhanced in image resolution, the resolution of the first image is greater than that of the second image, that is, the data volume of the first image is greater than that of the second image.
[0065] In operation S220, the second image data is input to an intelligent engine to obtain third image data.
[0066] Exemplarily, the intelligent engine may be an intelligent model capable of performing super-resolution processing on an image. For example, the original image (low resolution, small data volume) is input into the intelligent model for resolution enhancement to obtain a third image (high resolution), and the visual effect of the third image is better than that of the first image.
[0067] Here, the data amount of the third image may be equal to the data amount of the first image, or smaller than the data amount of the first image and larger than the data amount of the second image.
[0068] In operation S230, in response to the target instruction, third image data corresponding to the first image file is output.
[0069] Exemplarily, the target instruction may be an operation of the user triggering display of the third image, and the terminal displays the third image corresponding to the third image data and saves it in the terminal. When the third image and the first image are both saved in the terminal's photo album, the two images can be distinguished by naming the image files.
[0070] It is understandable that by using the original image data as the input for super-resolution processing by the intelligent engine, multiple false pixels obtained by digital interpolation in the first image can be eliminated, reducing the amount of data processed by the intelligent engine, greatly reducing the amount of calculation for super-resolution by the intelligent engine, and improving image processing efficiency. At the same time, this method uses stored image slice files for image processing, which not only ensures the smoothness of the user's normal image acquisition process, but also facilitates the user to enjoy the better visual image processing results brought by the intelligent engine.
[0071] It should be noted that the embodiment of the present disclosure does not specifically limit the number of target processing strategies, that is, the target processing strategy may be an interpolation algorithm, or a combination of an interpolation algorithm and a network model.
[0072] Figure 4The flowchart of the method for acquiring second image data according to an embodiment of the present disclosure is schematically shown.
[0073] As described above, in operation S210, second image data corresponding to the first image file is obtained. In one possible implementation, Figure 4 As shown, the operation may further include operation S211 to operation S212.
[0074] The first image file includes first image data and first description data of the first image file;
[0075] In operation S211, in the first description data, a target processing parameter indicating a target processing strategy is read.
[0076] In operation S212, the first image data is processed based on the target processing parameter to obtain second image data.
[0077] In the disclosed embodiment, the first image file includes first image data and first description data. The first description data can be understood as metadata of the first image. The metadata of an image refers to descriptive information attached to an image file, not just pixel data of the image. The metadata provides information about the background, shooting conditions, technical details, etc. of the image. The target processing parameters of the target processing strategy of the second image data (the associated information of the second image) are stored in the metadata of the first image as the first description data.
[0078] The second image data can be processed by different target processing strategies to obtain the first image data. For example, the target processing strategy can be an interpolation algorithm. There are many interpolation algorithms, such as the common linear interpolation algorithm, Lagrange interpolation algorithm, Newton interpolation algorithm, cubic spline interpolation algorithm, bilinear interpolation algorithm, triangle interpolation algorithm, etc. Different interpolation algorithms have different ways of calculating pixels. For example, a linear interpolation algorithm refers to an interpolation method in which the interpolation function is a first-order polynomial (linear function), and the interpolation error at the interpolation node is zero. When the target processing strategy adopts a linear interpolation algorithm, the target processing parameter can be understood as a linear function of the linear interpolation algorithm. For example, the pixels of the second image are processed according to the target processing strategy to obtain the pixels of the first image. When the target processing parameters (pixel processing parameters) of the target processing strategy are known, the pixels of the first image and the pixel processing parameters can be reversely deduced to obtain the pixels of the second image corresponding to the pixels of the first image.
[0079] For example, the target processing strategy is a linear interpolation algorithm. The first description data includes linear function information (target processing parameters) of the linear interpolation algorithm. The first image data (high resolution 4000*4000) is interpolated inversely according to the linear function information to obtain the original image data (low resolution 1000*1000).
[0080] It can be understood that saving the target processing parameters of the target processing strategy as the associated information of the second image data in the first image file can reduce the space occupied when the first image is saved.
[0081] Figure 5 The flowchart of the method for determining the second image data according to an embodiment of the present disclosure is schematically shown.
[0082] As described above, in operation S212, the first image data is processed based on the target processing parameter to obtain the second image data. Figure 5 As shown, the operation may further include operation S2121 to operation S2122.
[0083] The target processing parameters include inverse operation parameters and inverse effect parameters.
[0084] In operation S2121, the first image data is processed based on the inverse effect parameter to obtain fifth image data.
[0085] In operation S2122, the fifth image data is processed based on the inverse operation parameters to obtain second image data.
[0086] For example, there are many kinds of interpolation algorithms, such as the common linear interpolation algorithm, Lagrange interpolation algorithm, Newton interpolation algorithm, cubic spline interpolation algorithm, bilinear interpolation algorithm, triangle interpolation algorithm, etc. Moreover, even for the same interpolation algorithm, there are parameters for adjusting the interpolation effect. For example, in the cubic spline interpolation algorithm, the smoothness of the interpolation result and the shape of the curve can be affected by adjusting the first-order derivative and second-order derivative of the interpolation function at each data point. Common parameters include boundary conditions (such as natural boundary conditions, fixed boundary conditions, etc.), node spacing, the number of interpolation nodes, etc. By adjusting these parameters, the shape and smoothness of the interpolation result can be flexibly controlled to meet different needs. Therefore, the target processing parameters mentioned here include the type of interpolation algorithm and the parameters for adjusting the interpolation effect corresponding to the type.
[0087] When performing an inverse operation (e.g., restoring the original image from the interpolated image), the interpolated image (first image) is adjusted to obtain an image (fifth image) from which the interpolation adjustment effect is removed. The fifth image is then reversely processed according to the function of the interpolation algorithm to obtain the original image data (second image data).
[0088] It can be understood that firstly restoring the pixels in the first image data through the inverse effect parameters and then further restoring the pixels through the inverse operation parameters can ensure that the visual quality and details of the image remain consistent when the inverse operation is performed on the image, avoiding the introduction of additional errors or distortions due to improper adjustment and algorithm application.
[0089] As described above, in operation S210, second image data corresponding to the first image file is obtained. In another achievable manner, the operation may further include: reading the second image data in the first description data.
[0090] The first image file includes first image data and first description data of the first image file, and the first description data includes second image data.
[0091] For example, the first image file includes first image data and first description data, and the first description data includes second image data.
[0092] Here, the first description data can be understood as the metadata of the first image, and the second image data (pixel data) is stored in the metadata of the first image. For example, the second image data is stored in the metadata of the first image according to the XMP protocol. The second image data is hidden (invisible) in the first image file and cannot be directly displayed when the user views the first image. The second image data is stored in the first image file as attachment information of the first image data. When acquiring the second image data, it can be read out according to the rules (according to the XMP protocol) in the attachment information.
[0093] It can be understood that by directly saving the second image data of the second image in the first image file, the second image data in the first image file can be directly and quickly read, thereby improving the efficiency of obtaining the second image data.
[0094] Fig. 6A A flowchart of a method for generating a first image file according to an embodiment of the present disclosure is schematically shown; Figure 6B A schematic diagram of generating a first image file according to an embodiment of the present disclosure is schematically shown.
[0095] As described above, in operation S210, if a target instruction for performing resolution enhancement on the first image data is received, before acquiring the second image data corresponding to the first image data. Fig. 6A As shown, the operation may include operation S310a to operation S330a.
[0096] In operation S310a, fourth image data acquired by the image acquisition device is acquired.
[0097] In operation S320a, in response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is part of the fourth image data, the second image data is processed based on the target processing strategy to obtain the first image data.
[0098] In operation S330a, a first image file including first image data and a target processing parameter indicating a target processing strategy is generated.
[0099] Specifically, after the shooting application is started, the image acquisition device will continue to acquire the fourth image data and display the corresponding fourth display data on the page of the shooting application. As the user operates the shooting application, such as zooming in or out on the viewfinder screen, the image acquisition device will switch the lens to acquire the fourth image data. If the zoom-in or zoom-out ratio matches the configured lens, the fourth image data acquired by the image acquisition device will be fully displayed on the page of the shooting application. If the zoom-in ratio does not match the lens configured by the device, the image acquisition device will switch the lens to the lens closest to the current zoom ratio and crop the acquired fourth image data to obtain the second image data. And the second image data is displayed on the page of the shooting application. At this time, if the user selects the shooting operation, that is, the device detects that the shooting operation generates a storage instruction to store the fourth image data collected by the image acquisition device as an image file, the device will process the second image data through the target processing strategy to obtain the first image data, and store the obtained first image data as the first image file. In the process of storing the first image data as the first image file, the target processing parameters indicating the target processing strategy are stored in the first image file, so that other devices can confirm the processing strategy used by the second image data to the first image data by reading the target processing parameters of the first image file.
[0100] For example, refer to Figure 6B, when the camera generates photos, when the user shoots a scene through the mobile phone camera, an initial image (fourth image data, resolution 4000*4000) will be displayed on the mobile phone display screen, and the user zooms in (zooms, such as four times) a local position in the initial image to shoot, and then takes a photo to obtain a second image (second image data, resolution 1000*1000). In the process of saving the second image (original image data) to the album, the second image is processed by the interpolation algorithm to obtain the first image (first image data, resolution 4000*4000). At the same time, the interpolation algorithm identification information (i.e., the target processing parameters indicating the target processing strategy, which may include the interpolation algorithm type and the interpolation effect adjustment parameters) is stored in the MakerNotes field of the first image metadata according to the XMP protocol, and the MakerNotes field is usually included in the EXIF (Exchangeable Image File Format) information.
[0101] Correspondingly, in the stage of post-processing photos on the camera side, after the user selects the photo to be processed only with super-resolution, the terminal reads the MakerNotes field in the first image file according to the XMP protocol, obtains the interpolation algorithm identification information, and processes the first image data based on the difference algorithm identification information to obtain the original image data (second image data) corresponding to the first image data.
[0102] Fig. 7A A flowchart of a method for generating a first image file according to another embodiment of the present disclosure is schematically shown; Figure 7B A schematic diagram of generating a first image file according to another embodiment of the present disclosure is schematically shown.
[0103] As described above, in operation S210, if a target instruction for performing resolution enhancement on the first image data is received, before acquiring the second image data corresponding to the first image data. Fig. 7A As shown, the operation may include operation S310b to operation S330b.
[0104] In operation S310b, fourth image data acquired by the image acquisition device is acquired.
[0105] In operation S320b, in response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is part of the fourth image data, the second image data is processed based on the target processing strategy to obtain the first image data.
[0106] In operation S330b, a first image file including the first image data and the second image data is generated.
[0107] Exemplarily, when the image acquisition device is started, a first preview interface (fourth image data) is displayed. In a scene where image zoom is required, the user performs a first operation. The first operation includes one of a gesture operation, a screen click operation, or dragging a zoom progress bar for zooming the first preview image. For example, the user touches the screen to zoom the image to a desired magnification in the form of gesture control or clicking an image zoom switch on the photo preview interface, at which time the camera application displays a second preview interface, and based on the second preview interface, the camera captures a second image.
[0108] For example, refer to Figure 7B , in the stage of generating photos on the camera side, when the user shoots a scene through the mobile phone camera, an initial image (fourth image data, resolution 4000*4000) will be displayed on the mobile phone display screen. The user zooms in (zooms in four times) a certain local position in the initial image and takes a photo. After confirmation, the second image (second image data, resolution 1000*1000) is taken. When the second image (original image data) is saved to the album, the second image is processed by the interpolation algorithm to obtain the first image (first image data, resolution 4000*4000). At the same time, the second image data is stored in the Application Marker area of the metadata of the first image according to the XMP protocol.
[0109] In the JPG image format, there is a data segment called Application Marker. These markers are used to store information related to a specific application, such as EXIF data (metadata such as camera settings, shooting date, etc.) or color profiles.
[0110] Correspondingly, in the stage of post-processing photos on the camera side, after the user selects the photo for super-resolution processing only, the terminal reads the Application Marker area of the metadata of the first image file according to the XMP protocol to obtain the original image data (second image data) corresponding to the first image data.
[0111] It is understandable that when using the super-resolution processing model to process images, it is no longer processing the images with higher resolution stored in the terminal. Instead, it uses the real pixels (original image data) corresponding to the image to process, thereby achieving the same resolution and resolution image effect, theoretically saving 1600% of time.
[0112] Figure 8 The flowchart of a method for determining third image data according to yet another embodiment of the present disclosure is schematically shown.
[0113] As described above, in operation S220, the second image data is input into the intelligent engine to obtain the third image data. Figure 8 As shown, the operation may include operation S221 to operation S222.
[0114] In operation S221, the second image data is transmitted to the server so that the server inputs the second image data to the intelligent engine.
[0115] In operation S222, the third image data transmitted by the server is received.
[0116] In the embodiments of the present disclosure, continue to refer to Figure 2 The terminal sends the second image data to the server, and the server sends the second image data to the super-resolution processing model. The super-resolution processing model processes the second image data to obtain third image data, and sends the third image data to the server, and the server forwards the third image data to the terminal.
[0117] In other embodiments, in operation S220, the second image data is input into the intelligent engine to obtain the third image data. In another achievable manner, the operation may include: obtaining the second image data; inputting the second image data into the intelligent engine, the intelligent engine being used to process the second image data to obtain the third image; obtaining the third image.
[0118] For example, the server and the image acquisition device are integrated into the same terminal device. The server can call the model parameters of the super-resolution processing model and run the super-resolution processing model on the terminal. The server inputs the second image data into the super-resolution processing model for resolution enhancement to obtain the third image data.
[0119] It should be noted that the above operations S210 to S220, S211 to S212, S2121 to S2122, and S221 to S222 are image post-processing processes, and operations S310a to S330a and S310b to S330b are image generation processes. In one embodiment, the image generation process and the image post-processing process may be on the same device or on different devices, that is, in each embodiment consisting of operations S210 to S220, S211 to S212, S2121 to S2122, and S221 to S222 and operations S310a to S330a and S310b to S330b, the image generation process and the image post-processing process may be on the same device or on different devices.
[0120] For example, a device that performs an image generation process (referred to as a first device) obtains an image file and sends it to a device that performs an image post-processing process (referred to as a second device), so that the second device further optimizes the image file. Since the image file generated by the first device contains the original image data (second image data) or is capable of obtaining target processing parameters for the original image data (second image data), and the image file contains a larger amount of processed first image data than the original image data (second image data), the second device can input the original image data into the intelligent engine to speed up the second device to obtain third image data with better quality.
[0121] Based on the above image processing method, the present disclosure also provides an image processing device. Fig. 9 The device is described in detail.
[0122] Fig. 9 The structure block diagram of the image processing device according to the embodiment of the present disclosure is schematically shown.
[0123] like Fig. 9 As shown, the image processing device 500 of this embodiment includes an acquisition module 510 , a processing module 520 and an output module 530 .
[0124] The acquisition module 510 is used to acquire second image data corresponding to the first image data contained in the first image file if a target instruction for performing resolution enhancement on the first image file is received. In one embodiment, the acquisition module 510 can be used to perform the operation S210 described above, which will not be described in detail here.
[0125] The processing module 520 is used to input the second image data into the intelligent engine to obtain the third image data. In one embodiment, the processing module 520 can be used to perform the operation S220 described above, which will not be described in detail here.
[0126] The output module 530 is used to output the third image data corresponding to the first image file in response to the target instruction. In one embodiment, the output module 530 can be used to perform the operation S230 described above, which will not be described in detail here.
[0127] Among them, the second image data is the original image data collected by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data.
[0128] According to an embodiment of the present disclosure, any multiple modules among the acquisition module 510, the processing module 520 and the output module 530 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 510, the processing module 520 and the output module 530 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware and firmware or in any appropriate combination of any of them. Alternatively, at least one of the acquisition module 510, the processing module 520 and the output module 530 can be at least partially implemented as a computer program module, and when the computer program module is run, the corresponding function can be performed.
[0129] Fig.10 A block diagram of an electronic device suitable for implementing a display method according to an embodiment of the present disclosure is schematically shown.
[0130] The embodiment of the present disclosure discloses an electronic device, including an image acquisition device and a processor.
[0131] An image acquisition device is used to acquire image data. One or more processors are used to execute the above-mentioned image processing method.
[0132] The processor is used to, if receiving a target instruction for performing resolution enhancement on a first image file, obtain second image data corresponding to the first image data contained in the first image file; input the second image data into an intelligent engine to obtain third image data; and output the third image data corresponding to the first image file in response to the target instruction; wherein the second image data is the original image data acquired by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; and the data volume of the second image data is less than the data volume of the first image data.
[0133] like Fig.10As shown, the electronic device 400 according to the embodiment of the present disclosure includes an image acquisition device (not shown in the figure), a display screen (not shown in the figure) and a processor 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 to the random access memory (RAM) 403. The processor 401 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 401 may also include an on-board memory for caching purposes. The processor 401 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.
[0134] In RAM 403, various programs and data required for the operation of electronic device 400 are stored. Processor 401, ROM 402 and RAM 403 are connected to each other via bus 404. Processor 401 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 402 and / or RAM 403. It should be noted that the program can also be stored in one or more memories other than ROM 402 and RAM 403. Processor 401 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in one or more memories.
[0135] According to an embodiment of the present disclosure, the electronic device 400 may further include an input / output (I / O) interface 405, which is also connected to the bus 404. The electronic device 400 may further include one or more of the following components connected to the I / O interface 405: an input portion 406 including a keyboard, a mouse, etc.; an output portion 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 408 including a hard disk, etc.; and a communication portion 409 including a network interface card such as a LAN card, a modem, etc. The communication portion 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage portion 408 as needed.
[0136] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.
[0137] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 402 and / or RAM 403 described above and / or one or more memories other than ROM 402 and RAM 403.
[0138] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.
[0139] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.
Claims
1. An image processing method, comprising: If a target instruction for performing resolution enhancement on a first image file is received, acquiring second image data corresponding to first image data included in the first image file; Inputting the second image data into an intelligent engine to obtain third image data; In response to the target instruction, outputting third image data corresponding to the first image file; Among them, the second image data is the original image data collected by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data.
2. The method according to claim 1, wherein the first image file comprises first image data and first description data of the first image file, and the step of acquiring second image data corresponding to the first image file comprises: In the first description data, a target processing parameter indicating the target processing strategy is read; The first image data is processed based on the target processing parameter to obtain the second image data.
3. The method according to claim 2, wherein the target processing parameters include inverse operation parameters and inverse effect parameters; The processing of the first image data based on the target processing parameter to obtain the second image data includes: Processing the first image data based on the inverse effect parameter to obtain fifth image data; The fifth image data is processed based on the inverse operation parameters to obtain the second image data.
4. The method according to claim 1, wherein the first image file includes first image data and first description data of the first image file, the first description data includes the second image data, and the acquiring the second image data corresponding to the first image data included in the first image file comprises: In the first description data, the second image data is read.
5. The method according to claim 2, wherein: If a target instruction for performing resolution enhancement on the first image data is received, before acquiring second image data corresponding to the first image data, the method further includes: Acquiring fourth image data acquired by the image acquisition device; In response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is a part of the fourth image data, processing the second image data based on the target processing strategy to obtain the first image data; The first image file is generated including the first image data and target processing parameters indicating the target processing strategy.
6. The method according to claim 4, if a target instruction for performing resolution enhancement on the first image data is received, before acquiring the second image data corresponding to the first image data, the method further comprises: Acquiring fourth image data acquired by the image acquisition device; In response to receiving a storage instruction to store the fourth image data acquired by the image acquisition device as an image file, if the storage instruction indicates that the stored second image data is a part of the fourth image data, processing the second image data based on the target processing strategy to obtain the first image data; The first image file including the first image data and the second image data is generated.
7. The method according to claim 1, wherein the step of inputting the second image data into an intelligent engine to obtain the third image data comprises: sending the second image data to a server so that the server inputs the second image data into the intelligent engine; The third image data sent by the server is received.
8. An image processing device, comprising: an acquisition module, configured to acquire second image data corresponding to first image data included in the first image file if a target instruction for performing resolution enhancement on the first image file is received; A processing module, used for inputting the second image data into an intelligent engine to obtain third image data; an output module, configured to output third image data corresponding to the first image file in response to the target instruction; Among them, the second image data is the original image data collected by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data.
9. An electronic device, comprising: An image acquisition device, used for acquiring image data; One or more processors, configured to, if receiving a target instruction to perform resolution enhancement on a first image file, obtain second image data corresponding to first image data included in the first image file; The second image data is input into the intelligent engine to obtain the third image data; in response to the target instruction, the third image data corresponding to the first image file is output; wherein the second image data is the original image data acquired by the image acquisition device, and the first image data is the image data obtained by subjecting the original image data to a target processing strategy during the process of storing the original image data as an image file; the data volume of the second image data is smaller than the data volume of the first image data.
10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.
