Image processing method and device and electronic equipment

By acquiring the sawtooth intensity parameters and scene information of the image frame, adaptively determine the image processing algorithm, solving the problem of poor image quality improvement effect in the prior art, and achieving efficient image quality improvement on electronic devices.

CN119071531APending Publication Date: 2024-12-03VIVO MOBILE COMM CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411123193.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

When the prior art improves the image quality of electronic devices, the interpolation algorithm cannot effectively restore the picture details information, resulting in poor image quality improvement effect, especially in high-resolution and high-frame rate game scenes, which cannot meet users' requirements for high image quality.

Method used

By acquiring the sawtooth intensity parameters and image scene information of the image frame, the target image processing algorithm is determined, and the pixels in the image frame are processed, so as to adaptively optimize image quality by combining feature information of multiple dimensions.

Benefits of technology

It improves the accuracy of image processing algorithms and image quality improvement effect, and can improve the resolution and image quality of electronic devices for images without relying on high-end hardware, and meet users' needs for high-quality images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071531A_ABST
    Figure CN119071531A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device and electronic equipment, and belongs to the technical field of image.The method comprises the steps that a sawtooth intensity parameter and image scene information of a first image frame are obtained, the sawtooth intensity parameter represents the pixel smoothness of the first image frame, and the image scene information represents the image category to which the first image frame belongs; determining a target image processing algorithm based on the sawtooth strength parameter and the image scene information; and processing pixels in the first image frame based on a target image processing algorithm to obtain a second image frame.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of image technology, and specifically relates to an image processing method, device and electronic equipment. Background Art

[0002] With the popularization of high-definition devices, the total number of screen pixels of mobile phones and other electronic devices has increased to 1,500 or even higher, providing users with higher-definition picture effects.

[0003] Currently, users have high requirements for picture quality. For example, when users use mobile phones to play games, the mobile phone can first render the picture at a lower resolution, and then use an interpolation algorithm to perform interpolation calculations using the pixel values ​​of the picture to improve the resolution of the picture. Without upgrading the hardware, the pixel details in the picture can be improved through software algorithms, thereby making the mobile game screen have better picture quality and enhancing the user's gaming experience.

[0004] However, since the above interpolation algorithm only uses the pixel value of the picture to improve the resolution of the picture, it is unable to restore more detailed information of the picture, thus resulting in poor improvement effect of the electronic device on the image quality of the picture. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image processing method, device and electronic device, which can optimize the effect of improving the image quality of the electronic device.

[0006] In a first aspect, an embodiment of the present application provides an image processing method, which includes: obtaining a jaggies intensity parameter and image scene information of a first image frame, the jaggies intensity parameter characterizing the pixel smoothness of the first image frame, and the image scene information characterizing the image category to which the first image frame belongs; determining a target image processing algorithm based on the jaggies intensity parameter and the image scene information; and processing pixels in the first image frame based on the target image processing algorithm to obtain a second image frame.

[0007] In the second aspect, an embodiment of the present application provides an image processing device, which includes: an acquisition module, a determination module and a processing module; the acquisition module is used to acquire a jaggies intensity parameter and image scene information of a first image frame, the jaggies intensity parameter characterizes the pixel smoothness of the first image frame, and the image scene information characterizes the image category to which the first image frame belongs; the determination module is used to determine a target image processing algorithm based on the jaggies intensity parameter and the image scene information; the processing module is used to process pixels in the first image frame based on the target image processing algorithm to obtain a second image frame.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program / program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, a jaggies intensity parameter and image scene information of a first image frame are obtained, the jaggies intensity parameter characterizes the pixel smoothness of the first image frame, and the image scene information characterizes the image category to which the first image frame belongs; based on the jaggies intensity parameter and the image scene information, a target image processing algorithm is determined; based on the target image processing algorithm, pixels in the first image frame are processed to obtain a second image frame. In this solution, since the electronic device determines the image processing algorithm corresponding to the image frame based on the jaggies intensity parameter and the image scene information of the image frame, the electronic device determines the corresponding algorithm for a specific scene and image quality in combination with the scene of the image frame, thereby improving the accuracy of determining the image processing algorithm by combining feature information of multiple dimensions, and can achieve adaptive processing of the image frame, further optimizing the image quality improvement effect of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 This is one of the flow charts of an image processing method provided in an embodiment of the present application;

[0014] Figure 2 is a flowchart of a process of obtaining image frame information provided by an embodiment of the present application;

[0015] Figure 3 This is the second flowchart of an image processing method provided in an embodiment of the present application;

[0016] Figure 4 This is a schematic diagram of an example of displaying a setting window in a mobile game interface provided by an embodiment of the present application;

[0017] Figure 5 is a flowchart of a process of processing an image frame provided by an embodiment of the present application;

[0018] Figure 6 This is the third flowchart of an image processing method provided in an embodiment of the present application;

[0019] Figure 7 It is a schematic diagram of establishing a corresponding list according to an algorithm provided in an embodiment of the present application;

[0020] Figure 8 This is a fourth flowchart of an image processing method provided in an embodiment of the present application;

[0021] Fig. 9A It is one of the schematic diagrams of an image frame provided in an embodiment of the present application;

[0022] Fig. 9B This is a second schematic diagram of an image frame provided in an embodiment of the present application;

[0023] Fig. 9C This is a third schematic diagram of an image frame provided in an embodiment of the present application;

[0024] Fig.9D This is a fourth schematic diagram of an image frame provided in an embodiment of the present application;

[0025] Fig.10 This is the fifth flowchart of an image processing method provided in an embodiment of the present application;

[0026] Fig.11 This is the sixth flowchart of an image processing method provided in an embodiment of the present application;

[0027] Fig.12 It is one of the structural schematic diagrams of an image processing device provided in an embodiment of the present application;

[0028] Fig.13 This is a second structural diagram of an image processing device provided in an embodiment of the present application;

[0029] Fig.14 This is one of the hardware structure diagrams of an electronic device provided in an embodiment of the present application;

[0030] Fig.15 This is the second schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0032] The terms "first", "second", etc. in the specification of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0033] The terms "at least one (item)", "at least one of" and the like in the specification of the present application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0034] The following is an explanation of some concepts and / or terms involved in the image processing method, device, and electronic device provided in the embodiments of the present application.

[0035] Super-resolution algorithm, also known as super-resolution algorithm (SR), is to improve the resolution of the original image through hardware or software methods, aiming to reconstruct a high-resolution (HR) image from a low-resolution (LR) image. This technology can overcome or compensate for the problems of blurred imaging and poor image quality caused by the limitations of the image acquisition system or the acquisition environment itself, and provides important support for subsequent image processing such as feature extraction and information recognition.

[0036] The image processing method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0037] The image processing method provided in the embodiments of the present application can be applied to scenarios where an electronic device processes an image to obtain an image with higher picture quality. For example, a scenario where an electronic device processes a preview image, or a scenario where an electronic device processes a captured image, or a scenario where an electronic device processes a video image, or a scenario where an electronic device processes a real-time image, such as a game image or a live broadcast image.

[0038] Taking the processing of game images by electronic devices as an example, in the prior art, in a scenario where a user runs a game through an electronic device, since the user has a high degree of demand for the image quality of the game image, the electronic device can improve the resolution of the game image in real time by running a super-resolution algorithm in real time.

[0039] At present, although PC devices have relatively mature game super-resolution algorithms, such as FSR and DLSS series, the above algorithms have too much overhead during operation and can only be run on Nvidia high-end graphics cards, that is, the hardware requirements for the electronic devices running the algorithms are extremely high. Therefore, the above algorithms cannot be generally applied to the real-time operation of high-resolution and high-frame rate games on mobile devices, where high resolution can be a total of 1500 columns of pixels on the game screen, that is, 1.5K; high frame rate can be a frame rate of 120FPS+ for the game screen.

[0040] Therefore, when a high-resolution and high-frame-rate game is running in real time on a mobile device, the mobile device can run an interpolation algorithm in real time to improve the resolution of the game screen. This allows the software algorithm to improve the pixel details in the screen without upgrading the hardware, so that the game screen of the mobile device has better picture quality and increases the user's gaming experience.

[0041] However, existing interpolation algorithms, such as nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation, only use the spatial relationship between pixels of the picture to approximately estimate the lost details, so as to improve the resolution of the picture by interpolating the pixels of the low-resolution image, but cannot restore more detailed information of the picture, and cannot handle complex degraded scenes such as aliasing and blurring. As a result, the electronic device has a poor picture enhancement effect and cannot meet the user's high-quality requirements for mobile games.

[0042] In the image processing method, device and electronic device provided in the embodiments of the present application, the jaggies intensity parameter and image scene information of the first image frame are obtained, the jaggies intensity parameter characterizes the pixel smoothness of the first image frame, and the image scene information characterizes the image category to which the first image frame belongs; based on the jaggies intensity parameter and the image scene information, a target image processing algorithm is determined; based on the target image processing algorithm, pixels in the first image frame are processed to obtain a second image frame. In this solution, since the electronic device determines the image processing algorithm corresponding to the image frame based on the jaggies intensity parameter and the image scene information of the image frame, the electronic device determines the corresponding algorithm for a specific scene and image quality in combination with the scene of the image frame, thereby improving the accuracy of determining the image processing algorithm by combining feature information of multiple dimensions, and can achieve adaptive processing of the image frame, further optimizing the image quality improvement effect of the electronic device.

[0043] The image processing method provided in the embodiment of the present application is executed by an image processing device, which may be an electronic device, or a functional module or entity in an electronic device, and the embodiment of the present application does not limit this. The image processing method provided in the embodiment of the present application will be exemplarily described below using an electronic device as an example.

[0044] The present application provides an image processing method. Figure 1 FIG. 1 is a flow chart of an image processing method provided by an embodiment of the present application. Figure 1 As shown, the image processing method provided in the embodiment of the present application may include the following steps 201 to 203.

[0045] Step 201: The electronic device obtains a jaggies intensity parameter and image scene information of a first image frame.

[0046] In the embodiment of the present application, the first image frame is the image frame currently displayed by the electronic device.

[0047] It should be noted that the electronic device starts to obtain the aliasing intensity parameter and image scene information of the first image frame after the first image frame is rendered, and does not need to be performed during the rendering process of the image frame. Therefore, the electronic device can realize image processing of the image frame by running a graphics processing unit (GPU), without relying on a specific hardware graphics card or modification of the image rendering pipeline. In this way, since there is no limitation on the hardware of the electronic device, the image processing method provided by the present application can be generally applicable to any electronic device.

[0048] In the embodiment of the present application, the above-mentioned jaggies intensity parameter represents the pixel smoothness of the first image frame.

[0049] It should be noted that if the aliasing intensity parameter of the image frame is higher, the aliasing degree of the edge of the object in the image frame is higher, and the pixel smoothness of the image frame is lower. If the aliasing intensity parameter of the image frame is lower, the aliasing degree of the edge of the object in the image frame is lower, and the pixel smoothness of the image frame is higher.

[0050] In some embodiments of the present application, the above-mentioned sawtooth strength parameter can be represented by a specific numerical value or by a level.

[0051] For example, when the image aliasing intensity is represented by a specific numerical value, the numerical value may be: the number of image aliasing or the image aliasing ratio, wherein the image aliasing ratio is the ratio of the number of image aliasing in the image frame to the number of pixels in the image frame.

[0052] Exemplarily, when the above-mentioned jaggies intensity parameter is represented by a level, the level can be divided into three levels: "high", "medium" and "low". Specifically, when the jaggies intensity parameter of the image frame is greater than or equal to the first threshold, the level corresponding to the jaggies intensity parameter of the image frame is "high"; when the jaggies intensity parameter of the image frame is greater than or equal to the second threshold and less than the first threshold, the level corresponding to the jaggies intensity parameter of the image frame is "medium"; when the jaggies intensity parameter of the image frame is less than the second threshold, the level corresponding to the jaggies intensity parameter of the image frame is "low". The settings of the first threshold and the second threshold can be determined according to the actual needs of the user and are not limited here.

[0053] For example, assuming that the pixel size of the image frame is 100*100, the first threshold is 300, and the second threshold is 50. Therefore, when the number of image jaggies in the image frame is greater than or equal to 300, the level corresponding to the jaggies intensity parameter of the image frame is "high"; when the number of image jaggies in the image frame is greater than or equal to 50 and less than 300, the level corresponding to the jaggies intensity parameter of the image frame is "medium"; when the number of image jaggies in the image frame is less than 50, the level corresponding to the jaggies intensity parameter of the image frame is "low".

[0054] It should be noted that the number of specific levels of division can be determined according to the specific jaggies intensity parameters of the image frame and the actual needs of the user, and is not limited here.

[0055] In the embodiment of the present application, the above-mentioned image scene information represents the image category to which the first image frame belongs.

[0056] In some embodiments of the present application, the above-mentioned image scene information may include but is not limited to at least one of the following: text information, character information, building information, and natural scene information.

[0057] Exemplarily, the above text information can be understood as: the first image frame displayed by the electronic device contains text content, such as game announcements, plot introductions, story background introductions, etc.

[0058] Exemplarily, the above-mentioned character information can be understood as: information about a character object contained in the first image frame displayed by the electronic device.

[0059] Exemplarily, the above-mentioned building information can be understood as: the building corresponding to the content of the first image frame displayed by the electronic device, such as a palace, a courtyard, a castle, a campus, a hospital, etc.

[0060] Exemplarily, the above-mentioned natural scene information can be understood as: the background environment corresponding to the content of the first image frame displayed by the electronic device, for example, the sky, grassland, forest, ocean, etc.

[0061] In some embodiments of the present application, when the first image frame is a game screen or a game live broadcast screen, the above-mentioned image scene information may be a game battle scene, such as a personal battle scene, a team battle scene, a hand battle scene, and the like.

[0062] In some embodiments of the present application, the above-mentioned image categories may include but are not limited to at least one of the following: text category, character category, and landscape category. Optionally, these image categories are all image categories in game scenes.

[0063] Exemplarily, assuming that the content displayed by the image frame is "a user standing on a grassland", that is, the image frame contains person information and natural scenery information, the electronic device can determine that the image category to which the image frame belongs is the person category.

[0064] As another example, assuming that the content displayed by the image frame is “there is a cloud in the sky”, that is, the image frame contains natural scene information, the electronic device may determine that the image category to which the image frame belongs is the landscape category.

[0065] In some embodiments of the present application, Figure 2 As shown, the electronic device can acquire the first image frame to perform image quality estimation on the first image frame, such as jaggies analysis processing and scene recognition processing, to obtain jaggies intensity parameters and image scene information corresponding to the first image frame.

[0066] In some embodiments of the present application, the above step 201 is specifically implemented by the following steps 201a and 201b.

[0067] Step 201a: The electronic device performs aliasing analysis on the first image frame to obtain a aliasing intensity parameter.

[0068] In some embodiments of the present application, the electronic device may input the first image frame into a jaggy analysis model to perform jaggy analysis processing on the first image frame through the jaggy analysis model to obtain a jaggy intensity parameter of the first image frame.

[0069] In some embodiments of the present application, the electronic device may first perform grayscale processing on the first image frame through a sawtooth analysis model to obtain a first grayscale image corresponding to the first image frame, and then obtain the sawtooth intensity parameter of the first image frame based on the grayscale value of the first grayscale image.

[0070] It should be noted that the grayscale value of the grayscale image refers to: the grayscale value corresponding to the image content in the first grayscale image, and the image content may be part of the image content or all of the image content in the first grayscale image.

[0071] In some embodiments of the present application, when the electronic device determines the aliasing intensity parameter based on the grayscale value of part of the image content in the first grayscale image, the electronic device can determine the edge area of ​​the object contained in the first image frame based on the difference between the grayscale values ​​of each two adjacent pixels in the first grayscale image, and then obtain the amount of image aliasing contained in the edge area as the aliasing intensity parameter of the first image frame.

[0072] In some embodiments of the present application, when the difference between the grayscale values ​​of two adjacent pixels is greater than or equal to a preset difference, the electronic device may determine that the two adjacent pixels are pixels of a local edge of an object. For example, the preset difference may be 50.

[0073] Step 201b: The electronic device performs scene recognition processing on the first image frame to obtain image scene information.

[0074] In some embodiments of the present application, the electronic device may input the first image frame into a scene recognition model to perform scene recognition processing on the first image frame through the scene recognition model to obtain image scene information of the first image frame.

[0075] In some embodiments of the present application, the electronic device may obtain the object contained in the first image frame through a scene recognition model, and then obtain the image scene information of the first image frame according to the scene to which the object belongs.

[0076] Exemplarily, the electronic device detects that the first image frame contains a blue background, birds, and white irregular objects, and can determine that the blue background is the blue sky and the white irregular objects are white clouds, so the electronic device can determine that the image scene information of the first image frame is a sky scene.

[0077] As another example, the electronic device detects that the objects contained in the first image frame are: a table, a blackboard, and a book. Then, it can be determined that the scene to which the objects contained in the first image frame belong is a campus scene, and the electronic device can determine that the image scene information of the first image frame is a campus scene.

[0078] In some embodiments of the present application, the electronic device may detect text contained in the first image frame to obtain key words therein, and then determine the image scene information of the first image frame according to the scene to which the key words belong.

[0079] Exemplarily, assuming that the first image frame is a real-time game screen, and the first image frame contains the text "Please move from the classroom to the playground", the electronic device can obtain the keywords "classroom" and "playground" and determine that the scenes to which the "classroom" and "playground" belong are campus scenes. Therefore, the electronic device can determine that the image scene information of the first image frame is a campus scene.

[0080] In an embodiment of the present application, the electronic device detects the first image frame to obtain the aliasing intensity parameter and image scene information of the first image frame. In this way, the accuracy of the subsequent image processing algorithm can be improved by acquiring feature information of multiple dimensions of the first image frame.

[0081] It should be noted that in the embodiment of the present application, the execution order of the above step 201a and step 201b is not specifically limited. For example, step 201a can be executed before step 201b, or step 201a can also be executed after step 201b, or step 201a can also be executed simultaneously with step 201b.

[0082] In some embodiments of the present application, Figure 1 ,like Figure 3 As shown, before the above step 201, the image processing method provided in the embodiment of the present application further includes the following step 301, and the above step 201 can be specifically implemented by the following step 201c.

[0083] Step 301: While displaying a first image frame, the electronic device receives a first input from a user.

[0084] In an embodiment of the present application, the above-mentioned first image frame is an image frame in a game screen, or an image frame in a live broadcast screen.

[0085] In the embodiment of the present application, the first input is used to trigger the electronic device to start an image processing function to process the first image frame.

[0086] In some embodiments of the present application, the first input is an input to a designated physical key or a designated physical key combination of the electronic device; or, the first input is an input to a designated icon or control displayed on the screen of the electronic device.

[0087] In some embodiments of the present application, the above-mentioned first input may include any of the following: user's click input, sliding input, pressing input, voice input, gesture input, or other feasible input, which is not limited in the embodiments of the present application.

[0088] In some embodiments of the present application, the first input may include a first sub-input and a second sub-input. The first sub-input may be a user pressing a physical key to trigger the electronic device to display an image processing identifier; the second sub-input may be a user inputting the image processing identifier to trigger the electronic device to start acquiring the sawtooth intensity parameter and image scene information of the first image frame.

[0089] Step 201c: The electronic device obtains a jaggies intensity parameter and image scene information of the first image frame in response to the first input.

[0090] For example, the electronic device is a mobile phone. When a user is playing a mobile game on a mobile phone, if the user wants to adjust the clarity of the game screen displayed on the mobile phone, the user can press and hold the "volume up key" of the mobile phone, that is, the first sub-input mentioned above; Figure 4 As shown, the mobile phone can display a "Game Super Resolution Settings" window 10 on the screen to display game image processing marks, namely, a "Game Super Resolution (On)" mark 11 and a "Game Super Resolution (Off)" mark 12. In this way, the user can click the "Game Super Resolution (On)" mark 11, i.e., the second sub-input, to trigger the mobile phone to obtain the jagged intensity parameters and image scene information of the currently displayed game screen.

[0091] In the embodiment of the present application, when the first image frame is an image frame in a game screen or an image frame in a live broadcast screen, the electronic device can obtain the sawtooth intensity parameter and image scene information of the first image frame after receiving the user's input, so that the user can control the start time of the electronic device to perform image processing according to actual needs, thereby improving the convenience of image processing and improving the user's gaming experience. At the same time, it can also avoid the power consumption caused by the electronic device automatically performing image processing on the game screen when the user does not trigger the image processing function to start.

[0092] Step 202: The electronic device determines a target image processing algorithm based on the jaggies intensity parameter and the image scene information.

[0093] In some embodiments of the present application, the target image processing algorithm may be one of at least one algorithm in the electronic device, and the at least one algorithm may be an image processing algorithm for at least one specific scene generated by training according to a specific image scene. The specific training implementation method may refer to the description in the following embodiments, which will not be described in detail here.

[0094] In some embodiments of the present application, each of the at least one algorithm mentioned above has the capabilities of anti-aliasing and image enhancement at the same time.

[0095] It should be noted that the above-mentioned anti-aliasing (AA) refers to a technology that eliminates the jagged edges of images in the image output by the display, softens the edges of the image, makes the edges of the image look smoother and closer to the real object, and can improve the image quality of the image frame to make it softer.

[0096] It should be noted that the above-mentioned image enhancement is to add some information or transform data to the original image by certain means, selectively highlight the features of interest in the image, or suppress / mask certain unnecessary features in the image, which can improve the visual effect of the image.

[0097] In some embodiments of the present application, the at least one algorithm mentioned above may be a general image processing algorithm, or at least one image processing algorithm for a specific scene generated by training according to a specific scene.

[0098] It can be understood that, for any image frame, the electronic device can determine a corresponding algorithm for the arbitrary image frame according to the jaggies intensity parameter and image scene information of the arbitrary image frame, so as to perform image processing on the arbitrary image frame.

[0099] In some embodiments of the present application, the electronic device may first perform a first screening on at least one of the above-mentioned algorithms according to the jaggies intensity parameters of the first image frame to screen the algorithm that matches the jaggies intensity parameters of the first image frame, and then perform a second screening on the screened algorithm according to the image scene information of the first image frame to determine the target image processing algorithm that matches the first image frame.

[0100] Exemplarily, it is assumed that the at least one algorithm includes 6 algorithms, namely, Algorithm 1 to Algorithm 6. Among them, Algorithm 1 is used to process an image frame whose aliasing intensity parameter is "high" and whose image scene information is the sky, Algorithm 2 is used to process an image frame whose aliasing intensity parameter is "medium" and whose image scene information is the sky, Algorithm 3 is used to process an image frame whose aliasing intensity parameter is "low" and whose image scene information is the sky, Algorithm 4 is used to process an image frame whose aliasing intensity parameter is "high" and whose image scene information is grassland, Algorithm 5 is used to process an image frame whose aliasing intensity parameter is "medium" and whose image scene information is grassland, and Algorithm 6 is used to process an image frame whose aliasing intensity parameter is "low" and whose image scene information is grassland. If the aliasing intensity parameter of the first image frame is "medium" and whose image scene information is the sky, the electronic device may first perform a first screening of the above 6 algorithms according to the aliasing intensity parameter "medium" to obtain Algorithm 2 and Algorithm 4, and then the electronic device may perform a second screening according to the image scene information of the sky to determine Algorithm 2 that matches the first image frame.

[0101] In some embodiments of the present application, the electronic device may first perform a first screening on at least one of the above algorithms based on the image scene information of the first image frame to screen the algorithm that matches the image scene information of the first image frame, and then perform a second screening on the screened algorithm based on the aliasing intensity parameter of the first image frame to determine the target image processing algorithm that matches the first image frame.

[0102] Exemplarily, in combination with Algorithms 1 to 6 in the above examples, if the aliasing intensity parameter of the first image frame is "medium" and the image scene information is the sky, the electronic device can first perform a first screening of the above 6 algorithms according to the image scene information being the sky to obtain Algorithms 1, 2, and 3, and then the electronic device performs a second screening according to the aliasing intensity parameter "medium" to determine Algorithm 2 that matches the first image frame.

[0103] In some embodiments of the present application, Figure 2 ,like Figure 5 As shown, after the electronic device obtains the aliasing intensity parameters and image scene information of the first image frame, the electronic device can make an algorithmic decision to determine the image processing algorithm corresponding to the first image frame, that is, the target image processing algorithm, and then subsequently process the pixels in the first image frame based on the target image processing algorithm to obtain the second image frame.

[0104] Step 203: The electronic device processes pixels in the first image frame based on a target image processing algorithm to obtain a second image frame.

[0105] In the embodiment of the present application, the resolution of the second image frame is greater than the resolution of the first image frame.

[0106] In the embodiment of the present application, since the target image processing algorithm has the ability to both resist aliasing and enhance image quality, the aliasing intensity of the second image frame is lower than the aliasing intensity of the first image frame, and the image quality of the second image frame is better than the image quality of the first image frame.

[0107] For example, in combination Figure 2 ,like Figure 5 As shown, after the electronic device obtains the aliasing intensity parameters and image scene information of the first image frame, the electronic device can make an algorithmic decision to determine the image processing algorithm corresponding to the first image frame, that is, the target image processing algorithm, and then process the pixels in the first image frame based on the target image processing algorithm to obtain the second image frame.

[0108] It should be noted that the electronic device can process all pixels in the first image frame based on the target image processing algorithm to obtain a processed second image frame. The specific processing process can refer to the subsequent steps 203a to 203d and the contents of related steps, which are not repeated here.

[0109] In the image processing method performed by the electronic device provided in the embodiment of the present application, since the electronic device determines the image processing algorithm corresponding to the image frame based on the jaggies intensity parameter and image scene information of the image frame, the electronic device determines the corresponding algorithm for the specific scene and image quality in combination with the scene of the image frame, thereby improving the accuracy of determining the image processing algorithm by combining feature information of multiple dimensions, and can achieve adaptive processing of the image frame, further optimizing the image quality improvement effect of the electronic device.

[0110] In some embodiments of the present application, Figure 1 ,like Figure 6 As shown, the above step 203 can be specifically implemented through the following steps 203a to 203d.

[0111] Step 203a: The electronic device determines a list corresponding to the target image processing algorithm based on the target image processing algorithm.

[0112] In the embodiment of the present application, the list corresponding to the above-mentioned target image processing algorithm is a look-up table (LUT) of the pixel values ​​corresponding to the pixel values ​​processed by the target processing algorithm found according to the pixel values.

[0113] In an embodiment of the present application, the list corresponding to the above-mentioned target image processing algorithm includes at least one first identifier and at least one second identifier, each first identifier represents a pixel value of a pixel block of a preset size, and each second identifier represents a pixel value obtained after a pixel block of a preset size is processed based on the target image processing algorithm.

[0114] It can be understood that the at least one first identifier and the at least one second identifier correspond one to one.

[0115] In some embodiments of the present application, the pixel block of the preset size is a square pixel block composed of p*p pixels. Where p is a positive integer greater than or equal to 2. Alternatively, the pixel block of the preset size is a rectangular pixel block of a*b. Where a and b are positive integers greater than or equal to 2, and a and b are not equal. It should be noted that the following embodiments are illustrated by taking the pixel block of the preset size as a square pixel block composed of p*p pixels as an example.

[0116] In some embodiments of the present application, the electronic device may construct a p*p pixel block, and assign a value to each pixel point in the pixel block to obtain the pixel value of at least one pixel block represented by the above-mentioned at least one identifier. It should be noted that, since the range of pixel values ​​is 0 to 255, that is, a total of 256 values, there are 256^(p*p) situations for the pixel value of the at least one pixel block, that is, the number of the above-mentioned at least one first identifier is 256^(p*p).

[0117] In some embodiments of the present application, the electronic device processes the pixel values ​​of the pixel blocks in 256^(p*p) situations through the target image processing algorithm, thereby obtaining the pixel values ​​of the pixel blocks in 256^(p*p) situations after processing, that is, the pixel values ​​of the pixel blocks represented by the second identifier. In other words, the number of the at least one second identifier is 256^(p*p).

[0118] For example, if p=2, that is, a pixel block is formed by 2*2 pixel points, then there are 256^4 situations for the pixel value of the pixel block, and the electronic device can perform image processing on the pixel values ​​of the 256^4 pixel blocks through the target image processing algorithm, thereby obtaining 256^4 pixel values ​​after the pixel block processing. Therefore, the list corresponding to the target image processing algorithm includes 256^4 first identifiers and 256^4 first identifiers.

[0119] In some embodiments of the present application, the first identifier may be an index value used to characterize a pixel value of a pixel block, and a representation form of the index value is related to the pixel value of the pixel block.

[0120] In some embodiments of the present application, the first identifier corresponding to each pixel block may be represented in a first manner, where the first manner is a representation manner determined based on all pixel values ​​in each pixel block.

[0121] For example, Figure 7As shown, all pixel values ​​in a 2*2 pixel block are [I1, I2, I3, I4], and the pixel values ​​in the 2*2 pixel block obtained after processing based on the target image processing algorithm are [O1, O2, O3, O4]. Then the first identifier corresponding to the pixel block can be expressed as LUT [I1] [I2] [I3] [I4], and the second identifier can be expressed as LUT [I1] [I2] [I3] [I4] [O1, O2, O3, O4].

[0122] For example, all pixel values ​​in a 2*2 pixel block are [I1, I2, I3, I4], and the pixel values ​​in the 2*2 pixel block are [O1, O2, O3, O4] after being processed by an algorithm. Then the first identifier corresponding to the pixel block can be expressed as LUT [I1] [I2] [I3] [I4], and the second identifier can be expressed as LUT [I1] [I2] [I3] [I4] [O1, O2, O3, O4].

[0123] For example, all pixel values ​​in a 2*2 pixel block are [I1, I2, I3, I4], and the pixel values ​​in the 2*2 pixel block are [O1, O2, O3, O4] after being processed by an algorithm. Then the first identifier corresponding to the pixel block can be represented as a specific numerical value, that is, the index value X=I1*10^9+I2*10^6+I3+10^3+I4, and the second identifier can be represented as LUT[X][O1, O2, O3, O4].

[0124] It should be noted that the pixel values ​​of the pixel blocks in each case correspond to different index values. It can be understood that the pixel values ​​of the pixel blocks in 256^(p*p) cases correspond to 256^(p*p) index values ​​respectively. The specific method of setting the index values ​​is not limited here.

[0125] In some embodiments of the present application, the above-mentioned obtaining of the list corresponding to the target image processing algorithm may be: the electronic device reads the stored list corresponding to the target image processing algorithm. It should be noted that, in the case where the list corresponding to the target image processing algorithm is stored in the electronic device, the list corresponding to the target image processing algorithm may be updated as the system of the electronic device is updated.

[0126] In some embodiments of the present application, Figure 6 ,like Figure 8 As shown, after the above step 203a, the image processing method provided in the embodiment of the present application further includes the following steps 401 and 402.

[0127] Step 401: The electronic device adds at least one row of pixel points with preset pixel values ​​below the last row in the first image frame, and adds at least one column of pixel points with preset pixel values ​​to the right of the last column in the first image frame.

[0128] It can be understood that since the electronic device processes the corresponding pixel values ​​in units of pixel blocks of preset size, in order to form pixel blocks of preset size in the process of processing the pixel points in the edge area of ​​the first image frame, the electronic device can add at least one row of pixel points of preset pixel value below the last row in the first image frame, and add at least one column of pixel points of preset pixel value to the right of the last column in the first image frame. Thus, each pixel point in the first image frame can form a pixel block of preset size with adjacent pixel points, so as to perform the subsequent image processing process.

[0129] In some embodiments of the present application, the above-mentioned preset pixel value may be 0.

[0130] In some embodiments of the present application, the number of rows and columns of pixels added by the electronic device to the first image frame is related to the preset size of the pixel block represented by the first identifier and the second identifier. For example, when the pixel block of the preset size is a p*p pixel block, the electronic device can add (p-1) rows of pixel points with preset pixel values ​​at the bottom of the last row in the first image frame, and add (p-1) columns of pixel points with preset pixel values ​​at the right of the last column in the first image frame.

[0131] For example, Fig. 9A As shown, assuming that the first image frame is a 3*4 image frame, and the pixel block of the preset size is a 2*2 pixel block, the electronic device can add a row of pixels with a pixel value of 0 below the last row of pixels in the first image frame, and add a column of pixels with a pixel value of 0 to the right of the last column of the first image frame, such as Fig. 9B As shown, a first image frame of 4*5 is obtained after adding pixels.

[0132] Step 402: The electronic device divides all the pixels in the first image frame after the pixels are added into at least one pixel block of a preset size according to a preset size.

[0133] In some embodiments of the present application, the electronic device may divide all pixels in the first image frame after adding pixels into at least one pixel block of a preset size from top to bottom and from left to right.

[0134] For example, it is assumed that the pixel block of the preset size is a 2*2 pixel block. The electronic device can use the pixel point a11 of the first row and the first column in the first image frame as a reference, and combine Fig. 9A ,like Fig. 9C As shown, the electronic device first determines three pixel points adjacent to the pixel point a11, namely, the pixel point a12, the pixel point a21 and the pixel point a22, to obtain a 2*2 pixel block; then, the electronic device can use the pixel point a12 in the first row and second column in the first image frame as a reference to determine the pixel points adjacent to the pixel point a12, namely, the pixel point a13, the pixel point a22 and the pixel point a23, to obtain a 2*2 pixel block; further, the electronic device divides from left to right and from top to bottom according to the first image frame after adding the pixel point to obtain 12 2*2 pixel blocks. By analogy, the pixel block corresponding to each pixel point is obtained.

[0135] For the pixel points in the last column of the first image frame, with the pixel point a14 in the first row and fourth column of the first image frame as a reference, the electronic device can determine the adjacent pixel points for a14, namely, the pixel point b15, the pixel point a24 and the pixel point b25, so as to obtain a 2*2 pixel block. For the pixel points in the last row of the first image frame, with the pixel point a31 in the third row and first column of the first image frame as a reference, the electronic device can determine the adjacent pixel points for the pixel point a31, namely, the pixel point a32, the pixel point b41 and the pixel point b42, so as to obtain a 2*2 pixel block. At the same time, for the pixel point a34 in the last row and the last column of the first image frame, the electronic device can determine the adjacent pixel points for the pixel point a34, namely, the pixel point b35, the pixel point b44 and the pixel point b45, so as to obtain a 2*2 pixel block.

[0136] In the embodiment of the present application, in the process of adjusting the pixel value of a pixel point, the electronic device performs a subsequent image processing process by combining the pixel values ​​of all pixel points adjacent to the pixel point, wherein all the adjacent pixel points may include those adjacent below, adjacent to the right, and adjacent to the diagonal. The electronic device can increase the pixel points so that each pixel point in the first image frame can form a pixel block of a preset size with the adjacent pixel points, thereby avoiding the situation where the pixel points at the edge position have no adjacent pixel points or the number of adjacent pixel points is insufficient. In this way, the subsequent image processing process can be performed, which improves the convenience of the electronic device in performing image processing, and can improve the processing effect of the edge area of ​​the image frame.

[0137] Step 203b, for each pixel block of a preset size in the first image frame, the electronic device determines a first identifier corresponding to the first pixel block from a list corresponding to the target image processing algorithm, and determines a second identifier corresponding to the first pixel block based on the first identifier corresponding to the first pixel block.

[0138] In the embodiment of the present application, the first pixel block is a pixel block of any preset size of the first image frame.

[0139] It can be understood that the electronic device can divide the first image frame into multiple pixel blocks of preset sizes from top to bottom and from left to right based on the first image frame, that is, each pixel point corresponds to a pixel block of preset size, and determine the first identifier corresponding to the first pixel block based on the pixel value of the first pixel block, and then determine the second identifier corresponding to the first identifier in the list corresponding to the target image processing algorithm.

[0140] Step 203c: The electronic device replaces the pixel value of the first pixel point in the first pixel block based on the pixel value represented by the second identifier corresponding to the first pixel block.

[0141] For example, in combination Fig. 9C , the electronic device first determines a pixel block including pixel point a11, pixel point a12, pixel point a21 and pixel point a22, that is, the first pixel block mentioned above, and determines the corresponding first identifier, such as index value 1002003004, according to the pixel value of the first pixel block, such as [1,2,3,4]. Then the electronic device can determine the corresponding second identifier according to the index value in the list corresponding to the target image processing algorithm, and further determine the pixel value processed based on the target image processing algorithm, such as [2,2,2,2]. Therefore, the electronic device can replace the pixel value corresponding to pixel point a11 with [2,2,2,2], thereby realizing image processing of pixel point a11.

[0142] Again illustratively, in combination Fig. 9A , the electronic device first determines a pixel block including pixel a11, pixel a12, pixel a21 and pixel a22, that is, the first pixel block mentioned above, and determines the corresponding first identifier according to the pixel value of the first pixel block, and then the electronic device can determine the corresponding second identifier according to the index value in the list corresponding to the target image processing algorithm, and further determine the pixel value after processing based on the target image processing algorithm, such as the pixel value of the pixel block composed of pixel c11-1, pixel c11-2, pixel c11-3 and pixel c11-4. Thus, Fig.9D As shown, the electronic device can replace the pixel value of the pixel point a11 with the pixel value of the pixel block composed of the pixel point c11-1, the pixel point c11-2, the pixel point c11-3 and the pixel point c11-4.

[0143] Step 203d: The electronic device traverses all pixel blocks of a preset size of the first image frame, and replaces the pixel value of the first pixel point in all pixel blocks of the preset size to obtain a second image frame.

[0144] It can be understood that for each pixel block of a preset size in the first image frame, the pixel value of the first pixel in each pixel block of the preset size can be replaced by executing the above-mentioned step 402 and steps 203b to 203c, thereby achieving replacement processing for each pixel in the first image frame.

[0145] In some embodiments of the present application, the number of pixels included in the second image frame is p*p times the number of pixels included in the first image frame.

[0146] For example, Fig.9D As shown in the figure, since each pixel in the first image frame is replaced by a 2*2 pixel block, the number of pixels in each row of the second image frame is twice that of the first image frame, that is, each row in the second image frame contains 8 pixels; the number of pixels in each column of the second image frame is twice that of the first image frame, that is, each column in the second image frame contains 6 pixels. Therefore, the number of pixels contained in the second image frame is 4 times the number of pixels contained in the first image frame.

[0147] In the embodiment of the present application, since the list corresponding to the target image processing algorithm pre-stores the pixel values ​​of the pixel blocks obtained after processing corresponding to all situations of the pixel blocks, the image processing process can be achieved by searching for corresponding data in the list corresponding to the target image processing algorithm and replacing the pixel values ​​of the pixels. In this way, there is no need for the electronic device to perform floating-point calculations in real time through the target image processing algorithm to obtain the processed pixel values, thereby improving the efficiency of image processing of the electronic device. At the same time, since the power consumption of obtaining the processed results through searching is extremely low, the power consumption caused by real-time floating-point calculations can be reduced.

[0148] In some embodiments of the present application, the target image processing algorithm is an image algorithm corresponding to the aliasing intensity parameter and the image scene information among the M image algorithms. Figure 1 ,like Fig.10 As shown, before the above step 201, the image processing method provided in the embodiment of the present application also includes the following steps 501 to 504.

[0149] Step 501: The electronic device obtains a training data set.

[0150] In an embodiment of the present application, the above-mentioned training data set includes N image frames of at least one image category, where N is an integer greater than 1.

[0151] In some embodiments of the present application, the N image frames of at least one image category include the following three situations: each image category includes one image frame; each image category includes multiple image frames; a portion of image categories includes one image frame, and another portion of image categories includes multiple image frames. The present application embodiment is not limited here.

[0152] It should be noted that the image categories here can refer to the description in the above embodiment and will not be described in detail here.

[0153] Step 502: The electronic device obtains N image scene information and N aliasing intensity parameters of N image frames based on the training data set.

[0154] In some embodiments of the present application, the N image frames, the N image scene information, and the N jaggies intensity information correspond one to one.

[0155] In some embodiments of the present application, the electronic device may perform image detection on each of the N image frames to obtain at least one image scene information and at least one aliasing intensity parameter corresponding to the N image frames. The specific implementation method is similar to the method for obtaining the image scene information and aliasing intensity parameter of the first image frame, and can refer to the above steps 201a and 201b, which will not be described in detail here.

[0156] Step 503: The electronic device groups the N image frames based on the N image scene information and the N jaggies intensity parameters to obtain M image frame groups.

[0157] Wherein, M is a positive integer.

[0158] In some embodiments of the present application, the electronic device can divide image frames whose image categories represented by image scene information match and whose aliasing strength parameters meet the same level into the same group of image frames, i.e., one image frame group, thereby obtaining M image frame groups.

[0159] Exemplarily, assuming that the training data set includes 10 image frames, namely, image frames 1 to 10, the electronic device can determine the corresponding image scene information and aliasing strength information based on the 10 image frames. For example, the image category represented by the image scene information of image frames 1 to 3 is sky, and the image category represented by the image scene information of image frames 4 to 10 is grassland. Among them, the aliasing strength information of image frames 1, 4 to 6 is "high", the aliasing strength information of image frame 7 is "medium", and the aliasing strength information of image frames 2, 3, 8 to 10 is "low". Therefore, the electronic device can group the 10 image frames to obtain 5 image frame groups, namely <sky, high>, <sky, low>, <grassland, high>, <grassland, medium> and <grassland, low>. Among them, the image frame group <sky, high> includes image frame 1, <sky, low> includes image frames 2 and 3, the image frame group <grassland, high> includes image frames 4 to 6, the image frame group <grassland, middle> includes image frame 7, and the image frame group <grassland, low> includes image frames 8 to 10.

[0160] Step 504: The electronic device obtains an image algorithm based on each image frame group training to obtain M image algorithms.

[0161] It can be understood that the above-mentioned M image frame groups correspond one-to-one to the M image algorithms.

[0162] In some embodiments of the present application, the electronic device trains an image processing algorithm that matches the image frames in each image frame group, so that through the image processing model, the image frames that match the image scene information and aliasing intensity parameters corresponding to the image frame group can be processed in a targeted manner.

[0163] In some embodiments of the present application, for each image frame group, the electronic device can perform image processing on each image frame in an image frame group based on the initial image processing model, and calculate a loss function according to the processing result of each image frame and the corresponding reference processing result, and iteratively update the parameters in the initial image processing model by back propagation through the loss function until convergence, so as to obtain an image processing model corresponding to the image frame group, thereby obtaining an image algorithm corresponding to the image frame group. Then, each image frame group in the M image frame groups in the training data set is traversed to obtain an image algorithm corresponding to each image frame group, thereby obtaining M image algorithms.

[0164] In an embodiment of the present application, the electronic device trains and generates image processing models and algorithms corresponding to certain specific scenes according to comprehensive information of specific scenes and sawtooth intensity. In this way, in the process of determining the algorithm used for image processing based on the image frame, the accuracy of the algorithm determination can be improved, thereby improving the effect of image processing.

[0165] The present application provides an image processing method. Fig.11 FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present application. The image processing method provided by an embodiment of the present application will be described exemplarily by taking the first image frame as an image frame in a game screen as an example. Fig.11 As shown, the image processing method provided in the embodiment of the present application may include the following steps 1 to 16.

[0166] Step 1: The electronic device receives a long press input of the volume up button by the user.

[0167] Step 2: In response to the long press input, the electronic device displays a setting window on the screen.

[0168] In some embodiments of the present application, the above-mentioned setting window includes a game image processing identifier, and the user can input the identifier to trigger the electronic device to perform image processing on the game screen.

[0169] Step 3: The electronic device receives a click input from the user on a game image processing identifier in a setting window.

[0170] Step 4: In response to the click input, the electronic device starts to obtain the aliasing intensity parameter and image scene information of the first image frame currently displayed.

[0171] Step 5: The electronic device determines a target image processing algorithm corresponding to the first image frame according to the acquired aliasing intensity parameter and image scene information.

[0172] Step 6: The electronic device determines a list corresponding to the target image processing algorithm from at least one image processing algorithm stored in the storage space.

[0173] Step 7: The electronic device adds a row of pixels below the last row of pixels in the first image frame, and adds a column of pixels to the right of the last column of pixels.

[0174] Exemplarily, if the pixel size of the first image frame is 768*1024, the pixel size of the first image frame after adding pixels is 769*1025.

[0175] Step 8: The electronic device determines the first pixel point in the first image frame and the pixel points adjacent to the first pixel point as the first pixel block.

[0176] Step 9: The electronic device determines a first identifier corresponding to the pixel value of the first pixel block according to the pixel value of the first pixel block, and searches for a second identifier corresponding to the first pixel block in a list corresponding to the target image processing algorithm through the first identifier of the first pixel block.

[0177] Step 10: The electronic device replaces the pixel value of the first pixel point in the first image frame with the pixel value of the processed pixel block indicated by the second identifier corresponding to the first pixel block.

[0178] Step 11: The electronic device determines a second pixel point in the first image frame and pixel points adjacent to the second pixel point as a second pixel block.

[0179] Step 12: The electronic device determines a first identifier corresponding to the pixel value of the second pixel block according to the pixel value of the second pixel block, and searches for a second identifier corresponding to the second pixel block in a list corresponding to the target image processing algorithm through the first identifier of the second pixel block.

[0180] Step 13: The electronic device replaces the pixel value of the second pixel block in the first image frame with the pixel value of the processed pixel block indicated by the second identifier corresponding to the second pixel block.

[0181] Step 14: The electronic device traverses all pixel blocks of a preset size in the first image frame, and replaces the pixel value of the first pixel point in all pixel blocks of the preset size to obtain the second image frame.

[0182] Step 15: The electronic device obtains the aliasing intensity parameter and image scene information of the next image frame of the first image frame, and repeats the process of steps 5 to 14 above.

[0183] Step 16: When the electronic device game ends or the user manually turns off the image processing function, stop acquiring the jaggies intensity parameter and image scene information of subsequent image frames.

[0184] It should be noted that the above-mentioned method embodiments, or various possible implementation methods in each method embodiment, can be executed separately, or, under the premise that there is no contradiction, can also be executed in combination with each other. The specific implementation can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0185] It should be noted that the image processing method provided in the embodiment of the present application can be executed by an image processing device. In the embodiment of the present application, an image processing device executing the image processing method is taken as an example to illustrate the image processing device provided in the embodiment of the present application.

[0186] Fig.12A possible structural diagram of an image processing device involved in an embodiment of the present application is shown. Fig.12 As shown, the image processing device 70 may include: an acquisition module 71 , a determination module 72 and a processing module 73 .

[0187] Among them, the acquisition module 71 is used to obtain the aliasing intensity parameter and image scene information of the first image frame, the aliasing intensity parameter represents the pixel smoothness of the first image frame, and the image scene information represents the image category to which the first image frame belongs; the determination module 72 is used to determine the target image processing algorithm based on the aliasing intensity parameter and image scene information obtained by the acquisition module 71; the processing module 73 is used to process the pixels in the first image frame based on the target image processing algorithm determined by the determination module 72 to obtain the second image frame.

[0188] In the image processing device provided in the embodiment of the present application, since the image processing device determines the image processing algorithm corresponding to the image frame based on the aliasing intensity parameter and image scene information of the image frame, the image processing device determines the corresponding algorithm for a specific scene and image quality in combination with the scene of the image frame, thereby improving the accuracy of determining the image processing algorithm by combining feature information of multiple dimensions, and can achieve adaptive processing of the image frame, further improving the image quality improvement effect of the image processing device.

[0189] In a possible implementation, the processing module 73 is specifically used to: determine a list corresponding to the target image processing algorithm based on the target image processing algorithm, the list corresponding to the target image processing algorithm includes at least one first identifier and at least one second identifier, each first identifier represents a pixel value of a pixel block of a preset size, and each second identifier represents a pixel value obtained after a pixel block of a preset size is processed based on the target image processing algorithm; and for each pixel block of a preset size of the first image frame, determine the first identifier corresponding to the first pixel block from the list corresponding to the target image processing algorithm, and determine the second identifier corresponding to the first pixel block based on the first identifier corresponding to the first pixel block, the first pixel block being any pixel block of a preset size of the first image frame; and, based on the pixel value represented by the second identifier corresponding to the first pixel block, replace the pixel value of the first pixel point in the first pixel block; and, traverse all pixel blocks of the preset size of the first image frame, replace the pixel value of the first pixel point in all pixel blocks of the preset size, and obtain the second image frame.

[0190] In one possible implementation, the processing module 73 is further used to add at least one row of pixel points with preset pixel values ​​below the last row in the first image frame, and to add at least one column of pixel points with preset pixel values ​​to the right of the last column in the first image frame, after determining a list corresponding to the target image processing algorithm based on the target image processing algorithm; and, according to a preset size, divide all pixel points in the first image frame after the pixel points are added into at least one pixel block of a preset size.

[0191] In a possible implementation, the acquisition module 71 is specifically configured to perform a jaggies analysis process on the first image frame to obtain a jaggies intensity parameter, and perform a scene recognition process on the first image frame to obtain image scene information.

[0192] In a possible implementation, the target image processing algorithm is an image algorithm corresponding to the aliasing intensity parameter and the image scene information among the M image algorithms. The acquisition module 71 is also used to acquire a training data set before acquiring the aliasing intensity parameter and the image scene information of the first image frame, the training data set including N image frames of at least one image category, where N is an integer greater than 1; and, based on the training data set, acquire N image scene information and N aliasing intensity parameters of the N image frames; the processing module 73 is also used to group the N image frames based on the N image scene information and N aliasing intensity parameters acquired by the acquisition module 71 to obtain M image frame groups; and, based on each image frame group, train an image algorithm to obtain M image algorithms, where M is a positive integer.

[0193] In one possible implementation, combining Fig.12 ,like Fig.13 As shown, the image processing device 70 provided in the embodiment of the present application also includes: a receiving module 74; the receiving module 74 is used to receive a first input from a user when the first image frame is displayed before the acquisition module 71 acquires the jaggies intensity parameters and image scene information of the first image frame, the first image frame being an image frame in a game screen, or an image frame in a live broadcast screen; the above-mentioned acquisition module 71 is specifically used to acquire the jaggies intensity parameters and image scene information of the first image frame in response to the first input received by the receiving module 74.

[0194] The image processing device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0195] The image processing device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0196] The image processing device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0197] Alternatively, if Fig.14 As shown, an embodiment of the present application further provides an electronic device 90, including a processor 91 and a memory 92, wherein the memory 92 stores programs or instructions that can be executed on the processor 91, and when the program or instructions are executed by the processor 91, the various steps of the above-mentioned image processing method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0198] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0199] Fig.15 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0200] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0201] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.15 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0202] Here, the electronic device 100 is taken as an example:

[0203] Processor 110 is used to obtain a jaggies intensity parameter and image scene information of a first image frame, wherein the jaggies intensity parameter represents pixel smoothness of the first image frame, and the image scene information represents an image category to which the first image frame belongs; and based on the jaggies intensity parameter and the image scene information, determine a target image processing algorithm; and based on the target image processing algorithm, process pixels in the first image frame to obtain a second image frame.

[0204] In the electronic device provided in the embodiment of the present application, since the electronic device determines the image processing algorithm corresponding to the image frame based on the jaggies intensity parameter and image scene information of the image frame, the electronic device determines the corresponding algorithm for the specific scene and image quality in combination with the scene of the image frame, thereby improving the accuracy of determining the image processing algorithm by combining feature information of multiple dimensions, and can achieve adaptive processing of the image frame, further optimizing the image quality improvement effect of the electronic device.

[0205] Optionally, the processor 110 is specifically used to determine a list corresponding to the target image processing algorithm based on the target image processing algorithm, the list corresponding to the target image processing algorithm including at least one first identifier and at least one second identifier, each first identifier representing a pixel value of a pixel block of a preset size, and each second identifier representing a pixel value obtained after a pixel block of the preset size is processed based on the target image processing algorithm; and for each pixel block of the preset size of the first image frame, determine the first identifier corresponding to the first pixel block from the list corresponding to the target image processing algorithm, and determine the second identifier corresponding to the first pixel block based on the first identifier corresponding to the first pixel block, the first pixel block being any pixel block of the preset size of the first image frame; and replace the pixel value of the first pixel point in the first pixel block based on the pixel value represented by the second identifier corresponding to the first pixel block; and traverse all pixel blocks of the preset size of the first image frame, replace the pixel value of the first pixel point in all pixel blocks of the preset size, to obtain a second image frame.

[0206] Optionally, the processor 110 is further used to add at least one row of pixel points with preset pixel values ​​below the last row in the first image frame, and add at least one column of pixel points with preset pixel values ​​to the right of the last column in the first image frame, after determining a list corresponding to the target image processing algorithm based on the target image processing algorithm; and divide all pixel points in the first image frame after the pixel points are added into at least one pixel block of a preset size according to a preset size.

[0207] Optionally, the processor 110 is specifically configured to perform a jaggies analysis process on the first image frame to obtain a jaggies intensity parameter, and perform a scene recognition process on the first image frame to obtain image scene information.

[0208] Optionally, the target image processing algorithm is an image algorithm corresponding to the aliasing intensity parameter and the image scene information among the M image algorithms. The processor 110 is further used to obtain a training data set before obtaining the aliasing intensity parameter and the image scene information of the first image frame, the training data set including N image frames of at least one image category, where N is an integer greater than 1; and based on the training data set, obtain N image scene information and N aliasing intensity parameters of the N image frames; and based on the N image scene information and the N aliasing intensity parameters, group the N image frames to obtain M image frame groups; and train an image algorithm based on each image frame group to obtain M image algorithms, where M is a positive integer.

[0209] Optionally, the user input unit 107 is used to receive a first input from a user when the first image frame is displayed before the processor 110 obtains the jaggies intensity parameters and image scene information of the first image frame, where the first image frame is an image frame in a game screen or an image frame in a live broadcast screen; the processor 110 is specifically used to obtain the jaggies intensity parameters and image scene information of the first image frame in response to the first input.

[0210] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0211] The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, they will not be described again here.

[0212] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0213] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0214] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0215] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0216] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0217] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0218] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0219] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0220] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0221] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0222] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Acquire a jaggies intensity parameter and image scene information of a first image frame, wherein the jaggies intensity parameter represents pixel smoothness of the first image frame, and the image scene information represents an image category to which the first image frame belongs; Determining a target image processing algorithm based on the sawtooth intensity parameter and the image scene information; Based on the target image processing algorithm, pixels in the first image frame are processed to obtain a second image frame.

2. The method according to claim 1, characterized in that: The step of processing pixels in the first image frame based on the target image processing algorithm to obtain a second image frame includes: Based on the target image processing algorithm, determining a list corresponding to the target image processing algorithm, wherein the list corresponding to the target image processing algorithm includes at least one first identifier and at least one second identifier, each first identifier represents a pixel value of a pixel block of a preset size, and each second identifier represents a pixel value of a pixel block of a preset size obtained after being processed based on the target image processing algorithm; For each pixel block of a preset size of the first image frame, determine a first identifier corresponding to the first pixel block from the list corresponding to the target image processing algorithm, and determine a second identifier corresponding to the first pixel block based on the first identifier corresponding to the first pixel block, wherein the first pixel block is any pixel block of a preset size of the first image frame; Based on the pixel value represented by the second identifier corresponding to the first pixel block, a pixel value of a first pixel point in the first pixel block is replaced; All pixel blocks of a preset size of the first image frame are traversed, and the pixel value of the first pixel point in all the pixel blocks of the preset size is replaced to obtain the second image frame.

3. The method according to claim 2, characterized in that After determining the list corresponding to the target image processing algorithm based on the target image processing algorithm, the method further includes: Add at least one row of pixel points with a preset pixel value at a position below the last row in the first image frame, and add at least one column of pixel points with a preset pixel value at a position to the right of the last column in the first image frame; According to the preset size, all the pixels in the first image frame after the pixels are added are divided into at least one pixel block of the preset size.

4. The method according to claim 1, characterized in that: The target image processing algorithm is an image algorithm corresponding to the sawtooth intensity parameter and the image scene information among the M image algorithms; Before obtaining the sawtooth intensity parameter and the image scene information of the first image frame, the method further includes: Acquire a training data set, wherein the training data set includes N image frames of at least one image category, where N is an integer greater than 1; Based on the training data set, obtaining N image scene information and N sawtooth intensity parameters of the N image frames; Based on the N image scene information and the N sawtooth strength parameters, grouping the N image frames to obtain M image frame groups; An image algorithm is obtained based on each image frame group training to obtain the M image algorithms, where M is a positive integer.

5. The method according to claim 1, characterized in that Before obtaining the sawtooth intensity parameter and the image scene information of the first image frame, the method further includes: In the case of displaying the first image frame, receiving a first input from a user, where the first image frame is an image frame in a game screen or an image frame in a live broadcast screen; The step of obtaining the sawtooth intensity parameter and the image scene information of the first image frame includes: In response to the first input, a jaggies intensity parameter and image scene information of a first image frame are acquired.

6. An image processing device, characterized in that: The image processing device comprises: an acquisition module, a determination module and a processing module; The acquisition module is used to acquire a sawtooth intensity parameter and image scene information of the first image frame, wherein the sawtooth intensity parameter represents pixel smoothness of the first image frame, and the image scene information represents an image category to which the first image frame belongs; The determining module is used to determine a target image processing algorithm based on the jaggies intensity parameter and the image scene information acquired by the acquiring module; The processing module is used to process the pixels in the first image frame based on the target image processing algorithm determined by the determination module to obtain a second image frame.

7. The device according to claim 6, characterized in that The processing module is specifically used for: Based on the target image processing algorithm, determining a list corresponding to the target image processing algorithm, wherein the list corresponding to the target image processing algorithm includes at least one first identifier and at least one second identifier, each first identifier represents a pixel value of a pixel block of a preset size, and each second identifier represents a pixel value of a pixel block of a preset size obtained after being processed based on the target image processing algorithm; For each pixel block of a preset size of the first image frame, determine a first identifier corresponding to the first pixel block from the list corresponding to the target image processing algorithm, and determine a second identifier corresponding to the first pixel block based on the first identifier corresponding to the first pixel block, wherein the first pixel block is any pixel block of a preset size of the first image frame; Based on the pixel value represented by the second identifier corresponding to the first pixel block, a pixel value of a first pixel point in the first pixel block is replaced; All pixel blocks of a preset size of the first image frame are traversed, and the pixel value of the first pixel point in all the pixel blocks of the preset size is replaced to obtain the second image frame.

8. The device according to claim 7, characterized in that The processing module is also used to add at least one row of pixel points with preset pixel values ​​below the last row in the first image frame, and add at least one column of pixel points with preset pixel values ​​to the right of the last column in the first image frame, after determining a list corresponding to the target image processing algorithm based on the target image processing algorithm; and, according to the preset size, divide all pixel points in the first image frame after the pixel points are added into at least one pixel block of a preset size.

9. The device according to claim 6, characterized in that The target image processing algorithm is an image algorithm corresponding to the sawtooth intensity parameter and the image scene information among the M image algorithms; The acquisition module is further configured to acquire a training data set before acquiring the jaggies intensity parameter and the image scene information of the first image frame, wherein the training data set includes N image frames of at least one image category, where N is an integer greater than 1; and, based on the training data set, acquire N image scene information and N jaggies intensity parameters of the N image frames; The processing module is further used to group the N image frames based on the N image scene information and the N jaggies intensity parameters acquired by the acquisition module to obtain M image frame groups; And, an image algorithm is obtained based on each image frame group training to obtain the M image algorithms, where M is a positive integer.

10. The device according to claim 6, characterized in that The image processing device further includes: a receiving module; The receiving module is used to receive a first input from a user before the acquiring module acquires the sawtooth intensity parameter and the image scene information of the first image frame, when the first image frame is displayed, wherein the first image frame is an image frame in a game screen or an image frame in a live broadcast screen; The acquisition module is specifically configured to acquire the aliasing intensity parameter and image scene information of the first image frame in response to the first input received by the receiving module.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the image processing method according to any one of claims 1 to 5 are implemented.