Image processing method, apparatus, device, and medium

Abstract

The present disclosure relates to an image processing method, device, equipment and medium, wherein the method is applied to an electronic device, comprising: acquiring a to-be-processed image; acquiring reference information, the reference information comprising performance of the electronic device and / or specified information of the to-be-processed image; selecting a target model from a plurality of image processing models according to the reference information; wherein the processing accuracy of different image processing models is different; and processing the to-be-processed image by using the target model. The present disclosure can dynamically select a model that meets the current actual situation from a plurality of image processing models with different accuracies according to the current acquired device performance and / or specified information of the to-be-processed image, so as to better balance the image processing effect and the device running effect.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to an image processing method, apparatus, device, and medium. Background Technology

[0002] Nowadays, artificial intelligence technology has been widely applied in the field of image processing. More and more smart devices are beginning to use image processing models such as neural network models to perform processing such as object detection, object recognition, adding special effects or beautification on images.

[0003] The inventors discovered through research that, because image processing models rely on device hardware resources, the accuracy of the image processing model directly affects not only the image processing effect but also the device's operational performance. For example, higher model accuracy results in better image quality but consumes more hardware resources, making the device prone to issues such as overheating, high power consumption, lag, and crashes, negatively impacting device operation. Most existing technologies use a single, pre-set image processing model, applying the same fixed model to all devices, making it difficult to simultaneously achieve a good balance between image processing performance and device performance. Summary of the Invention

[0004] In order to solve the above-mentioned technical problems, or at least partially solve the above-mentioned technical problems, this disclosure provides an image processing method, apparatus, device and medium.

[0005] This disclosure provides an image processing method applied to an electronic device, comprising: acquiring an image to be processed; acquiring reference information, the reference information including the performance of the electronic device and / or specified information of the image to be processed; selecting a target model from a plurality of image processing models according to the reference information; wherein different image processing models have different processing accuracies; and processing the image to be processed using the target model.

[0006] Optionally, the specified information of the image to be processed includes: the sharpness of the image to be processed and / or the proportion of the target object in the image to be processed.

[0007] Optionally, the step of selecting a target model from multiple image processing models based on the reference information includes: when there are multiple types of reference information, obtaining a preset reference order; the reference order is used to indicate the reference priority of the multiple types of reference information; the reference information ranked earlier corresponds to a higher reference priority; and selecting a target model from multiple image processing models based on the reference order and the multiple types of reference information.

[0008] Optionally, the plurality of image processing models include a first image processing model and a second image processing model, wherein the accuracy of the first image processing model is higher than that of the second image processing model.

[0009] Optionally, the step of selecting a target model from multiple image processing models based on the reference sorting and multiple reference information includes: sequentially using the multiple reference information as target information to be judged according to the reference sorting until a target model is selected or until the reference information located at the end of the reference sorting is used as the target information; if the target information is reference information before the end of the reference sorting, determining whether the target information meets the low-quality discrimination condition corresponding to the target information; if it meets the condition, selecting the second image processing model as the target model; if it does not meet the condition, replacing the target information according to the reference sorting; if the target information is reference information at the end of the reference sorting, determining whether the target information meets the low-quality discrimination condition corresponding to the target information; if it meets the condition, selecting the second image processing model as the target model; if it does not meet the condition, selecting the first image processing model as the target model.

[0010] Optionally, the step of determining whether the target information meets the low-quality discrimination condition corresponding to the target information includes: obtaining the quantization value corresponding to the target information and the quantization threshold corresponding to the category to which the target information belongs; if the quantization value corresponding to the target information is less than the quantization threshold, then determining that the target information meets the low-quality discrimination condition corresponding to the target information; if the quantization value corresponding to the target information is not less than the quantization threshold, then determining that the target information does not meet the low-quality discrimination condition corresponding to the target information.

[0011] Optionally, the performance of the electronic device is ranked first in the reference order.

[0012] Optionally, the step of selecting a target model from multiple image processing models based on the reference information includes: when there are multiple types of reference information, obtaining the quantization value and weight of each type of reference information; performing weighted processing based on the quantization value and weight of each type of reference information to obtain a weighted value; determining the weighted interval range where the weighted value is located based on multiple pre-set weighted interval ranges; and selecting a target model from multiple image processing models based on the weighted interval range where the weighted value is located; wherein, the weighted interval ranges corresponding to different image processing models are different, and the weighted interval range where the weighted value is located is consistent with the weighted interval range corresponding to the target model.

[0013] Optionally, the step of selecting a target model from multiple image processing models based on the reference information includes: when the type of reference information is one, obtaining the quantized value of the reference information and multiple quantization interval ranges corresponding to the type of the reference information; determining the quantization interval range where the quantized value of the reference information is located based on the multiple quantization interval ranges; selecting a target model from multiple image processing models based on the quantization interval range where the quantized value of the reference information is located; wherein, the quantization interval ranges corresponding to different image processing models are different, and the quantization interval range where the quantized value of the reference information is located is consistent with the quantization interval range corresponding to the target model.

[0014] This disclosure also provides an image processing apparatus applied to an electronic device, comprising: an image acquisition module for acquiring an image to be processed; an information acquisition module for acquiring reference information, the reference information including the performance of the electronic device and / or specified information of the image to be processed; a model selection module for selecting a target model from multiple image processing models according to the reference information; wherein different image processing models have different processing accuracies; and a model processing module for processing the image to be processed using the target model.

[0015] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement any of the image processing methods described above.

[0016] This disclosure also provides a computer-readable storage medium storing a computer program for performing any of the image processing methods described above.

[0017] The technical solution provided in this disclosure can acquire reference information (the performance of the electronic device and / or specified information of the image to be processed), select a target model from multiple image processing models with different processing accuracies based on the reference information, and process the image to be processed using the target model. This method fully considers the correlation between factors such as device performance and / or image information and model accuracy, and also considers the different requirements for image processing effects and device operation effects under different situations (such as different device performance and / or different images). Therefore, based on the actual situation such as the currently acquired device performance and / or specified information of the image to be processed, a model (i.e., the target model) that conforms to the current actual situation can be dynamically selected from multiple image processing models with different accuracies for image processing, which helps to better balance image processing effects and device operation effects.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure;

[0022] Figure 2 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure;

[0023] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0025] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0026] Numerous specific details are set forth in the following description in order to provide a full understanding of this disclosure, but this disclosure may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only some, and not all, of the embodiments of this disclosure.

[0027] Existing image processing methods struggle to effectively balance image processing quality and device performance. While high-precision models can produce better images, they consume significant hardware resources, negatively impacting device performance, especially on lower-performance devices, leading to issues like lag, overheating, and crashes, severely affecting user experience. Conversely, low-precision models, while not significantly negatively impacting most devices, typically produce subpar image quality.

[0028] For example, existing image processing applications can use face detection models to perform face detection on images acquired by a camera, such as recognizing facial features in camera preview frames, outputting facial landmarks, and then applying special effects such as beautification, makeup, and stickers based on these landmarks. The higher the accuracy of the face detection model, the more facial landmarks it outputs, and the more closely the special effects applied to the image match the facial features, resulting in better processing effects. However, the recognition and rendering processes required by the model consume more hardware resources, leading to poor device performance.

[0029] Most existing image processing applications simply select a fixed model for processing. Regardless of the device on which the image processing application is installed, this fixed model must be used for image processing. However, this approach still struggles to effectively balance image processing quality and device performance. The inventors, through research, have discovered that in practical applications, the accuracy requirements for the image processing model are not particularly high in many situations (or scenarios). Images processed by models with lower accuracy can still meet user needs, or in these cases, will not negatively impact the user experience. For ease of understanding, the following scenario is provided as an example:

[0030] Scenario 1: High-precision models require significant computing power and take a long time to run, placing high demands on hardware resources. Devices with poor performance struggle to support high-precision model operation, often resulting in stuttering, overheating, excessive power consumption, and even crashes, leading to a poor user experience. In this scenario, using a low-precision model, while sacrificing some image quality (such as slight loss of effect integration during preview), is still within acceptable limits. Furthermore, it significantly improves device smoothness and effectively mitigates issues like stuttering, overheating, power consumption, and crashes, ensuring better device operation and preventing significant user impact from device malfunctions.

[0031] Scenario 2: When the target object (such as a face) that the user is more concerned about in the image to be processed occupies a relatively small proportion of the image, the resolution of the target object is usually also low. The image processing effect obtained by using a high-precision model is not much different from that obtained by using a low-precision model. For example, for a small face in the image, even if special effects are applied to the face, the final preview effect of the special effects obtained by the high-precision model and the low-precision model is similar, and there will be no particularly obvious difference.

[0032] Scenario 3: When the image to be processed is not very clear (such as an image taken in a low-light environment), the image processing effect obtained by using a high-precision model is not much different from that obtained by using a low-precision model.

[0033] The above are merely illustrative examples and should not be considered as limitations. Furthermore, the problems described above are the result of the inventors' practical experience and careful research. Therefore, the discovery process of these problems and the solutions proposed in the embodiments of this application below should be considered contributions made by the applicant to this application.

[0034] The inventors have taken into full account the different requirements for image processing effect and equipment operation effect under different situations. Therefore, they have proposed an image processing method, device, equipment and medium that can better balance image processing effect and equipment operation effect. For ease of understanding, the following is a detailed explanation.

[0035] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of the present disclosure. The method can be executed by an image processing device, which can be implemented using software and / or hardware, and is generally integrated into an electronic device. That is, the method can be applied to electronic devices, including but not limited to mobile phones, computers, servers, robots, smart wearable devices, and any other device with image processing capabilities. Figure 1 As shown, the method mainly includes the following steps S102 to S108:

[0036] Step S102: Obtain the image to be processed.

[0037] In practical applications, images captured by the camera of an electronic device can be used as images to be processed, or images downloaded from the network or uploaded by the user through a specified port can be used as images to be processed. This disclosure does not limit the method of acquiring the images to be processed.

[0038] Alternatively, images captured individually by the camera can be used as images to be processed, or each frame from the camera preview process can be used as an image to be processed.

[0039] Step S104: Obtain reference information, which includes the performance of the electronic device and / or specified information about the image to be processed. Exemplarily, the specified information about the image to be processed includes, but is not limited to, the clarity of the image to be processed and / or the proportion of the target object in the image to be processed. In this embodiment, the type of target object is not limited; any object of user interest can be used as a target object, such as a person (or a specified part of a person, such as a face), an animal (or a specified part of an animal), a plant (or a specified part of a plant), an item (or a specified part of an item), or a building (or a specified part of a building), etc., all of which can be used as target objects.

[0040] The performance of electronic devices (hereinafter referred to as device performance) directly affects how well they run models, and is significantly influenced by the model's accuracy. This is because model operation relies on device hardware resources; higher-accuracy models require more hardware resources. For example, for the same model, a high-performance device may still run it smoothly, while a low-performance device may experience stuttering, overheating, and other problems. Therefore, devices with different performance levels will perform differently when running the same model.

[0041] Furthermore, the specified information of the image to be processed is related to the image processing effect, but may be less affected by the model's accuracy. For example, images taken in low-light environments are usually not very sharp, or when the target object that the user is concerned about (such as a face) occupies a relatively small proportion in the image, the effect obtained by processing it with a low-precision model will not give the user a noticeable difference from the effect obtained by processing it with a high-precision model. In other words, users are often unaware of these differences in effect.

[0042] The above method fully considers the correlation between factors such as device performance and / or image information and model accuracy. Device performance and / or image information represent the current situation to a certain extent. This embodiment of the disclosure takes into account the different requirements for image processing effect and device operation effect under different situations (such as different device performance and / or different images). Therefore, this embodiment of the disclosure will also obtain reference information, which can be used to provide a reference for selecting a target model with appropriate accuracy from multiple image processing models.

[0043] Step S106: Select a target model from multiple image processing models based on reference information. Different image processing models have different processing accuracies, and the multiple image processing models have the same function. This embodiment does not limit the function of the image processing models (or the processing operations performed on the image). For example, multiple image processing models may all be detection models for the target object; for instance, if the target object is a face, multiple image processing models may all be face detection models. This is merely an example. In practical applications, multiple image processing models can also be special effects adding models, image optimization models, or other models that perform specified processing operations on the image. For example, multiple image processing models may be beautification models for the target object (such as face makeup models), etc., and this is not limited here.

[0044] This disclosure embodiment can pre-set multiple image processing models with different accuracies. Compared with the direct use of fixed models in related technologies, this disclosure embodiment can select the most suitable target model from multiple image processing models with different accuracies based on reference information such as device performance and / or specified information of the image to be processed, so as to take into account both image processing effect and device operation effect.

[0045] Step S108: The target model is used to process the image to be processed.

[0046] For example, assuming the target model is a target object detection model, then the target model is used to detect the target object in the image to be processed, and the location of the target object in the image is output. Or, assuming the target model is an effects-adding model, then the target model is used to add corresponding effects to the image to be processed. The way the target model processes the image to be processed mainly depends on the model's functionality, and no restrictions are imposed here.

[0047] The above method fully considers the correlation between factors such as device performance and / or image information and model accuracy. It also takes into account the different requirements for image processing effect and device operation effect under different situations (such as different device performance and / or different images). Therefore, based on the actual situation such as the currently acquired device performance and / or specified information of the image to be processed, the model that conforms to the current actual situation (i.e., the target model) can be dynamically selected from multiple image processing models with different accuracies for image processing, which helps to better balance image processing effect and device operation effect.

[0048] For ease of understanding, this disclosure provides specific implementation examples for obtaining reference information, which can be found in 1) to 2) of the implementation:

[0049] 1) When the reference information includes the performance of an electronic device, in some implementations, the hardware information of the electronic device can be obtained. This hardware information includes, but is not limited to, the CPU (Central Processing Unit) model and / or GPU (Graphics Processing Unit) model. Then, based on the hardware information, the corresponding performance is queried from a pre-set device performance database. Device performance can be represented in the form of a score, or in multiple levels such as Level 1, Level 2, Level 3, etc., with higher levels indicating better device performance. In practical applications, the performance of the electronic device can also be obtained in advance and stored in a designated location. When needed, the performance of the electronic device can be retrieved directly from the designated location.

[0050] 2) When the reference information includes specified information about the image to be processed, in some implementations, the specified information can be extracted directly by analyzing the image to be processed. For example, if the specified information about the image to be processed includes the image's sharpness, a sharpness evaluation algorithm can be directly used to obtain the image's sharpness. Alternatively, the ambient light level of the image to be processed can be acquired first, and the sharpness of the image can be determined based on the ambient light level. Images acquired in insufficient ambient light have low sharpness; that is, the darker the light, the lower the image sharpness. For example, if the specified information about the image to be processed includes the proportion of the target object in the image, the size of the target object in the image can be detected first, and the proportion of the target object in the image can be obtained by comparing the size of the target object with the size of the entire image.

[0051] As mentioned above, the obtained reference information includes, but is not limited to, the performance of the electronic device and / or specified information of the image to be processed. The specified information of the image to be processed includes, but is not limited to, the sharpness of the image to be processed and / or the proportion of the target object in the image to be processed. Therefore, there may be one type of reference information or multiple types of reference information. This disclosure provides specific implementation methods for selecting a target model from multiple image processing models based on the number of types of reference information obtained. Specifically, please refer to the following (I) and (II) for implementation:

[0052] (a) There are various types of reference information.

[0053] This disclosure provides two possible implementation methods for selecting a target model from multiple image processing models based on the reference information when there are multiple types of reference information. The following describes implementation method one and implementation method two in detail:

[0054] Implementation Method 1:

[0055] Step A: Obtain the preset reference sorting; the reference sorting is used to indicate the reference priority of various reference information; the earlier the reference information is in the sorting, the higher its reference priority.

[0056] This disclosure fully considers that the degree of correlation between different types of reference information and model accuracy may vary, and that their impact or requirements on image processing effects and device operation effects are also different. By comprehensively weighing these factors, various types of reference information can be ranked to facilitate comprehensive analysis of these references later. This disclosure also fully considers that model accuracy has a significant impact on the performance of electronic devices, and that the operation of electronic devices directly affects user experience. Therefore, the performance of the electronic device can be prioritized in the reference ranking. For example, the reference ranking from first to last is: performance of the electronic device, clarity of the image to be processed, and the proportion of the target object in the image to be processed. Alternatively, the reference ranking from first to last can also be: performance of the electronic device, the proportion of the target object in the image to be processed, and clarity of the image to be processed.

[0057] Step B involves selecting a target model from multiple image processing models based on the reference sorting and various reference information.

[0058] By using the above methods, when multiple reference information is available, the model can be selected based on the reference factors with a focus, so as to better ensure the user experience while taking into account both image processing effect and device operation effect.

[0059] In some implementation examples, to facilitate device processing, multiple image processing models can be provided, including a first image processing model and a second image processing model, where the first image processing model has higher precision than the second image processing model. That is, the first image processing model is a high-precision model, and the second image processing model is a low-precision model. Setting only two precision models not only facilitates device storage, but the high / low precision models can also basically meet the needs of various situations. When electronic devices need to perform image processing, they can more easily and quickly select the model that best suits the current scenario from the two image processing models.

[0060] Based on this, step B can be implemented by referring to B1 to B3 as follows:

[0061] B1 uses multiple reference information as target information to be judged in order of reference ranking until the target model is selected or until the reference information at the end of the reference ranking is used as the target information.

[0062] B2. If the target information is reference information before the last position in the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it does, select the second image processing model as the target model; if it does not, replace the target information according to the reference sorting.

[0063] Different types of reference information have different low-quality judgment criteria. The determination of whether the corresponding target information indicates a poor current situation is primarily based on these criteria. For example, poor device performance or poor image quality (poor clarity, the target object of interest occupying a small proportion of the image, etc.) both fall under the category of a poor current situation. When the current target information indicates a poor current situation, a low-precision model can be directly used to avoid the negative impact of using a high-precision model, which would not only consume more hardware resources and negatively affect device performance but also fail to deliver significant image processing results. If the current target information indicates that the corresponding reference factors perform well under the current situation, then the target information should be changed, i.e., the reference factors should be changed to reassess whether there are any unfavorable factors in the current situation.

[0064] B3. If the target information is the last reference information in the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it does, select the second image processing model as the target model; if it does not, select the first image processing model as the target model.

[0065] If all reference information in the reference ranking that is not the last in the ranking has been judged as low-quality from beginning to end and all of them have performed well, then the last reference information in the ranking will be judged as low-quality. If this reference information also performs well, that is, there are no bad factors in the current situation, then the high-precision model can be directly used as the target model; otherwise, the low-precision model will be used.

[0066] In steps B1 through B3 above, various reference information are evaluated from beginning to end according to their reference order. If any reference information performs poorly, a low-precision model is used directly to avoid the negative impact of a high-precision model on device performance, or to prevent the high-precision model from consuming excessive hardware resources without significantly improving the user-perceived image processing effect. Only if all reference information performs well is a high-precision model used to further enhance image processing. This method dynamically selects between high-precision and low-precision models based on actual conditions, effectively ensuring a good user experience.

[0067] In both B1 and B2 above, it is necessary to determine whether the target information meets the low-quality discrimination condition corresponding to the target information. In some specific implementations, the quantized value corresponding to the target information and the quantization threshold corresponding to the category to which the target information belongs can be obtained first. If the quantized value corresponding to the target information is less than the quantization threshold, it is determined that the target information meets the low-quality discrimination condition corresponding to the target information; if the quantized value corresponding to the target information is not less than the quantization threshold, it is determined that the target information does not meet the low-quality discrimination condition corresponding to the target information. For example, the performance of an electronic device corresponds to a performance score threshold, the sharpness of the image to be processed corresponds to a sharpness threshold, and the proportion of the target object in the image to be processed corresponds to a proportion threshold. By quantizing each reference information and comparing it with the quantization threshold, the quality of each reference information can be evaluated more objectively and accurately.

[0068] To facilitate understanding of the specific implementation methods B1 to B3 above, taking the following order from first to last as an example: the performance of the electronic device, the clarity of the image to be processed, and the proportion of the target object in the image to be processed, the performance of the electronic device is first used as the target information. At this point, it is determined whether the performance of the electronic device meets the corresponding low-quality discrimination conditions (such as whether the performance of the electronic device is less than a preset performance score threshold). If so, it indicates that the performance of the electronic device is poor, and a low-precision model is directly selected as the target model. If not, the clarity of the image to be processed can then be used as the target information. At this point, it is determined whether the clarity of the image to be processed meets the corresponding low-quality discrimination conditions. (For example, determining whether the clarity of the electronic device is less than a preset clarity threshold). If so, it means that the clarity of the electronic device is low, and a low-precision model is directly selected as the target model. If not, the proportion of the target object in the image to be processed (which is already at the end of the reference sort) can be used as the target information. At this time, it is determined whether the proportion of the target object in the image to be processed meets the low-quality discrimination condition corresponding to the target information (such as determining whether the proportion of the target object in the image to be processed is less than a preset proportion threshold). If so, it means that the target object is small in the image, and a low-precision model is directly selected as the target model. If not, a high-precision model is finally selected as the target model.

[0069] Through the first implementation method described above, multiple reference information can be evaluated one by one according to the reference order. If any reference information performs poorly, a low-precision model is directly adopted. Only when all reference information performs well is a high-precision model used. This approach fully considers that using a high-precision model under unfavorable conditions would not only significantly impact device operation but also fail to substantially improve image processing performance. Therefore, a low-precision model is used to ensure a good user experience, while a high-precision model is used under favorable conditions to further improve image processing performance while ensuring the device functions normally.

[0070] Implementation Method Two:

[0071] Step a: Obtain the quantization value and weight of each type of reference information.

[0072] This disclosure allows for the quantification of each type of reference information, such as quantifying the performance of an electronic device as a performance score, or quantifying the sharpness of an image to be processed as a sharpness value. This disclosure allows for the determination of appropriate weights based on the influence of different types of reference information on device performance or image processing effects.

[0073] Step b: Perform weighted processing based on the quantization value and weight of each reference information to obtain a weighted value.

[0074] Considering that the quantification value ranges of different types of reference information may be different, the quantification values ​​of various reference information can be normalized during the weighting process. By weighting the normalized value and weight of each type of reference information, a more objective weighted value can be obtained.

[0075] Step c: Determine the weighted interval range where the weighted value is located based on the multiple pre-set weighted interval ranges.

[0076] Step d: Select the target model from multiple image processing models based on the weighted interval range where the weighted value is located; wherein, the weighted interval ranges corresponding to different image processing models are different, and the weighted interval range where the weighted value is located is consistent with the weighted interval range corresponding to the target model.

[0077] In practical applications, multiple weighted interval ranges can be preset, with each image processing model corresponding to a weighted interval range. The image processing model corresponding to the weighted interval range where the weighted value is located is the target model.

[0078] The above-described second implementation method can comprehensively measure the impact of various reference information on the current situation, and can better balance the overall equipment operation effect and image processing effect.

[0079] When there are multiple types of reference information, either Implementation Method 1 or Implementation Method 2 can be flexibly selected according to actual needs, and no restrictions are imposed here.

[0080] (ii) The type of reference information is one

[0081] This disclosure provides an embodiment of a method for selecting a target model from multiple image processing models based on the reference information when there is only one type of reference information. For example, it can be implemented by referring to the following steps (1) to (3):

[0082] Step (1): Obtain the quantized value of the reference information and the range of multiple quantization intervals corresponding to the category to which the reference information belongs;

[0083] Step (2): Determine the quantization interval range where the quantization value of the reference information is located based on multiple quantization interval ranges;

[0084] Step (3): Select the target model from multiple image processing models based on the quantization interval range of the quantization value of the reference information; wherein, the quantization interval ranges corresponding to different image processing models are different, and the quantization interval range of the quantization value of the reference information is consistent with the quantization interval range corresponding to the target model.

[0085] In practical applications, different types of reference information can be set for each image processing model, corresponding to different quantization ranges. When only one type of reference information is available, the target model can be found based on the quantization range of the quantized value of that reference information. This method is simple, feasible, and objectively reliable.

[0086] Based on the foregoing embodiments, for ease of understanding, this disclosure provides an image processing method, using a face detection and special effects addition model as an example for illustration, and only sets up high-precision and low-precision models, see [link to documentation]. Figure 2 The flowchart of an image processing method shown here mainly includes the following steps:

[0087] In step S202, the image processing function of the electronic device is activated, and an image to be processed is acquired. For example, the image to be processed may be an image frame captured by the electronic device in preview mode.

[0088] Step S204: Determine whether the performance of the electronic device is lower than a preset performance score threshold. If yes, proceed to step S210b; otherwise, proceed to step S206. In practical applications, the performance score of the electronic device can be obtained in advance when the image processing function is started. Then, based on the performance score threshold, it is determined whether the electronic device has poor performance. If the electronic device has poor performance, a low-precision model is directly selected for processing. If the electronic device has good performance, further judgment operations can be performed.

[0089] Step S206: Determine whether the image sharpness is lower than a preset sharpness threshold. If yes, proceed to step S210b; otherwise, proceed to step S208. For example, during the preview process, the brightness of the current shooting scene can be identified, that is, the image sharpness is determined based on the shooting environment. Generally speaking, images shot in low-light environments have lower sharpness. Alternatively, other sharpness evaluation algorithms can be used to obtain image sharpness; this is not limited here. If the image sharpness is low, a low-precision model is directly selected for processing; if the image sharpness is not low, further judgment operations can be performed.

[0090] Step S208: Determine whether the proportion of faces in the image is lower than a preset proportion threshold. If yes, proceed to step S210b; otherwise, proceed to step S210a. For example, face detection can be performed on the image to be processed first, and the proportion of detected face regions in the entire image can be calculated. Based on the face proportion and the proportion threshold, it can be determined whether the face proportion is low. If the face proportion is low, a low-precision model is directly selected for processing; if the face proportion is not low, a high-precision model is ultimately selected.

[0091] Step S210a: Select a high-precision model as the target model.

[0092] Step S210b: Select a low-precision model as the target model.

[0093] Step S212: Use the target model to identify facial key points in the image to be processed, and add specified effects based on the facial key points.

[0094] The above method allows for a simple and convenient selection of a model suitable for the current situation. It also takes into full account that using a high-precision model in unfavorable situations (poor device performance, low clarity, or high proportion of faces, etc.) would not only significantly affect device operation but also fail to improve image processing effects. Therefore, a low-precision model is used to ensure the user's device experience, while a high-precision model is used only when the current situation is good to further improve image processing effects while ensuring that the current device can operate normally.

[0095] In summary, the image processing method provided in this disclosure fully considers the correlation between factors such as device performance and / or image information and model accuracy. It also considers the different requirements for image processing effect and device operation effect under different situations (such as different device performance and / or different images). Therefore, based on the actual situation such as the currently acquired device performance and / or specified information of the image to be processed, the model (i.e., the target model) that conforms to the current actual situation can be dynamically selected from multiple image processing models with different accuracies for image processing, which helps to better balance image processing effect and device operation effect.

[0096] Corresponding to the aforementioned image processing method, this disclosure provides an image processing apparatus. Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure. The apparatus can be implemented by software and / or hardware, and is generally integrated into an electronic device, such as... Figure 3 As shown, the device includes:

[0097] Image acquisition module 302 is used to acquire the image to be processed;

[0098] The information acquisition module 304 is used to acquire reference information, including the performance of the electronic device and / or specified information of the image to be processed;

[0099] The model selection module 306 is used to select a target model from multiple image processing models based on reference information; wherein, different image processing models have different processing accuracies;

[0100] The model processing module 308 is used to process the image to be processed using the target model.

[0101] The aforementioned device fully considers the correlation between factors such as equipment performance and / or image information and model accuracy. It also takes into account the different requirements for image processing effect and equipment operation effect under different situations (such as different equipment performance and / or different images). Therefore, it can dynamically select the model (i.e. the target model) that conforms to the current actual situation from multiple image processing models with different accuracies for image processing based on the actual situation such as the currently acquired equipment performance and / or the specified information of the image to be processed. This helps to better balance the image processing effect and the equipment operation effect.

[0102] In some implementations, the specified information of the image to be processed includes: the sharpness of the image to be processed and / or the proportion of the target object in the image to be processed.

[0103] In some implementations, the model selection module 306 is specifically used to: when there are multiple types of reference information, obtain a preset reference order; the reference order is used to indicate the reference priority of the multiple types of reference information; the reference information that ranks higher in the order corresponds to a higher reference priority; and select a target model from multiple image processing models according to the reference order and the multiple types of reference information.

[0104] In some implementations, the plurality of image processing models include a first image processing model and a second image processing model, wherein the accuracy of the first image processing model is higher than that of the second image processing model.

[0105] In some implementations, the model selection module 306 is specifically used to: sequentially select multiple reference information as target information to be judged according to the reference sorting, until a target model is selected or until the reference information located at the end of the reference sorting is selected as the target information; if the target information is reference information before the end of the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it meets the condition, select the second image processing model as the target model; if it does not meet the condition, replace the target information according to the reference sorting; if the target information is reference information at the end of the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it meets the condition, select the second image processing model as the target model; if it does not meet the condition, select the first image processing model as the target model.

[0106] In some implementations, the model selection module 306 is specifically used to: obtain the quantization value corresponding to the target information and the quantization threshold corresponding to the category to which the target information belongs; if the quantization value corresponding to the target information is less than the quantization threshold, then determine that the target information meets the low-quality discrimination condition corresponding to the target information; if the quantization value corresponding to the target information is not less than the quantization threshold, then determine that the target information does not meet the low-quality discrimination condition corresponding to the target information.

[0107] In some implementations, the performance of the electronic device is ranked first in the reference order.

[0108] In some implementations, the model selection module 306 is specifically used for: when there are multiple types of reference information, obtaining the quantization value and weight of each type of reference information; performing weighting processing based on the quantization value and weight of each type of reference information to obtain a weighted value; determining the weighting interval range where the weighted value is located based on a plurality of pre-set weighting interval ranges; selecting a target model from multiple image processing models based on the weighting interval range where the weighted value is located; wherein, the weighting interval ranges corresponding to different image processing models are different, and the weighting interval range where the weighted value is located is consistent with the weighting interval range corresponding to the target model.

[0109] In some implementations, the model selection module 306 is specifically used to: when the type of reference information is one, obtain the quantized value of the reference information and multiple quantization interval ranges corresponding to the type of the reference information; determine the quantization interval range where the quantized value of the reference information is located based on the multiple quantization interval ranges; select a target model from multiple image processing models based on the quantization interval range where the quantized value of the reference information is located; wherein, the quantization interval ranges corresponding to different image processing models are different, and the quantization interval range where the quantized value of the reference information is located is consistent with the quantization interval range corresponding to the target model.

[0110] The image processing apparatus provided in this disclosure can execute the image processing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0111] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device embodiments can be referred to the corresponding process in the method embodiments, and will not be repeated here.

[0112] This disclosure also provides an electronic device, which includes: a processor; a memory for storing processor-executable instructions; and a processor for reading executable instructions from the memory and executing the instructions to implement any of the above-described image processing methods.

[0113] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Figure 4 As shown, the electronic device 400 includes one or more processors 401 and memory 402.

[0114] The processor 401 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 400 to perform desired functions.

[0115] The memory 402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 401 may execute the program instructions to implement the image processing method of the embodiments of this disclosure described above and / or other desired functions. Various contents such as input signals, signal components, and noise components may also be stored in the computer-readable storage medium.

[0116] In one example, the electronic device 400 may also include an input device 403 and an output device 404, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0117] In addition, the input device 403 may also include, for example, a keyboard, a mouse, etc.

[0118] The output device 404 can output various information to the outside, including determined distance information, direction information, etc. The output device 404 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0119] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device 400 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 400 may include any other suitable components depending on the specific application.

[0120] In addition to the methods and devices described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the image processing method provided in the embodiments of this disclosure.

[0121] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0122] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the image processing method provided in embodiments of this disclosure.

[0123] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] This disclosure also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the image processing method of this disclosure.

[0125] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0126] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized in that, Applied to electronic devices, including: Obtain the image to be processed; Obtain reference information, which includes the performance of the electronic device and specified information of the image to be processed; wherein, the specified information of the image to be processed includes: the sharpness of the image to be processed and / or the proportion of the target object in the image to be processed; A target model is selected from multiple image processing models based on the reference information; wherein the processing accuracy of different image processing models is different. The target model is used to process the image to be processed; The step of selecting a target model from multiple image processing models based on the reference information includes: Based on whether the reference information meets its corresponding low-quality discrimination condition, a target model is selected from multiple image processing models; wherein, the low-quality discrimination condition is determined based on the relationship between the quantization value corresponding to the reference information and the quantization threshold corresponding to the category to which the reference information belongs.

2. The method according to claim 1, characterized in that, The step of selecting a target model from multiple image processing models based on the reference information includes: When there are multiple types of reference information, a preset reference order is obtained; the reference order is used to indicate the reference priority of the multiple types of reference information; the reference information that appears earlier in the order has a higher reference priority; Based on the reference sorting and various reference information, a target model is selected from multiple image processing models.

3. The method according to claim 2, characterized in that, The plurality of image processing models include a first image processing model and a second image processing model, wherein the accuracy of the first image processing model is higher than that of the second image processing model.

4. The method according to claim 3, characterized in that, The step of selecting a target model from multiple image processing models based on the reference sorting and various reference information includes: Multiple reference information are sequentially used as target information to be judged according to the reference sorting until a target model is selected or until the reference information at the end of the reference sorting is used as target information; If the target information is reference information preceding the last position in the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it does, select the second image processing model as the target model; if it does not, replace the target information according to the reference sorting. If the target information is the last reference information in the reference sorting, determine whether the target information meets the low-quality discrimination condition corresponding to the target information; if it does, select the second image processing model as the target model; if it does not, select the first image processing model as the target model.

5. The method according to claim 4, characterized in that, The step of determining whether the target information meets the low-quality discrimination condition corresponding to the target information includes: Obtain the quantized value corresponding to the target information and the quantized threshold corresponding to the category to which the target information belongs; If the quantization value corresponding to the target information is less than the quantization threshold, then the target information is determined to meet the low-quality discrimination condition corresponding to the target information. If the quantization value corresponding to the target information is not less than the quantization threshold, then the target information is determined not to meet the low-quality discrimination condition corresponding to the target information.

6. The method according to any one of claims 2 to 5, characterized in that, The performance of the electronic device ranks first in the reference ranking.

7. The method according to claim 1, characterized in that, The step of selecting a target model from multiple image processing models based on the reference information includes: When there are multiple types of reference information, obtain the quantization value and weight of each type of reference information; A weighted value is obtained by weighting the quantized value and weight of each type of reference information. The weighted interval range in which the weighted value lies is determined based on multiple pre-set weighted interval ranges; A target model is selected from multiple image processing models based on the weighted interval range in which the weighted value is located; wherein, the weighted interval ranges corresponding to different image processing models are different, and the weighted interval range in which the weighted value is located is consistent with the weighted interval range corresponding to the target model.

8. The method according to claim 1, characterized in that, The step of selecting a target model from multiple image processing models based on the reference information includes: When the type of reference information is one, obtain the quantized value of the reference information and multiple quantization interval ranges corresponding to the type of reference information; Based on the multiple quantization interval ranges, determine the quantization interval range in which the quantization value of the reference information is located; Based on the quantization interval range of the quantization value of the reference information, a target model is selected from multiple image processing models; wherein, different image processing models correspond to different quantization interval ranges, and the quantization interval range of the quantization value of the reference information is consistent with the quantization interval range of the target model.

9. An image processing apparatus, characterized in that, Applied to electronic devices, including: The image acquisition module is used to acquire the image to be processed. An information acquisition module is used to acquire reference information, which includes the performance of the electronic device and specified information of the image to be processed; wherein, the specified information of the image to be processed includes: the clarity of the image to be processed and / or the proportion of the target object in the image to be processed; The model selection module is used to select a target model from multiple image processing models based on the reference information; wherein, the processing accuracy of different image processing models is different; The model processing module is used to process the image to be processed using the target model; The model selection module is specifically used for: Based on whether the reference information meets its corresponding low-quality discrimination condition, a target model is selected from multiple image processing models; wherein, the low-quality discrimination condition is determined based on the relationship between the quantization value corresponding to the reference information and the quantization threshold corresponding to the category to which the reference information belongs.

10. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the image processing method according to any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the image processing method according to any one of claims 1-8.

Images