Super-resolution chip, super-resolution algorithm updating method and electronic device

Abstract

The present application relates to a method, device, super-resolution chip, computer equipment and storage medium for updating a super-resolution algorithm. The method comprises: obtaining a user-personalized image through a super-resolution chip; using the user-personalized image as a training sample for super-resolution learning to obtain a target super-resolution algorithm; and updating an initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm. The method can be used to update the super-resolution algorithm in the super-resolution chip.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a super-resolution chip, a method for updating a super-resolution algorithm, an electronic device, and a computer-readable storage medium. Background Art

[0002] When processing pictures or videos, electronic devices will directly stretch or reduce the content source to the corresponding resolution of the screen in the display link, and then output it to the screen for display. However, after simple stretching, it is found that the originally clear picture has blurred edges after being enlarged. Such stretching affects the viewing effect of users and distorts the picture of the content source.

[0003] With the development of computer technology, super-resolution technology has emerged, which is to improve the resolution of the original image through hardware or software methods. Traditional super-resolution technology usually burns the super-resolution algorithm into the super-resolution chip, and then uses the super-resolution chip to call the super-resolution algorithm to perform super-resolution processing on the image. However, in traditional super-resolution technology, the super-resolution chip usually only burns one set of super-resolution algorithms, and the super-resolution algorithm in the super-resolution chip cannot be updated. Summary of the invention

[0004] The embodiments of the present application provide a method, device, electronic device, and computer-readable storage medium for updating a super-resolution algorithm, which can update the super-resolution algorithm in a super-resolution chip.

[0005] A method for updating a super-resolution algorithm, comprising:

[0006] Obtain user-personalized images through super-resolution chips;

[0007] Using the user personalized image as a training sample to perform super-resolution learning to obtain a target super-resolution algorithm;

[0008] The initial super-resolution algorithm burned into the super-resolution chip is updated based on the target super-resolution algorithm.

[0009] A super-resolution algorithm updating device, comprising:

[0010] An image acquisition module, used to acquire user-personalized images through a super-resolution chip;

[0011] An algorithm acquisition module, used to perform super-resolution learning using the user personalized image as a training sample to obtain a target super-resolution algorithm;

[0012] An updating module is used to update the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm.

[0013] A super-resolution chip, comprising:

[0014] Algorithm storage hardware module, used to store the burned-in initial super-resolution algorithm;

[0015] Update the interface for obtaining user personalized images;

[0016] The update hardware module is connected to the update interface and the algorithm storage hardware module respectively, and is used to use each of the user personalized images as a training sample to perform super-resolution learning to obtain a target super-resolution algorithm, and to update the burned initial super-resolution algorithm stored in the algorithm storage hardware module based on the target super-resolution algorithm.

[0017] An electronic device comprises the above-mentioned super-resolution chip.

[0018] A super-resolution chip comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the super-resolution algorithm update method as described above.

[0019] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0020] The above-mentioned super-resolution algorithm updating method, device, electronic device, super-resolution chip, and computer-readable storage medium obtain user personalized images through the super-resolution chip; use the user personalized images as training samples for super-resolution learning to obtain a target super-resolution algorithm that meets the user's personalized needs, and then based on the newly obtained target super-resolution algorithm, the initial super-resolution algorithm burned in the super-resolution chip can be updated, the super-resolution algorithm burned in the super-resolution chip can be optimized and improved, and can meet the user's personalized needs for super-resolution processing of images. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1is a flow chart of a method for updating a super-resolution algorithm in one embodiment;

[0023] Figure 2 A flowchart of a step in an embodiment in which a user personalized image is used as a training sample for super-resolution learning to obtain a target super-resolution algorithm;

[0024] Figure 3 is a flow chart of a method for updating a super-resolution algorithm in another embodiment;

[0025] Figure 4 is a schematic diagram of an updating method of a super-resolution algorithm in one embodiment;

[0026] Figure 5 is a schematic diagram of a process flow of updating a super-resolution algorithm in another embodiment;

[0027] Figure 6 is a structural block diagram of an updating device for a super-resolution algorithm in one embodiment;

[0028] Figure 7 Schematic diagram of the internal structure of a super-resolution chip in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first priority may be referred to as a second priority, and similarly, a second priority may be referred to as a first priority. Both the first priority and the second priority are priorities, but they are not the same priority.

[0031] In one embodiment, Figure 1 As shown, a method for updating a super-resolution algorithm is provided. This embodiment is illustrated by applying the method to a super-resolution chip of an electronic device. The electronic device may be a terminal or a server. The terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server may be implemented as an independent server or a server cluster consisting of multiple servers. It is understandable that the method may also be applied to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0032] Step 102: Acquire a user-personalized image through a super-resolution chip.

[0033] Super resolution (SR) refers to the process of restoring a high-resolution image (HR) from a low-resolution image (LR) or image sequence. A high-resolution image means that the image has a high pixel density and can provide more details, which often play a key role in applications.

[0034] A super-resolution chip is a chip used to perform super-resolution processing on an image. A super-resolution algorithm is stored in the super-resolution chip, and calling the super-resolution algorithm can perform super-resolution processing on an image to improve the resolution of the image. The super-resolution chip is integrated into an electronic device.

[0035] A user-personalized image is an image that represents the user's personalized needs. The user's personalized needs may include at least portrait processing needs and landscape processing needs. Among them, portrait processing needs include face beautification needs and background blur needs. Landscape processing needs include adding filters and brightness adjustment needs. A user-personalized image is, for example, a selfie image, and the user's personalized need is to perform face beautification processing on the selfie image. A user-personalized image is, for example, a landscape photo, and the user's personalized need is to adjust the brightness of the landscape photo to make the landscape photo more realistic.

[0036] The user's personalized image is obtained from each application through the super-resolution chip. Among them, the application can be a third-party application or a system application. A third-party application is a related software developed by other organizations or individuals who are not the software programmers to address the functional deficiencies of a certain software or application. Third-party applications include music applications, social applications, video applications, etc. downloaded by users. System applications are applications that come with the operating system of an electronic device. System applications include album applications and desktop applications that come with the operating system of an electronic device. Among them, the desktop application can obtain UI (User Interface) images.

[0037] Step 104: Use the user personalized image as a training sample to perform super-resolution learning to obtain a target super-resolution algorithm.

[0038] Super-resolution learning refers to the learning of super-resolution processing by the super-resolution chip. The target super-resolution algorithm refers to the super-resolution algorithm obtained by super-resolution learning of user-personalized images.

[0039] Specifically, the electronic device controls the super-resolution chip to use each user's personalized image as a training sample, and uses a machine learning algorithm to perform super-resolution learning on the training sample to obtain a target super-resolution algorithm. Among them, the machine learning algorithm includes at least one of mechanical learning, learning from instruction or learning by being told, learning by deduction, analytical learning, supervised learning, and unsupervised learning.

[0040] Step 106 , updating the initial super-resolution algorithm programmed in the super-resolution chip based on the target super-resolution algorithm.

[0041] The initial super-resolution algorithm refers to the super-resolution algorithm that has been burned into the super-resolution chip. The number of initial super-resolution algorithms is not limited, and it can be one or more. Burning refers to burning data into a storage medium through a tool such as a burner. The super-resolution chip includes an algorithm storage hardware module, which stores the burned initial super-resolution algorithm. Among them, the algorithm storage hardware module can be an MTP (Multiple-Time Programmable) module. The MTP module is burned with a set of initial super-resolution algorithms based on learning before the super-resolution chip leaves the factory, wherein the initial super-resolution algorithm set based on learning is obtained after super-resolution learning based on a trained atlas.

[0042] The electronic device controls the super-resolution chip to update the initial super-resolution algorithm burned in the super-resolution chip based on the target super-resolution algorithm. In one embodiment, the super-resolution chip replaces the burned initial super-resolution algorithm with the target super-resolution algorithm to obtain an updated super-resolution algorithm, and the updated super-resolution algorithm is the target super-resolution algorithm. In another embodiment, the super-resolution chip adds a target super-resolution algorithm on the basis of the burned initial super-resolution algorithm to obtain an updated super-resolution algorithm. In other embodiments, the super-resolution chip can also replace the specified super-resolution algorithm in the burned initial super-resolution algorithm with the target super-resolution algorithm to obtain an updated super-resolution algorithm. Among them, the specified super-resolution algorithm can be set as needed. The way in which the super-resolution chip updates the burned initial super-resolution algorithm is not limited here.

[0043] In this embodiment, a user's personalized image is obtained through a super-resolution chip; the user's personalized image is used as a training sample for super-resolution learning, and a target super-resolution algorithm that meets the user's personalized needs can be obtained. Based on the newly obtained target super-resolution algorithm, the initial super-resolution algorithm burned in the super-resolution chip can be updated, and the super-resolution algorithm burned in the super-resolution chip can be optimized and improved, and the user's personalized needs for super-resolution processing of images can be met.

[0044] In another embodiment, the super-resolution chip can also send the user personalized image to a designated server; the designated server uses the user personalized image as a training sample to perform super-resolution learning to obtain a target super-resolution algorithm, and then sends the target super-resolution algorithm to the super-resolution chip of the electronic device; the super-resolution then updates the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm.

[0045] In one embodiment, Figure 2 As shown, the user personalized image is used as a training sample for super-resolution learning to obtain the target super-resolution algorithm, including:

[0046] Step 202: Determine the image content type of each user's personalized image.

[0047] The image content type refers to the type corresponding to the content of the user's personalized image. The image content type includes at least a portrait type and a landscape type. Furthermore, the portrait type may specifically include a face type, a full-body portrait type, a multi-person photo type, etc. The landscape type may specifically include a mountain type, an ocean type, a forest type, etc.

[0048] Specifically, the electronic device controls the super-resolution chip to obtain the image content type of each user's personalized image. In one embodiment, the electronic device controls the central processor to use the image content type analysis model to process each user's personalized image respectively, determine the image content type of each user's personalized image, and use the image content type as the attribute information of the corresponding user's personalized image, and transmit it to the super-resolution chip together with the user's personalized image. In another embodiment, the electronic device controls the super-resolution chip to use the image content type analysis model to process each user's personalized image respectively, and determine the image content type of each user's personalized image.

[0049] Step 204: Filter out target content types based on the image content types of each user's personalized image.

[0050] The target content type refers to the content type of the user-personalized image used for super-resolution learning. The target content type can be at least one of the image content types.

[0051] The electronic device controls the super-resolution chip to filter out a target content type from each image content type based on the image content type of each user's personalized image. In one embodiment, the super-resolution chip can count the number of images belonging to each image content type, and use the image content type with the highest number of images as the target content type. In another embodiment, the super-resolution chip can randomly select one or more image content types as the target content type. In another embodiment, the super-resolution chip can also use the image content type specified by the user as the target content type from each image content type. The method of filtering out the target content type is not limited here.

[0052] Step 206 , using the user personalized images of the target content type as training samples to perform super-resolution learning, and obtaining a target super-resolution algorithm for super-resolution processing of the user personalized images of the target content type.

[0053] The super-resolution chip uses user-personalized images of the target content type as training samples for super-resolution learning. Then, the resulting target super-resolution algorithm can more accurately perform super-resolution processing on images of the target content type, solve user-personalized needs for images of the target content type, and obtain images of the target content type with higher resolution and clarity.

[0054] In this embodiment, the super-resolution chip screens out the target content type, uses the user-personalized images of the target content type as training samples for super-resolution learning, and obtains a target super-resolution algorithm for the scene of the target content type. The images of the scene of the target content type can be super-resolution processed more accurately to obtain images of the target content type with higher resolution and clearer images.

[0055] In one embodiment, based on the image content type of each user's personalized image, the target content type is screened out, including: counting the number of images belonging to each image content type; and based on the number of each image, screening out the target content type from each image content type.

[0056] In one embodiment, the super-resolution chip counts the number of images belonging to each image content type, and uses the image content type with the highest number of images as the target content type. It can be understood that the image content type with the highest number of images indicates that the type is the type that is closest to the personalized needs of the user, so the image content type with the highest number of images is used as the target content type, and super-resolution learning can be performed on the target content type, so as to obtain a target super-resolution algorithm for super-resolution processing of images of the target content type, which can more accurately meet the personalized needs of the user and obtain images of the target content type with higher resolution and clearer resolution.

[0057] In another embodiment, the super-resolution chip counts the number of images belonging to each image content type, and uses the image content type with the second highest number of images as the target content type. In another embodiment, the super-resolution chip counts the number of images belonging to each image content type, and uses the image content type with the lowest number of images as the target content type. The way in which the super-resolution chip selects the target content type from each image content type based on the number of each image can be set as needed and is not limited here.

[0058] In another embodiment, determining the image content type of each user's personalized image includes: determining the image content type of each user's personalized image within a preset time period; filtering out a target content type based on the image content type of each user's personalized image, including: counting the type usage frequency of each image content type within the preset time period; filtering out a target content type from each image content type based on the type usage frequency. The preset time period can be set as needed. Type usage frequency refers to the frequency with which an image content type is used. The higher the type usage frequency of an image content type, the more the user likes to use images of the image content type, and the image content type can better represent the user's personalized needs.

[0059] Optionally, the super-resolution chip may use the image content type with the highest usage frequency as the target content type, or may use the image content type with the second highest usage frequency as the target content type, but is not limited thereto.

[0060] In one embodiment, obtaining a user personalized image through a super-resolution chip includes: obtaining a user personalized image sent by a target application through a super-resolution chip; the target application is determined by the electronic device from each candidate application based on the usage frequency of each candidate application.

[0061] The candidate application is an application installed in the electronic device and used to determine the target application. The target application is an application determined from the candidate applications and used for super-resolution learning of the generated user-personalized image.

[0062] Candidate applications can be third-party applications or system applications. Third-party applications are related software developed by other organizations or individuals who are not the software developers to address the functional deficiencies of a certain software or application. Third-party applications include music applications, social applications, video applications, etc. downloaded by users. System applications are applications that come with the operating system of an electronic device. System applications include photo album applications and desktop applications that come with the operating system of an electronic device. Similarly, the target application determined from the candidate applications can be a third-party application or a system application.

[0063] It is understandable that when a user is using an electronic device, the electronic device may record the user's usage of each candidate application, including the frequency of use, usage time, etc. of each candidate application. The electronic device determines a target application from each candidate application based on the frequency of use of each candidate application. In one embodiment, the electronic device counts the frequency of use of each candidate application, and determines the candidate application with the highest frequency of use as the target application. In another embodiment, the electronic device counts the frequency of use of each candidate application, and determines the candidate application with the second highest frequency of use as the target application. In another embodiment, the electronic device counts the frequency of use of each candidate application within a preset time period, and determines the candidate application with the highest frequency of use within the preset time period as the target application. The manner in which the electronic device determines the target application is not limited and is not limited here.

[0064] The user personalized image sent by the target application is acquired through the super-resolution chip. Optionally, when the target application sends the user personalized image in real time, the user personalized image sent by the target application is acquired in real time through the super-resolution chip; when the target application sends the user personalized image at a fixed time, the user personalized image sent by the target application is acquired at a fixed time through the super-resolution chip.

[0065] In this embodiment, a user personalized image sent by a target application is obtained through a super-resolution chip, and the target application is determined based on the usage frequency of each candidate application. The user's personalized needs can be more accurately judged based on the user's usage frequency of each candidate application, thereby more accurately obtaining a user personalized image that meets the user's personalized needs.

[0066] In one embodiment, Figure 3 As shown, the above method also includes:

[0067] Step 302 : After updating the initial super-resolution algorithm programmed in the super-resolution chip based on the target super-resolution algorithm, an updated super-resolution algorithm is obtained.

[0068] The super-resolution chip obtains an updated super-resolution algorithm, and the updated super-resolution algorithm includes a target super-resolution algorithm.

[0069] Step 304 : when a test image that needs to be processed by super-resolution is obtained, determine the test image content type of the test image.

[0070] The test image is an image that needs to be processed by super-resolution. The test image content type is the type corresponding to the image content of the test image. The test image content type includes at least a portrait type and a landscape type. Further, the portrait type may specifically include a face type, a full-body portrait type, a multi-person photo type, etc. The landscape type may specifically include a mountain type, an ocean type, a forest type, etc.

[0071] When the super-resolution chip obtains a test image that needs to be processed by super-resolution, it obtains the test image content type of the test image. In one embodiment, the electronic device controls the central processor to process the test image using an image content type analysis model to determine the test image content type of the test image, and transmits the test image content type as attribute information of the test image to the super-resolution chip together with the test image. In another embodiment, the electronic device controls the super-resolution chip to process the test image using an image content type analysis model to determine the test image content type of the test image.

[0072] Step 306 : According to the test image content type of the test image, determine a test super-resolution algorithm corresponding to the test image content type from the updated super-resolution algorithms.

[0073] The test super-resolution algorithm is a super-resolution algorithm corresponding to the test image content type, and can perform super-resolution processing on images of the test image content type more accurately.

[0074] After the super-resolution chip obtains the test image content type of the test image, it matches the test image content type with the type processed by each super-resolution algorithm in the updated super-resolution algorithm. When the match is successful, the test super-resolution algorithm matching the test image content type is obtained.

[0075] Step 308: Calling a test super-resolution algorithm through the super-resolution chip to perform super-resolution processing on the test image.

[0076] The super-resolution chip calls the test super-resolution algorithm to perform super-resolution processing on the test image of the test image content type to obtain a higher resolution and clearer image.

[0077] In this embodiment, when the super-resolution chip obtains a test image that needs to be super-resolution processed, it determines the test image content type of the test image, and determines a test super-resolution algorithm corresponding to the test image content type from an updated super-resolution algorithm based on the test image content type of the test image. By calling the test super-resolution algorithm, super-resolution processing can be performed more accurately on the test image belonging to the test image content type to obtain an image with higher resolution and clearer clarity.

[0078] In one embodiment, the super-resolution chip includes an update interface, an update hardware module and an algorithm storage hardware module; the update interface is used to obtain user personalized images; the update hardware module is used to use each user personalized image as a training sample for super-resolution learning to obtain a target super-resolution algorithm; the algorithm storage hardware module is used to store the burned-in initial super-resolution algorithm; the above method also includes: obtaining a first priority of the update hardware module and a second priority of the algorithm storage hardware module; when the first priority is higher than the second priority, updating the burned-in initial super-resolution algorithm in the super-resolution chip based on the target super-resolution algorithm by using the update hardware module.

[0079] The update interface, update hardware module and algorithm storage hardware module are all hardware modules in the super-resolution chip. The super-resolution chip obtains user personalized images through the update interface, and can also obtain test images that need to be super-resolution processed through the update interface. The update interface then transmits the obtained user personalized images to the update hardware module, and uses each user personalized image as a training sample for super-resolution learning through the update hardware module to obtain the target super-resolution algorithm, and then transmits the target super-resolution algorithm to the algorithm storage hardware module to update the burned initial super-resolution algorithm stored in the algorithm storage hardware module. Among them, the algorithm storage hardware module can be an MTP module. The initial super-resolution algorithm burned in the MTP module belongs to the factory default setting.

[0080] The first priority is the priority of updating the hardware module. The second priority is the priority of the algorithm storage hardware module. Both the first priority and the second priority can be set according to user needs. For example, if the first priority is level three and the second priority is level two, the first priority is higher than the second priority, and the priority of updating the hardware module is higher. For another example, if the first priority is level two and the second priority is level four, the first priority is lower than the second priority, and the priority of updating the hardware module is lower.

[0081] When the first priority is higher than the second priority, it means that the priority of updating the hardware module is higher, that is, it is more important to update the initial super-resolution algorithm burned in the algorithm storage hardware module, and then the initial super-resolution algorithm burned in the super-resolution chip is updated based on the target super-resolution algorithm by updating the hardware module. Optionally, the update method can be to add a target super-resolution algorithm on the basis of the initial super-resolution algorithm, or to replace the initial super-resolution algorithm with the target super-resolution algorithm, but is not limited thereto.

[0082] When the first priority is lower than or equal to the second priority, the initial super-resolution algorithm burned into the algorithm storage hardware module may not be updated, or other operations may be performed according to user instructions.

[0083] In this embodiment, an update interface and an update hardware module are added to the super-resolution chip, and user personalized images can be obtained through the update interface; each user personalized image is used as a training sample for super-resolution learning through the update hardware module to obtain a target super-resolution algorithm, so that when the first priority is higher than the second priority, the initial super-resolution algorithm burned in the super-resolution chip is updated based on the target super-resolution algorithm through the update hardware module, thereby realizing the updating of the initial super-resolution algorithm burned in the super-resolution chip.

[0084] In one embodiment, Figure 4 As shown, the super-resolution chip 402 includes a super-resolution module, an algorithm storage hardware module, an update hardware module and an update interface. The super-resolution chip is connected to the access point (AP) through the update interface, and the access point can obtain user personalized images including third-party applications and system applications. Among them, the system application includes the album application and video application that come with the operating system of the electronic device. The update hardware module is connected to the update interface and the algorithm storage hardware module respectively. The update hardware module receives the user personalized image transmitted by the update interface, uses the user personalized image as a training sample for super-resolution learning, obtains the target super-resolution algorithm, and transmits the target super-resolution algorithm to the algorithm storage hardware module to update the burned initial super-resolution algorithm. The algorithm storage hardware module is used to store the burned initial super-resolution algorithm.

[0085] When the super-resolution chip 402 obtains a test image that needs to be super-resolution processed, the super-resolution module calls the super-resolution algorithm in the algorithm storage hardware module to perform super-resolution processing on the test image, thereby obtaining a higher resolution and clearer image.

[0086] Figure 5 1 is a flow chart of updating the super-resolution algorithm in another embodiment. Step 502, produce a super-resolution chip. The super-resolution chip executes step 504, and the super-resolution chip burns the initial super-resolution algorithm. The server executes step 506, and the server-side super-resolution learning obtains the target super-resolution algorithm, and sends the target super-resolution algorithm to the super-resolution chip in the electronic device. The super-resolution chip executes step 508, and the super-resolution chip updates the super-resolution algorithm. Next, step 510, the super-resolution chip is integrated into the electronic device, and the super-resolution chip in the electronic device can execute step 512, perform super-resolution processing on the test image, and obtain a high-resolution test image 514.

[0087] It should be understood that although Figures 1 to 3The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figures 1 to 3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0088] Figure 6 FIG. 4 is a structural block diagram of an updating device of a super-resolution algorithm according to an embodiment of the present invention. Figure 6 As shown, a super-resolution algorithm updating device is provided, comprising: an image acquisition module 602, an algorithm acquisition module 604 and an updating module 606, wherein:

[0089] The image acquisition module 602 is used to acquire a user-personalized image through a super-resolution chip.

[0090] The algorithm acquisition module 604 is used to perform super-resolution learning using the user's personalized image as a training sample to obtain a target super-resolution algorithm.

[0091] The updating module 606 is used to update the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm.

[0092] The above-mentioned super-resolution algorithm updating device obtains user personalized images through a super-resolution chip; uses the user personalized images as training samples for super-resolution learning, and can obtain a target super-resolution algorithm that meets the user's personalized needs. Based on the newly acquired target super-resolution algorithm, the initial super-resolution algorithm burned in the super-resolution chip can be updated, and the super-resolution algorithm burned in the super-resolution chip can be optimized and improved, and can meet the user's personalized needs for super-resolution processing of images.

[0093] In one embodiment, the algorithm acquisition module 604 is also used to determine the image content type of each user's personalized image; based on the image content type of each user's personalized image, screen out the target content type; use the user's personalized images of the target content type as training samples for super-resolution learning to obtain a target super-resolution algorithm for super-resolution processing of the user's personalized images of the target content type.

[0094] In one embodiment, the algorithm acquisition module 604 is further used to count the number of images belonging to each image content type; based on the number of each image, the target content type is filtered out from each image content type.

[0095] In one embodiment, the image acquisition module 602 is further used to acquire a user-personalized image sent by a target application through a super-resolution chip; the target application is determined by the electronic device from among the candidate applications based on the usage frequency of the candidate applications.

[0096] In one embodiment, the above-mentioned device also includes a super-resolution processing module, which is used to obtain an updated super-resolution algorithm after updating the initial super-resolution algorithm burned in the super-resolution chip based on the target super-resolution algorithm; when a test image that needs to be super-resolution processed is obtained, determine the test image content type of the test image; according to the test image content type of the test image, determine a test super-resolution algorithm corresponding to the test image content type from the updated super-resolution algorithms; call the test super-resolution algorithm through the super-resolution chip to perform super-resolution processing on the test image.

[0097] In one embodiment, the super-resolution chip includes an update interface, an update hardware module, and an algorithm storage hardware module; the update interface is used to obtain user-personalized images; the update hardware module is used to use each user-personalized image as a training sample for super-resolution learning to obtain a target super-resolution algorithm; the algorithm storage hardware module is used to store the burned-in initial super-resolution algorithm. The above-mentioned update module 606 is also used to obtain the first priority of the update hardware module and the second priority of the algorithm storage hardware module; when the first priority is higher than the second priority, the burned-in initial super-resolution algorithm in the super-resolution chip is updated based on the target super-resolution algorithm by the update hardware module.

[0098] The division of the various modules in the above-mentioned updating device of the super-resolution algorithm is only for illustration. In other embodiments, the updating device of the super-resolution algorithm may be divided into different modules as needed to complete all or part of the functions of the above-mentioned updating device of the super-resolution algorithm.

[0099] For the specific definition of the updating device of the super-resolution algorithm, please refer to the definition of the updating method of the super-resolution algorithm in the above text, which will not be repeated here. Each module in the above-mentioned updating device of the super-resolution algorithm can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0100] The present application provides a super-resolution chip, including an algorithm storage hardware module, an update interface and an update hardware module. The algorithm storage hardware module is used to store a burned-in initial super-resolution algorithm. The update interface is used to obtain a user-personalized image. The update hardware module is connected to the update interface and the algorithm storage hardware module, respectively, and is used to use each user's personalized image as a training sample for super-resolution learning to obtain a target super-resolution algorithm, and to update the burned-in initial super-resolution algorithm stored in the algorithm storage hardware module based on the target super-resolution algorithm.

[0101] In one possible implementation, the update interface may be an I / O interface, a USB (Universal Serial Bus) interface, etc. The algorithm storage hardware module may be various memories. The update hardware module may be a processor such as a DSP (Digital Signal Processing), an MCU (Microcontroller Unit), or a CPU (central processing unit).

[0102] In one embodiment, the above-mentioned update hardware module is also used to determine the image content type of each user's personalized image; based on the image content type of each user's personalized image, screen out the target content type; use the user's personalized images of the target content type as training samples for super-resolution learning, and obtain a target super-resolution algorithm for super-resolution processing of the user's personalized images of the target content type.

[0103] In one embodiment, the updating hardware module is further used to count the number of images belonging to each image content type; based on the number of each image, the target content type is filtered out from each image content type.

[0104] In one embodiment, the update interface is further used to obtain a user-personalized image sent by a target application; the target application is determined by the electronic device from among the candidate applications based on the usage frequency of the candidate applications.

[0105] In one embodiment, the super-resolution chip further includes a super-resolution module, which is used to obtain an updated super-resolution algorithm after updating the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm; when a test image that needs to be super-resolution processed is obtained, determine the test image content type of the test image; according to the test image content type of the test image, determine a test super-resolution algorithm corresponding to the test image content type from the updated super-resolution algorithms; and call the test super-resolution algorithm to perform super-resolution processing on the test image.

[0106] In one embodiment, the above-mentioned update hardware module is also used to obtain the first priority of the update hardware module and the second priority of the algorithm storage hardware module, and when the first priority is higher than the second priority, the initial super-resolution algorithm burned in the super-resolution chip is updated based on the target super-resolution algorithm.

[0107] The present application also provides an electronic device, which includes a super-resolution chip. The steps of the method described in the embodiments of the present application can be implemented by the super-resolution chip.

[0108] Figure 7 FIG. 1 is a schematic diagram of the internal structure of a super-resolution chip in one embodiment. Figure 7 As shown, the super-resolution chip includes a processor and a memory connected via a system bus. The processor is used to provide computing and control capabilities to support the operation of the entire super-resolution chip. The memory can be an algorithm storage hardware module, and the algorithm storage hardware module can be an MTP (Multiple-Time Programmable) module. The non-volatile storage medium stores a computer program. The computer program can be executed by the processor to implement a super-resolution algorithm update method provided in each of the following embodiments.

[0109] The implementation of each module in the updating device of the super-resolution algorithm provided in the embodiment of the present application can be in the form of a computer program. The computer program can be run on a super-resolution chip on a terminal or a server. The program module constituted by the computer program can be stored in the memory of the super-resolution chip of the electronic device.

[0110] When the computer program is executed by a processor, the steps of the method described in the embodiments of the present application are implemented.

[0111] The embodiment of the present application also provides a computer-readable storage medium, one or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, enable the processors to perform the steps of the super-resolution algorithm update method.

[0112] A computer program product comprising instructions, when executed on a computer, causes the computer to perform an updating method of a super-resolution algorithm.

[0113] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0114] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A super-resolution chip, characterized in that: include: Algorithm storage hardware module, used to store the burned-in initial super-resolution algorithm; Update the interface to obtain the user personalized image sent by the target application; The target application is determined by the electronic device from each candidate application based on the usage frequency of each candidate application, and the user personalized image is an image representing the personalized needs of the user; The update hardware module is connected to the update interface and the algorithm storage hardware module respectively, and is used to determine the image content type of each of the user personalized images within a preset time period; count the type usage frequency of each image content type within the preset time period, and screen out a target content type from each image content type based on the type usage frequency; the type usage frequency refers to the frequency at which the image content type is used, and the target content type refers to the content type of the user personalized image used for super-resolution learning; the user personalized image of the target content type is used as a training sample for super-resolution learning to obtain a target super-resolution algorithm for super-resolution processing of the user personalized image of the target content type, obtain a first priority of the update hardware module, and obtain a second priority of the algorithm storage hardware module, and when the first priority is higher than the second priority, update the burned initial super-resolution algorithm stored in the algorithm storage hardware module based on the target super-resolution algorithm.

2. The super-resolution chip according to claim 1, characterized in that: The updating hardware module is further used for counting the number of images belonging to each of the image content types; and filtering out a target content type from each of the image content types based on the number of each of the images.

3. The super-resolution chip according to claim 1, characterized in that: The super-resolution chip also includes: A super-resolution module is used to obtain an updated super-resolution algorithm after updating the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm; when a test image that needs to be super-resolution processed is obtained, determine the test image content type of the test image; according to the test image content type of the test image, determine a test super-resolution algorithm corresponding to the test image content type from the updated super-resolution algorithms; and call the test super-resolution algorithm to perform super-resolution processing on the test image.

4. The super-resolution chip according to claim 1, characterized in that: The candidate application with the highest frequency of use is determined as the target application.

5. A method for updating a super-resolution algorithm, characterized in that: include: Acquiring a user-personalized image sent by a target application through an update interface of a super-resolution chip; the super-resolution chip includes an update interface, an update hardware module, and an algorithm storage hardware module, wherein the algorithm storage hardware module is used to store a burned-in initial super-resolution algorithm; The target application is determined by the electronic device from each candidate application based on the usage frequency of each candidate application, and the user personalized image is an image representing the personalized needs of the user; The updating hardware module is used to determine the image content type of each of the user personalized images within a preset time period; the type usage frequency of each image content type within the preset time period is counted, and based on the type usage frequency, a target content type is screened out from each image content type; the type usage frequency refers to the frequency at which the image content type is used, and the target content type refers to the content type of the user personalized image used for super-resolution learning; the user personalized images of the target content type are used as training samples for super-resolution learning to obtain a target super-resolution algorithm for super-resolution processing of the user personalized images of the target content type; the first priority of the updating hardware module and the second priority of the algorithm storage hardware module are obtained; when the first priority is higher than the second priority, the initial super-resolution algorithm burned into the super-resolution chip is updated based on the target super-resolution algorithm.

6. The method according to claim 5, characterized in that Based on the image content type of each of the user personalized images, a target content type is screened out, including: Counting the number of images belonging to each of the image content types; Based on the number of each of the images, a target content type is screened out from the content types of each of the images.

7. The method according to claim 5, characterized in that The method further comprises: After updating the initial super-resolution algorithm burned into the super-resolution chip based on the target super-resolution algorithm, an updated super-resolution algorithm is obtained; When a test image that needs to be super-resolution processed is acquired, determining a test image content type of the test image; Determining, according to a test image content type of the test image, a test super-resolution algorithm corresponding to the test image content type from the updated super-resolution algorithms; The test super-resolution algorithm is called by the super-resolution chip to perform super-resolution processing on the test image.

8. An electronic device, characterized in that: Comprising the super-resolution chip as claimed in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 5 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 5 to 7 are implemented.

Images