Image processing method, device, electronic device and computer readable storage medium

Abstract

The present invention relates to an image processing method, device, electronic device and computer-readable storage medium. In the process of processing an image to be processed by the method, which target functional unit processes the image to be processed and the parameters of the target functional unit are determined by the content output by the functional unit determination network. For engineers, it is no longer necessary to spend a lot of energy to determine the processing order between the various functional units and the unit parameters of each functional unit. Therefore, the workload of engineers can be reduced, thereby alleviating the debugging pressure of engineers.

Description

Technical Field

[0001] The present application belongs to the field of image processing, and specifically relates to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] After the original image is acquired through an image acquisition device such as a camera or a capture device, in order to achieve a better effect when the original image is used later, it is generally necessary to process the original image before subsequent processing. In the process of image processing, it is generally necessary to process multiple image attributes, such as the color, contrast, saturation, brightness and other image attributes of the original image. Therefore, when processing the original image, it is generally necessary for multiple different types of functional module units to work together.

[0003] In the prior art, the image processing task of the original image is generally performed by an Image Signal Processor (ISP) channel. The ISP channel includes a plurality of functionally independent functional module units, and the plurality of functionally independent functional module units are composed in a certain order.

[0004] In actual use, the parameters of each functional module unit need to be debugged by engineers according to the usage scenario. Since the processing of images by different functional modules is often coupled and affects each other, it often happens that after the parameters of one functional module unit are debugged, the parameters of another functional module unit cannot be debugged. In addition, since the processing order of the functional module units and the parameters of each functional module unit may be different for different original images (usage scenarios), when using traditional ISP channels to process original images, engineers need to make a lot of modifications to the ISP channels, which causes great debugging pressure for engineers. Summary of the invention

[0005] In view of this, the purpose of the present application is to provide an image processing method, device, electronic device and computer-readable storage medium, which can reduce the debugging workload of engineers and reduce the debugging pressure of engineers when performing image processing through this method.

[0006] The embodiment of the present application is implemented as follows:

[0007] In a first aspect, an embodiment of the present application provides an image processing method, the method comprising: obtaining a current image to be processed; determining current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network; and processing the current image to be processed according to the current target functional unit information.

[0008] In the above process, since which target functional unit processes the image to be processed and the parameters of the target functional unit are determined by the content output by the functional unit determination network, engineers no longer need to spend a lot of energy to determine the processing order between the functional units and the unit parameters of each functional unit. Therefore, the workload of engineers can be reduced, thereby alleviating the debugging pressure of engineers.

[0009] In combination with the first aspect embodiment, in a possible implementation, the function unit determination network includes a unit type determination subnetwork and a unit parameter determination subnetwork; the current target function unit information is determined by inputting the current image to be processed into a pre-trained function unit determination network, including: inputting the current image to be processed into the unit type determination subnetwork and the unit parameter determination subnetwork respectively; determining the category of the target function unit used to process the current image to be processed through the unit type determination subnetwork, and determining the unit parameters corresponding to the target function unit through the unit parameter determination subnetwork; the target function unit information includes the category of the target function unit and the unit parameters corresponding to the target function unit.

[0010] In combination with the first aspect embodiment, in a possible implementation manner, the processing of the current image to be processed according to the current target functional unit information includes: selecting a functional unit corresponding to the category of the target functional unit, and debugging the corresponding functional unit through the unit parameters corresponding to the target functional unit; inputting the current image to be processed into the corresponding functional unit after debugging for processing.

[0011] In combination with the first aspect embodiment, in a possible implementation manner, before the current target functional unit information is determined by inputting the current image to be processed into the pre-trained functional unit determination network, the method further includes: judging whether the current image to be processed meets the condition of stopping being processed; when the judgment result is no, the current target functional unit information is determined by inputting the current image to be processed into the functional unit determination network; accordingly, after the current image to be processed is processed according to the current target functional unit information, the image obtained is the image to be processed in the next stage. In the embodiment of the present application, the image processing is performed on the image to be processed for multiple times in a cycle, and the device judges whether to stop the cycle according to the current image to be processed, and the current target functional unit information determined each time is independent of each other, so that the workload of the engineer in determining the processing order between each functional unit can be saved, thereby reducing the debugging pressure of the engineer. In combination with the first aspect embodiment, in a possible implementation manner, judging whether the current image to be processed meets the condition of stopping being processed includes: scoring the image quality of the current image to be processed according to the pre-acquired image quality scoring rule; when it is determined that the score is less than the scoring threshold, it is determined that the current image to be processed does not meet the condition of stopping being processed.

[0012] In combination with the first aspect embodiment, in a possible implementation manner, determining whether the current image to be processed meets the conditions for stopping being processed includes: obtaining the number of times the current image to be processed has been processed before this time; when the number is less than a number threshold, determining that the current image to be processed does not meet the conditions for stopping being processed.

[0013] In combination with the embodiment of the first aspect, in a possible implementation manner, determining whether the current image to be processed meets the conditions for stopping being processed includes: obtaining the number of times the current image to be processed has been processed before this time; scoring the image quality of the current image to be processed according to a pre-acquired image quality scoring rule; when the number of times is less than a number threshold and the score is less than a score threshold, determining that the current image to be processed does not meet the conditions for stopping being processed.

[0014] In combination with the first aspect embodiment, in a possible implementation, the method further includes: when the image to be processed meets the condition for stopping being processed, stopping processing the current image to be processed, and determining the current image to be processed as the final image.

[0015] In a second aspect, an embodiment of the present application provides an image processing device, the device comprising: an acquisition module, an execution module and a processing module. The acquisition module is used to acquire a current image to be processed; the execution module is used to determine the current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network; the processing module is used to process the current image to be processed according to the current target functional unit information.

[0016] In combination with the second aspect embodiment, in a possible implementation, the functional unit determination network includes a unit type determination subnetwork and a unit parameter determination subnetwork; the execution module is used to input the current image to be processed into the unit type determination subnetwork and the unit parameter determination subnetwork respectively; the category of the target functional unit used to process the current image to be processed is determined by the unit type determination subnetwork, and the unit parameters corresponding to the target functional unit are determined by the unit parameter determination subnetwork; the target functional unit information includes the category of the target functional unit and the unit parameters corresponding to the target functional unit.

[0017] In combination with the second aspect embodiment, in a possible implementation, the processing module is used to select a functional unit corresponding to the category of the target functional unit, and debug the corresponding functional unit through the unit parameters corresponding to the target functional unit; and input the current image to be processed into the corresponding functional unit after debugging for processing.

[0018] In combination with the second aspect embodiment, in a possible implementation, the device further includes a judgment module for judging whether the current image to be processed meets the condition for stopping being processed; when the judgment result is no, the current target functional unit information is determined by inputting the current image to be processed into the functional unit determination network, and the current image to be processed is processed according to the current target functional unit information, and the obtained image is the image to be processed in the next stage. In combination with the second aspect embodiment, in a possible implementation, the judgment module is used to score the image quality of the current image to be processed according to the pre-acquired image quality scoring rules; when it is determined that the score is less than the scoring threshold, it is determined that the current image to be processed does not meet the condition for stopping being processed.

[0019] In combination with the second aspect embodiment, in a possible implementation, the judgment module is used to obtain the number of times the current image to be processed has been processed before this time; when the number is less than the number threshold, it is determined that the current image to be processed does not meet the conditions for stopping processing.

[0020] In combination with the second aspect embodiment, in a possible implementation, the judgment module is used to obtain the number of times the current image to be processed has been processed before this time; score the image quality of the current image to be processed according to the pre-acquired image quality scoring rules; when the number of times is less than the number threshold and the score is less than the score threshold, determine that the current image to be processed does not meet the conditions for stopping processing.

[0021] In combination with the second aspect embodiment, in a possible implementation manner, the execution module is further used to stop processing the current image to be processed when the judgment result is yes, and determine the current image to be processed as the final image.

[0022] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are connected; the memory is used to store programs; the processor calls the program stored in the memory to execute the method provided in the above-mentioned first aspect embodiment and / or any possible implementation method in combination with the first aspect embodiment.

[0023] In a fourth aspect, an embodiment of the present application further provides a non-volatile computer-readable storage medium (hereinafter referred to as computer-readable storage medium), on which a computer program is stored. When the computer program is run by a computer, it executes the method provided by the above-mentioned first aspect embodiment and / or any possible implementation method in combination with the first aspect embodiment.

[0024] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by implementing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings without creative work. As shown in the drawings, the above and other purposes, features and advantages of the present application will be clearer. The same reference numerals indicate the same parts in all the drawings. The drawings are not deliberately scaled to the actual size, and the focus is on showing the main purpose of the present application.

[0026] Figure 1 A flowchart of an image processing method provided in an embodiment of the present application is shown.

[0027] Figure 2AOne of the structural block diagrams of the functional unit determination network provided in an embodiment of the present application is shown.

[0028] Figure 2B The second structural block diagram of the functional unit determining the network provided in the embodiment of the present application is shown.

[0029] Figure 3 The basic sub-network structure framework provided by the embodiment of the present application is shown.

[0030] Figure 4 A structural block diagram of an image processing device provided in an embodiment of the present application is shown.

[0031] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown.

[0032] Icon: 100-electronic device; 110-processor; 120-memory; 200-functional unit determination network; 210-unit type determination subnetwork; 220-unit parameter determination subnetwork; 230-evaluation score output subnetwork; 240-environmental parameter subnetwork; 300-basic subnetwork structure framework; 310-convolutional layer; 320-fully connected layer; 400-image processing device; 410-acquisition module; 420-execution module; 430-processing module. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0034] It should be noted that similar numbers and letters represent similar items in the following figures, so once an item is defined in one figure, it does not need to be further defined and explained in the subsequent figures. At the same time, in the description of this application, relational terms such as "first", "second", etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the term "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements includes not only those elements, but also includes other elements that are not clearly listed, or also includes elements inherent to such process, method, article or equipment. In the absence of more restrictions, the elements defined by the statement "comprise one..." do not exclude the existence of other identical elements in the process, method, article or equipment including the elements.

[0035] Furthermore, the term "and / or" in this application is merely a term used to describe the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone.

[0036] In addition, the defects existing in the image processing solutions in the prior art (which cause greater debugging pressure for engineers) are the results obtained by the applicant after practice and careful research. Therefore, the process of discovering the above defects and the solutions to the above defects proposed in the embodiments of the present application below should be recognized as the applicant's contribution to this application.

[0037] In order to solve the above problems, the embodiments of the present application provide an image processing method, device, electronic device and computer-readable storage medium, which can reduce the debugging workload of engineers and reduce the debugging pressure of engineers.

[0038] The technology can be implemented by using corresponding software, hardware or a combination of software and hardware. The following is a detailed introduction to the embodiments of the present application.

[0039] In the embodiment of the present application, when an original image is acquired and needs to be processed, it is generally necessary to perform multiple rounds of image processing on the original image.

[0040] Among them, the original image can be an image transmitted from an image acquisition device; it can also be an image saved from the cloud or locally and has not been processed by the image processing method proposed in this application.

[0041] The following will take one round of processing flow as an example to introduce the image processing method provided in the embodiment of the present application.

[0042] Please refer to Figure 1 The image processing method provided in the application embodiment may include the following contents.

[0043] Step S110: Acquire the current image to be processed.

[0044] As mentioned above, in the embodiment of the present application, after the original image is acquired, it is necessary to perform multiple image processing on the original image. Therefore, the content referred to by the current image to be processed mentioned in the embodiment of the present application changes in real time with the changing time of the number of times the same image is processed.

[0045] Optionally, when the current image processing flow is the first image processing for the original image, correspondingly, the current image to be processed refers to the original image.

[0046] Optionally, when the current image processing flow is the Nth image processing for the original image, correspondingly, the current image to be processed refers to an image obtained after the original image is processed N-1 times.

[0047] That is to say, in the embodiment of the present application, the image to be processed processed by each image processing process is the processed image obtained by the previous image processing process, and the processed image obtained by this image processing process serves as the image to be processed for the next stage of the image processing process.

[0048] After the current image to be processed is acquired, in some implementations, the processing flow shown in the subsequent step S120 may be directly executed on the current image to be processed.

[0049] In other implementations, after the current image to be processed is acquired, step S111 may be further executed, and the judgment result may be used to determine whether to execute the processing flow shown in subsequent step S120 for the current image to be processed.

[0050] Wherein, step S111 may be: determining whether the current image to be processed meets the condition for stopping being processed.

[0051] Since the image processing process involved in the embodiment of the present application is an iterative processing process, in order to prevent unlimited processing of the same image, it is necessary to predetermine the stop condition of the iterative processing, that is, to determine the condition under which the current image to be processed stops being processed.

[0052] In some embodiments, after the current image to be processed is acquired, the image quality of the current image to be processed can be scored according to the image quality scoring rules acquired in advance. If the scoring score X obtained after the image to be processed is scored is less than the scoring threshold Y, it means that the image to be processed has not yet met the image quality requirements and needs to continue image processing. Correspondingly, at this time, it can be determined that the current image to be processed does not meet the conditions for stopping processing; otherwise, it means that the image to be processed has met the image quality requirements and it can be determined that the current image to be processed meets the conditions for stopping processing.

[0053] Among them, the image quality scoring rule can directly adopt the existing third-party image quality scoring rule that scores various image quality attributes.

[0054] In this rule, a corresponding threshold interval and a corresponding score are set in advance for each image quality attribute of the image. If a certain image quality attribute of the image to be rated reaches the corresponding threshold interval, the score of the image quality attribute of the image to be rated is the score corresponding to the threshold interval. Of course, when the third-party image quality rating rule includes multiple image quality attributes, the final rating score is the weighted sum of the scores corresponding to each image quality attribute.

[0055] In addition, image quality scoring rules can also be specified by engineers based on actual conditions.

[0056] The image quality attributes may include, but are not limited to: one or more of color, contrast, saturation, hue, brightness, sharpness, etc.

[0057] In addition, in some implementations, after each iterative processing of the same original image, the number of times the original image has been processed may be recorded, that is, the number of times the original image has been processed.

[0058] In this implementation, after the current image to be processed is obtained, the number of times N that the original image corresponding to the current image to be processed has been processed before this time can be obtained, and it can be determined whether N is less than a preset number threshold M.

[0059] If N is less than M, it means that the original image corresponding to the image to be processed has not been over-processed, and the current image to be processed can continue to be processed. Correspondingly, at this time, it can be determined that the current image to be processed does not meet the conditions for stopping processing; otherwise, it can be determined that the current image to be processed meets the conditions for stopping processing.

[0060] In addition, in some implementations, the above two methods may be combined to determine whether the current image to be processed meets the condition for stopping being processed.

[0061] For example, when it is determined that the scoring score X obtained after the current image to be processed is less than the scoring threshold Y, and when it is determined that the number of times N that the original image corresponding to the current image to be processed has been processed before this time is less than the number threshold M, it is determined that the current image to be processed does not meet the conditions for stopping processing; otherwise, it can be determined that the current image to be processed meets the conditions for stopping processing.

[0062] In this implementation, as long as X is greater than or equal to Y or N is greater than or equal to M, it can be determined that the current image to be processed meets the condition to stop being processed. At this time, in order to save computing resources as much as possible (it can be understood that the computing resources required to obtain the number of times N that the original image corresponding to the current image to be processed has been processed before this time are less than the computing resources required to calculate the score X of the current image to be processed), in some implementations, after obtaining the current image to be processed, the relationship between the number of times N that the original image corresponding to the current image to be processed has been processed before this time and the number threshold M can be determined.

[0063] When N is greater than or equal to M, it can be directly concluded that the current image to be processed meets the condition for stopping processing, and there is no need to calculate the score X of the current image to be processed.

[0064] When N is less than M, the score X of the current image to be processed is calculated again, and the relationship between X and the score threshold Y is determined, and then based on the relationship between X and Y, it is determined whether the current image to be processed meets the condition for stopping processing.

[0065] Correspondingly, under this implementation, if the judgment result for the current image to be processed is that the condition for stopping processing is not satisfied, it means that it is first determined that the number of times N that the original image corresponding to the current image to be processed has been processed before this time is less than the number threshold M, and then it is determined that the score X of the current image to be processed is less than the score threshold Y.

[0066] In some embodiments of the present application, when it is determined through any of the above embodiments that the current image to be processed meets the conditions for stopping processing, it means that the current image to be processed has met the requirements. At this time, the processing of the current image to be processed can be stopped, and the current image to be processed can be determined as the final image, and other image tasks can be performed based on the final image.

[0067] In addition, in some embodiments of the present application, when it is determined through the above process that the current image to be processed does not meet the condition for stopping being processed, step S120 can be performed based on the current image to be processed.

[0068] Step S120: Determine current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network.

[0069] In an embodiment of the present application, each time image processing is performed on an image to be processed, only the content corresponding to a picture quality attribute category of the image to be processed is processed. Accordingly, each time image processing is performed on an image to be processed, the debugging information that needs to be determined includes: the picture quality attribute category that needs to be processed, and the parameters needed to use in the process of processing the picture quality attribute category.

[0070] Specifically, in the implementation mode of the present application, in order to reduce the workload of engineers, a neural network (hereinafter referred to as the functional unit determination network) can be used to determine some debugging information (hereinafter referred to as the target functional unit information) required for image processing of the current image to be processed, thereby reducing the debugging pressure on engineers.

[0071] Please refer to Figure 2A The functional unit determination network 200 may include a unit type determination subnetwork 210 and a unit parameter determination subnetwork 220. The unit type determination subnetwork 210 and the unit parameter determination subnetwork 220 are independent of each other and have no connection relationship.

[0072] Among them, after the current image to be processed is input into the unit type determination subnetwork 210 and the unit parameter determination subnetwork 220 respectively, the unit type determination subnetwork 210 outputs the probability that each existing functional unit needs to process the current image to be processed, and the unit parameter determination subnetwork 220 outputs the unit parameters required by each existing functional unit when processing the current image to be processed this time.

[0073] It is worth pointing out that the existing functional units process different types of image quality attributes. For example, there are seven functional units, namely functional unit 1, functional unit 2, functional unit 3, functional unit 4 and functional unit 5.

[0074] Among them, function unit 1 is responsible for adjusting color, function unit 2 is responsible for adjusting contrast, function unit 3 is responsible for adjusting saturation, function unit 4 is responsible for adjusting sharpness, and function unit 5 is responsible for adjusting brightness.

[0075] After the unit type determines the probability of each functional unit output by the subnetwork 210 for the current image to be processed, the category corresponding to the functional unit with the highest probability can be determined as the category of the target functional unit, so that the functional unit corresponding to the category of the target functional unit can subsequently perform corresponding image quality attribute processing on the current image to be processed.

[0076] In addition, after the unit parameters required by each functional unit output by the unit parameter determination sub-network 220 for the current image to be processed in this processing process, the unit parameters corresponding to the current target functional unit can also be determined.

[0077] That is, after the current image to be processed is input into the function unit determination network 200 , the current target function unit information can be determined. The target function unit information includes: the type of the target function unit and the unit parameters corresponding to the target function unit.

[0078] Which functional unit processes the current image to be processed is determined by the category of the target functional unit; the unit parameters of the target functional unit in this processing are determined by the unit parameters corresponding to the target functional unit.

[0079] Step S130: Process the current image to be processed according to the current target functional unit information.

[0080] After determining the target functional unit information required for processing the current image to be processed (i.e., the current target functional unit information), the functional unit corresponding to the category of the target functional unit can be selected from the existing functional units, and the corresponding functional unit can be debugged through the unit parameters corresponding to the target functional unit. Then, the current image to be processed is input into the corresponding functional unit after debugging for image processing.

[0081] If, before the current image to be processed is processed, it is determined that the current image to be processed does not meet the condition for stopping processing, then the image obtained after processing in step S130 is the image to be processed in the next stage. In other words, the currently obtained image to be processed in the next stage may continue to be processed.

[0082] As to whether the image to be processed in the next stage needs to be further processed, the next stage continues to make a judgment according to the content introduced in the above step S111.

[0083] An image processing method provided by an embodiment of the present application, since in the process of processing an image to be processed, which target functional unit processes the image to be processed and the parameters of the target functional unit are determined by the content output by the functional unit determination network 200, engineers no longer need to spend a lot of energy to determine the processing order between the various functional units and the unit parameters of each functional unit. Therefore, the workload of engineers can be reduced, thereby alleviating the debugging pressure of engineers.

[0084] In addition, in order to enable the functional unit determination network 200 to have the above-mentioned ability to output the probability of each functional unit and the unit parameters corresponding to each functional unit, the training process of the functional unit determination network 200 will be introduced below.

[0085] Please refer to Figure 2BDuring the training phase, the functional unit determination network 200 may further include an evaluation score output subnetwork 230 and an environment parameter subnetwork 240 .

[0086] Among them, the unit type determination subnetwork 210, the unit parameter determination subnetwork 220, the evaluation score output subnetwork 230 and the environmental parameter subnetwork 240 are independent of each other and have no connection relationship.

[0087] In addition, the structural frameworks of the unit type determination subnetwork 210, the unit parameter determination subnetwork 220, the evaluation score output subnetwork 230, and the environmental parameter subnetwork 240 can all be adopted as follows: Figure 3 The basic sub-network structure framework 300 shown includes a convolutional layer 310 and a fully connected layer 320 .

[0088] Among them, the output of the unit type determination subnetwork 210 is N channels, N is determined by the number of existing functional units. For example, in the previous example, there are 5 functional units, then the output of the unit type determination subnetwork 210 is 5 channels, and the value of each channel represents the probability of each functional unit being used to process the current input image to be processed.

[0089] The output of the unit parameter determination subnetwork 220 is N channels, where N is determined by the number N of existing functional units and the unit parameters of each functional unit. For example, there are 5 functional units, of which functional unit 1 requires 1 unit parameter, functional unit 2 requires 1 unit parameter, functional unit 3 requires 2 unit parameters, functional unit 4 requires 1 unit parameter, and functional unit 5 requires 1 unit parameter, then the output of the unit parameter determination subnetwork 220 is 6 channels.

[0090] The output of the evaluation score output subnetwork 230 is 1 channel, which is used to represent the score of each image processing.

[0091] The output of the environmental parameter subnetwork 240 is 1-channel.

[0092] Before the training process, it is necessary to obtain training samples, wherein each training sample includes two images, namely, the original image and the standard processed image obtained after the original image is processed by professional image processing personnel.

[0093] During the training process, the traditional reinforcement learning training framework was adopted.

[0094] Specifically, after the original image in each training sample is input into the training unit type determination subnetwork 210 and the unit parameter determination subnetwork 220, the original image is processed according to the obtained result to obtain an intermediate processed image. The intermediate processed image is then input into the evaluation score output subnetwork 230 and the environmental parameter subnetwork 240.

[0095] Among them, the training unit type determination subnetwork 210 and the unit parameter determination subnetwork 220 correspond to the "actor" part in the reinforcement learning framework. In each iteration, different strategies are selected as outputs according to different input images, and the gradient of the policy gradient is used to update the internal parameters.

[0096] The evaluation score output subnetwork 230 corresponds to the "critic" part in the reinforcement learning framework, and is responsible for evaluating the quality of the entire processing flow of the training unit type determination subnetwork 210 and the unit parameter determination subnetwork 220 on the input image.

[0097] In traditional reinforcement learning, the environment is known, but in the application scenario proposed in this application, the environment is unknown. Therefore, the environment parameter subnetwork 240 plays the role of simulating the environment to evaluate the image quality after the complete processing flow.

[0098] The training goal of the environmental parameter subnetwork 240 is to make the intermediate processed image input therein close to the standard processed image corresponding to the intermediate processed image, and to update the processing score value output by the evaluation score output subnetwork 230 by comparing the similarity between the currently input intermediate processed image and the corresponding standard processed image.

[0099] Subsequently, the loss functions obtained by the evaluation score output subnetwork 230 and the environmental parameter subnetwork 240 are back-propagated in the training unit type determination subnetwork 210 and the unit parameter determination subnetwork 220, so as to correct the parameters in the training unit type determination subnetwork 210 and the unit parameter determination subnetwork 220.

[0100] like Figure 4 As shown, the embodiment of the present application further provides an image processing device 400 , which may include: an acquisition module 410 , an execution module 420 , and a processing module 430 .

[0101] An acquisition module 410 is used to acquire the current image to be processed;

[0102] An execution module 420 is used to determine current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network;

[0103] The processing module 430 is used to process the current image to be processed according to the current target functional unit information.

[0104] In a possible implementation, the functional unit determination network includes a unit type determination subnetwork and a unit parameter determination subnetwork; the execution module 420 is used to input the current image to be processed into the unit type determination subnetwork and the unit parameter determination subnetwork respectively; the category of the target functional unit used to process the current image to be processed is determined by the unit type determination subnetwork, and the unit parameters corresponding to the target functional unit are determined by the unit parameter determination subnetwork; the target functional unit information includes the category of the target functional unit and the unit parameters corresponding to the target functional unit.

[0105] In a possible implementation, the processing module 430 is used to select a functional unit corresponding to the category of the target functional unit, and debug the corresponding functional unit through the unit parameters corresponding to the target functional unit; and input the current image to be processed into the corresponding functional unit after debugging for processing.

[0106] In a possible implementation, the device further includes a judgment module for judging whether the current image to be processed satisfies the condition for stopping being processed; when the judgment result is no, the current target functional unit information is determined by inputting the current image to be processed into the functional unit determination network, and the current image to be processed is processed according to the current target functional unit information, and the obtained image is the image to be processed in the next stage.

[0107] In a possible implementation, the judgment module is used to score the image quality of the current image to be processed according to a pre-acquired image quality scoring rule; when it is determined that the score is less than a scoring threshold, it is determined that the current image to be processed does not meet the conditions for stopping processing.

[0108] In a possible implementation, the judgment module is used to obtain the number of times the current image to be processed has been processed before this time; when the number is less than a number threshold, it is determined that the current image to be processed does not meet the condition for stopping being processed.

[0109] In a possible implementation, the judgment module is used to obtain the number of times the current image to be processed has been processed before this time; score the image quality of the current image to be processed according to a pre-acquired image quality scoring rule; when the number of times is less than a number threshold and the score is less than a score threshold, determine that the current image to be processed does not meet the conditions for stopping processing.

[0110] In a possible implementation manner, the execution module 420 is further configured to stop processing the current image to be processed and determine the current image to be processed as the final image when the judgment result is yes.

[0111] The image processing device 400 provided in the embodiment of the present application has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0112] In addition, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the steps included in the above-mentioned image processing method are executed.

[0113] In addition, please refer to Figure 5 The embodiment of the present application also provides an electronic device 100 for implementing the image processing method and device of the embodiment of the present application.

[0114] Optionally, the electronic device 100 may be, but not limited to, a personal computer (PC), a smart phone, a tablet computer, a mobile Internet device (MID), a personal digital assistant, a server, etc. The server may be, but not limited to, a network server, a cloud server, etc.

[0115] The electronic device 100 may include: a processor 110 and a memory 120 .

[0116] It should be noted that Figure 5 The components and structures of the electronic device 100 shown are only exemplary and non-restrictive. The electronic device 100 may also have other components and structures as required. For example, in some cases, the electronic device 100 may also include a display for displaying the image after image processing.

[0117] The processor 110, the memory 120, and other components that may appear in the electronic device 100 are directly or indirectly electrically connected to each other to achieve data transmission or interaction. For example, the processor 110, the memory 120, and other components that may appear may be electrically connected to each other via one or more communication buses or signal lines.

[0118] The memory 120 is used to store programs, for example, a program corresponding to the image processing method mentioned above or an image processing device mentioned above. Optionally, when the image processing device is stored in the memory 120, the image processing device includes at least one software function module that can be stored in the memory 120 in the form of software or firmware.

[0119] Optionally, the software function modules included in the image processing apparatus may also be embedded in the operating system (OS) of the electronic device 100 .

[0120] The processor 110 is used to execute executable modules stored in the memory 120, such as software function modules or computer programs included in the image processing device. After receiving the execution instruction, the processor 110 can execute the computer program, for example, to execute: obtaining the current image to be processed; determining the current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network; and processing the current image to be processed according to the current target functional unit information.

[0121] Of course, the method disclosed in any embodiment of the present application can be applied to the processor 110 or implemented by the processor 110.

[0122] In summary, the image processing method, device, electronic device and computer-readable storage medium proposed in the embodiments of the present invention, since in the process of processing the image to be processed, which target functional unit processes the image to be processed and the parameters of the target functional unit are determined by the content output by the functional unit determination network 200, for engineers, it is no longer necessary to spend a lot of energy to determine the processing order between the various functional units and the unit parameters of each functional unit. Therefore, the workload of engineers can be reduced, thereby alleviating the debugging pressure of engineers.

[0123] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0124] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0125] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0126] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a laptop, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0127] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An image processing method, characterized in that: The method comprises: Get the current image to be processed; Determine the current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network; Processing the current image to be processed according to the current target functional unit information; The functional unit determination network includes a unit type determination subnetwork and a unit parameter determination subnetwork; the method of determining the current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network includes: inputting the current image to be processed into the unit type determination subnetwork and the unit parameter determination subnetwork respectively; determining the category of the target functional unit used for processing the current image to be processed through the unit type determination subnetwork, and determining the unit parameters corresponding to the target functional unit through the unit parameter determination subnetwork; the target functional unit information includes the category of the target functional unit and the unit parameters corresponding to the target functional unit; The processing of the current image to be processed according to the current target functional unit information includes: selecting a functional unit corresponding to the category of the target functional unit, and debugging the corresponding functional unit through the unit parameters corresponding to the target functional unit; and inputting the current image to be processed into the debugged corresponding functional unit for processing.

2. The method according to claim 1, characterized in that Before determining the current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network, the method further includes: Determine whether the current image to be processed meets the condition to stop being processed; When the judgment result is no, the current target functional unit information is determined by inputting the current image to be processed into the functional unit determination network; Correspondingly, after the current image to be processed is processed according to the current target functional unit information, the obtained image is the image to be processed in the next stage.

3. The method according to claim 2, characterized in that The determining whether the current image to be processed satisfies the condition for stopping processing includes: Scoring the image quality of the current image to be processed according to the pre-acquired image quality scoring rules; When the score is less than the score threshold, it is determined that the current image to be processed does not meet the condition for stopping being processed.

4. The method according to claim 2, characterized in that: The determining whether the current image to be processed satisfies the condition for stopping processing includes: Obtain the number of times the current image to be processed has been processed before this time; When the number of times is less than the number threshold, it is determined that the current image to be processed does not meet the condition for stopping being processed.

5. The method according to claim 2, characterized in that: The determining whether the current image to be processed satisfies the condition for stopping processing includes: Obtain the number of times the current image to be processed has been processed before this time; Scoring the image quality of the current image to be processed according to the pre-acquired image quality scoring rules; When the number of times is less than the number threshold and the score is less than the score threshold, it is determined that the current image to be processed does not meet the condition for stopping being processed.

6. The method according to any one of claims 2 to 5, characterized in that: The method further comprises: When the judgment result is yes, the processing of the current image to be processed is stopped, and the current image to be processed is determined as the final image.

7. An image processing device, characterized in that: The device comprises: An acquisition module is used to acquire the current image to be processed; An execution module, used for determining current target functional unit information by inputting the current image to be processed into a pre-trained functional unit determination network; A processing module, used for processing the current image to be processed according to the current target functional unit information; The functional unit determination network includes a unit type determination subnetwork and a unit parameter determination subnetwork; the execution module is specifically used to input the current image to be processed into the unit type determination subnetwork and the unit parameter determination subnetwork respectively; the category of the target functional unit used to process the current image to be processed is determined by the unit type determination subnetwork, and the unit parameter determination subnetwork is used to determine the unit parameter corresponding to the target functional unit; the target functional unit information includes the category of the target functional unit and the unit parameter corresponding to the target functional unit; The processing module is specifically used to select a functional unit corresponding to the category of the target functional unit, and debug the corresponding functional unit through the unit parameters corresponding to the target functional unit; and input the current image to be processed into the debugged corresponding functional unit for processing.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are connected; The memory is used to store programs; The processor calls the program stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a computer, the method according to any one of claims 1 to 6 is executed.

Images