Image sharpening recognition model training, sharpening recognition processing method and device

Abstract

The present disclosure relates to an image sharpening recognition model training, sharpening recognition processing method and device. The image sharpening recognition model training method includes obtaining preset sharpening index data corresponding to a first sample image set and a second sample image set; performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence; determining at least one sharpened sample image pair for each sample image from the sharpened image sequence; inputting the at least one sharpened sample image pair into a sharpening recognition model to be trained to obtain first sharpening index data; determining a sorting penalty loss based on the first sharpening index data; training the sharpening recognition model to be trained based on the sorting penalty loss to obtain an initial sharpening recognition model; and training the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain an image sharpening recognition model. The embodiments of the present disclosure can improve the accuracy of sharpening recognition.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular to an image sharpening recognition model training, sharpening recognition processing method and device. Background Art

[0002] Sharpening is a widely used processing method for multimedia content, such as images and videos, to highlight details and edge information, enhancing clarity. However, due to the varying levels of sharpness found in multimedia content and the varying degrees of sharpness inherent in the content, inappropriate sharpening can degrade image quality. For example, excessive sharpening can cause white fringing near edges, severely impacting viewing experience. For texture-rich multimedia content, sharpening can increase encoding difficulty and incur additional bandwidth costs. Therefore, it is necessary to accurately identify the sharpness level of multimedia content, such as images and videos.

[0003] In related technologies, sharpening recognition based on deep learning models often requires supervised training based on pre-labeled sharpening indicator data, which incurs significant labeling costs and results in low model training efficiency. Furthermore, models that rely solely on sharpening recognition loss to adjust model parameters often fail to effectively guarantee recognition accuracy. Therefore, a more efficient and reliable solution is needed. Summary of the Invention

[0004] The present disclosure provides an image sharpening recognition model training, sharpening recognition processing method and device to at least solve the technical problems of high sharpening recognition model training cost, low training efficiency and poor sharpening recognition accuracy in related technologies. The technical solutions of the present disclosure are as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a sharpening recognition processing method is provided, comprising:

[0006] Obtaining preset sharpening index data corresponding to a first sample image set and a second sample image set, where the second sample image set is a subset of the first sample image set;

[0007] performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image, wherein the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images after each sample image is subjected to sharpening processing at different sharpening degrees;

[0008] Determining at least one sharpened sample image pair for each sample image from the sharpened image sequence corresponding to each sample image; each sharpened sample image pair for each sample image is two sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening degrees;

[0009] Inputting at least one sharpened sample image pair corresponding to each sample image into the sharpening recognition model to be trained for sharpening recognition processing, and obtaining first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair;

[0010] Determining a sorting penalty loss corresponding to each sharpened sample image pair according to the first sharpening index data corresponding to each sharpened sample image pair;

[0011] Based on the sorting penalty loss corresponding to each pair of sharpened sample images, the sharpening recognition model to be trained is trained to obtain an initial sharpening recognition model;

[0012] Based on the second sample image set and the preset sharpening index data, the initial sharpening recognition model is trained to obtain an image sharpening recognition model.

[0013] In an optional embodiment, performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image includes:

[0014] Get multiple sharpening parameters corresponding to different sharpening degrees;

[0015] Based on the multiple sharpening parameters, each sample image is sharpened to obtain a sharpened image sequence corresponding to each sample image.

[0016] In an optional embodiment, performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image includes:

[0017] Inputting each sample image into a preset sharpening processing model for sharpening processing to obtain a sharpened image sequence corresponding to each sample image;

[0018] In which, the preset sharpening processing model includes multiple sharpening processing models connected in sequence; the multiple sharpened sample images in the sharpened image sequence corresponding to each sample image are the output images of the multiple sharpening processing models; the input of the first sharpening processing model among the multiple sharpening processing models is each sample image; the input of the non-first sharpening processing model among the multiple sharpening processing models is the output of the corresponding previous sharpening processing model.

[0019] In an optional embodiment, the method further includes:

[0020] Segmenting a pair of sample image blocks corresponding to each sharpened sample image pair from a common area corresponding to two sharpened sample images in each sharpened sample image pair;

[0021] Inputting at least one sharpened sample image pair corresponding to each sample image into the sharpening recognition model to be trained for sharpening recognition processing to obtain first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair includes: inputting the sample image block pair corresponding to each sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing to obtain first sharpening index data corresponding to each sample image block in each sample image block pair;

[0022] Determining the sorting penalty loss corresponding to each sharpened sample image pair according to the first sharpening index data corresponding to each sharpened sample image pair includes: determining the sorting penalty loss corresponding to each sample image block pair according to the first sharpening index data corresponding to each sample image block pair;

[0023] The training of the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each pair of sharpened sample images to obtain the initial sharpening recognition model includes: training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each pair of sample image blocks to obtain the initial sharpening recognition model.

[0024] In an optional embodiment, the training of the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain the image sharpening recognition model includes:

[0025] Inputting the sample images in the second sample image set into the initial sharpening recognition model to perform sharpening recognition processing, thereby obtaining second sharpening index data corresponding to the sample images in the second sample image set;

[0026] determining a sharpening recognition loss according to the second sharpening index data and the preset sharpening index data;

[0027] Based on the sharpening recognition loss, the initial sharpening recognition model is trained to obtain the image sharpening recognition model.

[0028] According to a second aspect of an embodiment of the present disclosure, a sharpening recognition processing method is provided, comprising:

[0029] Acquire at least one image to be recognized;

[0030] The at least one image to be identified is input into the image sharpening recognition model trained by any image sharpening recognition model training method provided in the first aspect for sharpening recognition processing to obtain third sharpening index data corresponding to each image to be identified, and the third sharpening index data represents the sharpness degree of each image to be identified.

[0031] In an optional embodiment, the method further includes:

[0032] Performing a reduction process on each image to be identified to obtain a reduced image;

[0033] Performing image segmentation on the reduced image to obtain a plurality of image blocks corresponding to each image to be identified;

[0034] Inputting the at least one image to be identified into the image sharpening recognition model for sharpening recognition processing to obtain third sharpening index data corresponding to each image to be identified includes:

[0035] Inputting the plurality of image blocks corresponding to each image to be recognized into the image sharpening recognition model for sharpening recognition processing to obtain fourth sharpening index data corresponding to each image block;

[0036] The plurality of fourth sharpening index data corresponding to each image to be identified are fused to obtain the third sharpening index data corresponding to each image to be identified; the plurality of fourth sharpening index data are the fourth sharpening index data corresponding to the corresponding plurality of image blocks.

[0037] In an optional embodiment, when the at least one image to be recognized is at least one frame of a video image in a video to be recognized, the method further includes:

[0038] The third sharpening index data corresponding to the at least one frame of video image is fused to obtain the sharpening index data corresponding to the video to be identified.

[0039] According to a third aspect of an embodiment of the present disclosure, there is provided an image sharpening recognition model training device, comprising:

[0040] a training data acquisition module configured to acquire preset sharpening index data corresponding to a first sample image set and a second sample image set, where the second sample image set is a subset of the first sample image set;

[0041] a multi-level sharpening processing module configured to perform multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image, wherein the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images after each sample image is sharpened at different degrees of sharpening;

[0042] a sharpened sample image pair determination module configured to determine at least one sharpened sample image pair for each sample image from a sharpened image sequence corresponding to each sample image; each sharpened sample image pair for each sample image is two sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening degrees;

[0043] A first sharpening recognition processing module is configured to input at least one sharpened sample image pair corresponding to each sample image into a sharpening recognition model to be trained to perform sharpening recognition processing, thereby obtaining first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair;

[0044] a sorting penalty loss determination module configured to determine a sorting penalty loss corresponding to each sharpened sample image pair based on the first sharpening index data corresponding to each sharpened sample image pair;

[0045] A first model training module is configured to perform training on the to-be-trained sharpening recognition model based on the sorting penalty loss corresponding to each pair of sharpened sample images to obtain an initial sharpening recognition model;

[0046] The second model training module is configured to train the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain an image sharpening recognition model.

[0047] In an optional embodiment, the multi-level sharpening processing module includes:

[0048] a sharpening parameter acquiring unit, configured to acquire a plurality of sharpening parameters corresponding to different sharpening degrees;

[0049] The first sharpening processing unit is configured to perform sharpening processing on each sample image based on the multiple sharpening parameters to obtain a sharpened image sequence corresponding to each sample image.

[0050] In an optional embodiment, the multi-level sharpening processing module includes:

[0051] A second sharpening processing unit is configured to input each sample image into a preset sharpening processing model for sharpening processing, to obtain a sharpened image sequence corresponding to each sample image;

[0052] In which, the preset sharpening processing model includes multiple sharpening processing models connected in sequence; the multiple sharpened sample images in the sharpened image sequence corresponding to each sample image are the output images of the multiple sharpening processing models; the input of the first sharpening processing model among the multiple sharpening processing models is each sample image; the input of the non-first sharpening processing model among the multiple sharpening processing models is the output of the corresponding previous sharpening processing model.

[0053] In an optional embodiment, the device further comprises:

[0054] The first image segmentation module is configured to segment the same area corresponding to the two sharpened sample images in each sharpened sample image pair into a pair of sample image blocks corresponding to the sharpened sample image pair;

[0055] The first sharpening recognition processing module is further configured to input the sample image block pair corresponding to each sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing, thereby obtaining first sharpening index data corresponding to each sample image block in each sample image block pair;

[0056] The sorting penalty loss determination module is further configured to determine the sorting penalty loss corresponding to each sample image block pair based on the first sharpening index data corresponding to each sample image block pair;

[0057] The first module training module is further configured to perform training on the sharpening recognition model to be trained based on the ranking penalty loss corresponding to each pair of sample image blocks to obtain an initial sharpening recognition model.

[0058] In an optional embodiment, the second model training module includes:

[0059] a first sharpening recognition processing unit configured to input the sample images in the second sample image set into the initial sharpening recognition model to perform sharpening recognition processing, and obtain second sharpening index data corresponding to the sample images in the second sample image set;

[0060] a sharpening recognition loss determining unit, configured to determine a sharpening recognition loss based on the second sharpening index data and the preset sharpening index data;

[0061] The model training unit is configured to train the initial sharpening recognition model based on the sharpening recognition loss to obtain the image sharpening recognition model.

[0062] According to a fourth aspect of an embodiment of the present disclosure, there is provided a sharpening recognition processing apparatus, comprising:

[0063] An image acquisition module to be identified is configured to acquire at least one image to be identified;

[0064] The second sharpening recognition processing module is configured to perform sharpening recognition processing on the at least one image to be recognized input into the image sharpening recognition model trained by any image sharpening recognition model training method provided in the first aspect, and obtain third sharpening index data corresponding to each image to be recognized, wherein the third sharpening index data represents the sharpness degree of each image to be recognized.

[0065] In an optional embodiment, the device further comprises:

[0066] A reduction processing module is configured to perform reduction processing on each image to be recognized to obtain a reduced image;

[0067] a second image segmentation module configured to perform image segmentation on the reduced image to obtain a plurality of image blocks corresponding to each image to be identified;

[0068] The second sharpening recognition processing module includes:

[0069] The second sharpening recognition processing unit is configured to input the plurality of image blocks corresponding to each image to be recognized into the image sharpening recognition model for performing sharpening recognition processing, and obtain fourth sharpening index data corresponding to each image block;

[0070] The index fusion processing unit is configured to perform fusion processing on the multiple fourth sharpening index data corresponding to each image to be identified to obtain the third sharpening index data corresponding to each image to be identified; the multiple fourth sharpening index data are the fourth sharpening index data corresponding to the corresponding multiple image blocks.

[0071] In an optional embodiment, when the at least one image to be recognized is at least one frame of a video image in a video to be recognized, the apparatus further includes:

[0072] The index fusion processing module is configured to perform fusion processing on the third sharpening index data corresponding to the at least one frame of video image to obtain the sharpening index data corresponding to the video to be identified.

[0073] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method as described in any one of the first or second aspects above.

[0074] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is capable of executing any one of the methods described in the first or second aspect of the embodiment of the present disclosure.

[0075] According to a seventh aspect of an embodiment of the present disclosure, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the method as described in any one of the first or second aspects above.

[0076] The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

[0077] In the image sharpening recognition model training process, a multi-level sharpening process is first performed on the sample images in the first sample image set to obtain a sharpened image sequence containing sharpened sample images with different sharpening degrees. Then, a sharpened sample image pair containing two sharpened sample images with different sharpening degrees is determined from the sharpened image sequence, and the sharpened sample image pair is input into the sharpening recognition model to be trained for sharpening recognition process to obtain first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair. The sharpening recognition model to be trained is trained based on the first sharpening index data corresponding to each sharpened sample image pair. This allows the model to learn the differences between different sharpening degrees. Even when the model annotation samples are reduced, the generalization ability of the model can be effectively guaranteed. At the same time, the model is trained based on the preset sharpening index data corresponding to some sample images in the first sample image set (the second sample image set). This can effectively reduce the model annotation cost while also improving the sharpening recognition accuracy of the model. This can effectively ensure the sharpening recognition accuracy of the image sharpening recognition model, improve the subsequent sharpening effect of multimedia content such as images and videos, reduce encoding costs, and improve system performance.

[0078] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0080] Figure 1 Schematic diagram of an application environment of a sharpening recognition processing generation method provided in an embodiment of the present application;

[0081] Figure 2 is a flowchart of a method for training an image sharpening recognition model according to an exemplary embodiment;

[0082] Figure 3 is a schematic diagram of a training process of another image sharpening recognition model according to an exemplary embodiment;

[0083] Figure 4 is a flowchart showing a sharpening recognition processing method according to an exemplary embodiment;

[0084] Figure 5 This is a block diagram of an image sharpening recognition model training device according to an exemplary embodiment;

[0085] Figure 6 is a block diagram of a sharpening recognition processing device according to an exemplary embodiment;

[0086] Figure 7 is a block diagram of an electronic device for image sharpening recognition model training according to an exemplary embodiment;

[0087] Figure 8 The present invention is a block diagram of an electronic device for sharpening recognition processing according to an exemplary embodiment. DETAILED DESCRIPTION

[0088] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0089] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0090] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0091] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0092] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0093] The solutions provided in the embodiments of the present application involve technologies such as deep learning of artificial intelligence. Specifically, they may involve image sharpening recognition model training and sharpening recognition based on deep learning, and are specifically described through the following embodiments:

[0094] See also Figure 1 , Figure 1 1 is a schematic diagram of an application environment of a sharpening recognition processing generation method provided in an embodiment of the present application. The application environment may include at least a server 100 and a terminal 200.

[0095] In an optional embodiment, server 100 can be used to perform training processing of an image sharpening recognition model. Server 100 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud computing services.

[0096] In an optional embodiment, the terminal 200 can be used to provide services such as sharpening recognition based on an image sharpening recognition model. Specifically, the terminal 200 may include, but is not limited to, electronic devices such as smartphones, desktop computers, tablet computers, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, smart wearable devices, vehicle-mounted terminals, and smart TVs. It may also be software running on the above electronic devices, such as applications and applets. The operating systems running on the electronic devices in the embodiments of the present application may include, but are not limited to, Android, iOS, Linux, Windows, etc.

[0097] In addition, it should be noted that Figure 1 What is shown is merely an application environment of the sharpening recognition processing method, and the embodiments of this specification are not limited to the above.

[0098] In the embodiments of this specification, the server 100 and the terminal 200 may be directly or indirectly connected via wired or wireless communication, which is not limited in this application.

[0099] Figure 2 This is a flow chart of a method for training an image sharpening recognition model according to an exemplary embodiment. The method can be used in electronic devices such as terminals and servers. Figure 2 As shown, the following steps may be included:

[0100] In step S201, preset sharpening index data corresponding to the first sample image set and the second sample image set are obtained;

[0101] In a specific embodiment, the first sample image set may include a large number of sample images used to train an image sharpening recognition model. Optionally, in some scenarios where sharpening recognition processing is performed on videos, the sample images in the first sample image set may be video frames from multiple sample videos. In some scenarios where sharpening recognition processing is performed on individually captured images, a large number of individually captured images may be used as sample images in the first sample image set, depending on the actual scenario requirements.

[0102] In a specific embodiment, the second sample image set is a subset of the first sample image set, and the preset sharpening index data corresponding to any sample image in the second sample image set can represent the sharpening degree of the sample image. In a specific embodiment, as shown in Table 1, Table 1 is an illustrative example of the sharpening degree corresponding to different preset sharpening index data provided in an embodiment of the present application.

[0103]

[0104] Table 1

[0105] In step S203, a multi-level sharpening process is performed on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image.

[0106] In a specific embodiment, the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening levels. In an optional embodiment, performing multi-level sharpening processing on each sample image in the first sample image set to obtain the sharpened image sequence corresponding to each sample image may include:

[0107] Get multiple sharpening parameters corresponding to different sharpening degrees;

[0108] Based on multiple sharpening parameters, each sample image is sharpened to obtain a sharpened image sequence corresponding to each sample image.

[0109] In a specific embodiment, the sharpening parameters may be parameters for controlling the intensity of the sharpening effect during the sharpening process; optionally, taking the sharpening process based on Unsharp Mask (USM unsharp filter) as an example, the above-mentioned multiple sharpening parameters may be multiple quantities corresponding to different sharpening effect intensities; optionally, in the process of performing sharpening processing on each sample image in combination with Unsharp Mask, the sharpening radius and threshold (pixel range of the sharpening effect) may be controlled to remain unchanged, and multiple quantities may be combined to generate a sharpened image sequence corresponding to each sample image.

[0110] In addition, it should be noted that the multiple in the embodiments of this specification can be at least two.

[0111] In the above embodiment, by setting multiple sharpening parameters corresponding to different sharpening degrees, each sample image is sharpened separately to obtain a sharpened image sequence corresponding to each sample image, so that multiple sharpened sample images with different sharpening degrees corresponding to each sample image can be quickly obtained.

[0112] In an optional embodiment, performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image may include:

[0113] Input each sample image into a preset sharpening processing model for sharpening processing, and obtain a sharpened image sequence corresponding to each sample image;

[0114] In a specific embodiment, the preset sharpening processing model may include multiple sharpening processing models connected in sequence; the multiple sharpened sample images in the sharpened image sequence corresponding to each sample image are the output images of the multiple sharpening processing models; the input of the first sharpening processing model among the multiple sharpening processing models is each sample image; and the input of the multiple sharpening processing models other than the first sharpening processing model is the output of the corresponding previous sharpening processing model. Specifically, the sharpening degree of the output image of any sharpening processing model is greater than the sharpening degree of the output image of the previous sharpening processing model.

[0115] In a specific embodiment, the sharpening processing model to be trained can be trained in combination with a preset sharpened image and the original image corresponding to the preset sharpened image to obtain a sharpening processing model; then, multiple sharpening processing models are copied, and the multiple sharpening processing models are connected in sequence to obtain the above-mentioned preset sharpening processing model.

[0116] In one embodiment, the sharpening processing model to be trained may be a preset deep learning model, and the specific model structure depth may be set in combination with actual application requirements.

[0117] In the above embodiment, by inputting each sample image into a preset sharpening processing model including multiple sharpening processing models connected in sequence for sharpening processing, and using the output images of the multiple sharpening processing models as the sharpened image sequence corresponding to each sample image, it can be effectively ensured that the sharpened image sequence contains multiple sharpened sample images with different degrees of sharpening.

[0118] In step S205 , at least one sharpened sample image pair for each sample image is determined from the sharpened image sequence corresponding to each sample image.

[0119] In a specific embodiment, two sharpened sample images may be arbitrarily selected from the sharpened image sequence corresponding to each sample image to form a sharpened sample image pair corresponding to the sample image.

[0120] In step S207, at least one sharpened sample image pair corresponding to each sample image is input into the to-be-trained sharpness recognition model for sharpness recognition processing, and first sharpness index data corresponding to each sharpened sample image in each sharpened sample image pair is obtained.

[0121] In a specific embodiment, each sharpened sample image pair of each sample image can be two sharpened sample images obtained by performing sharpening processing on each sample image with different sharpening degrees; the first sharpness index data can represent the sharpening degree of each sharpened sample image.

[0122] In step S209, according to the first sharpness index data corresponding to each sharpened sample image pair, the ranking penalty loss corresponding to each sharpened sample image pair is determined.

[0123] In a specific embodiment, the ranking penalty loss corresponding to each sharpened sample image pair can represent the recognition accuracy of the to-be-trained sharpness recognition model for the sharpening degrees of the two sharpened sample images in the sharpened sample image pair.

[0124] In a specific embodiment, the above-mentioned determination of the ranking penalty loss corresponding to each sharpened sample image pair according to the first sharpness index data corresponding to each sharpened sample image pair can be combined with the following formula:

[0125] Loss = max(0, y1 - y2 + ε)

[0126] Where, Loss represents the ranking penalty loss corresponding to any sharpened sample image pair, y1 represents the first sharpness index data corresponding to the sharpened sample image with a lighter sharpening degree in any sharpened sample image pair, y2 represents the first sharpness index data corresponding to the sharpened sample image with a heavier sharpening degree in any sharpened sample image pair, ε is a preset parameter, and ε is a constant greater than 0, which can enhance the learning ability of the model for sharpness approximation samples, that is, it is required that the model not only considers y1 to be smaller, but also significantly smaller; specifically, when y1 < y2, it means that the model prediction is correct, and y1 - y2 + ε is often less than 0, correspondingly, the ranking penalty loss is equal to 0 and no penalty is generated; on the contrary, when y1 > y2, it means that the model prediction is incorrect and the model needs to be corrected, and the ranking penalty loss is equal to y1 - y2 + ε.

[0127] In step S211, based on the ranking penalty loss corresponding to each sharpened sample image pair, the to-be-trained sharpness recognition model is trained to obtain an initial sharpness recognition model.

[0128] In a specific embodiment, the sorting penalty losses corresponding to all sharpened sample image pairs (the sharpened sample image pairs corresponding to each sample image in the first sample image set) can be added together to obtain a total sorting penalty loss; and based on the total sorting penalty loss, the model parameters can be adjusted in combination with the gradient descent method. Then, based on the adjusted sharpened recognition model to be trained, the above-mentioned step of inputting at least one sharpened sample image pair corresponding to each sample image into the sharpened recognition model to be trained for sharpening recognition processing is repeated to obtain the first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair to the training iteration step of adjusting the model parameters until the first preset convergence condition is met, and the sharpened recognition model to be trained when the first preset convergence condition is met is used as the initial sharpened recognition model.

[0129] In a specific embodiment, the above-mentioned first preset convergence condition can be that the total loss of the sorting penalty is less than or equal to the first preset loss threshold, or the number of training iteration steps reaches the first preset number, etc. Specifically, the first preset loss threshold and the first preset number can be set in combination with the model accuracy and training speed requirements in actual applications.

[0130] In step S213, the initial sharpening recognition model is trained based on the second sample image set and the preset sharpening index data to obtain an image sharpening recognition model.

[0131] In an optional embodiment, the training of the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain the image sharpening recognition model may include:

[0132] Inputting the sample images in the second sample image set into the initial sharpening recognition model for sharpening recognition processing to obtain second sharpening index data corresponding to the sample images in the second sample image set;

[0133] determining a sharpening recognition loss according to the second sharpening index data and the preset sharpening index data;

[0134] Based on the sharpening recognition loss, the initial sharpening recognition model is trained to obtain the image sharpening recognition model.

[0135] In a specific embodiment, the second sharpening index data may represent the sharpening degree of each sample image in the second sample image set. Specifically, the sharpening recognition loss may represent the sharpening recognition accuracy of the initial sharpening recognition model. Specifically, the sharpening recognition loss may be calculated in combination with a preset loss function; optionally, the preset loss function may be set in combination with actual application requirements, such as a mean absolute loss function, a mean square loss function, a cross entropy loss function, etc.

[0136] In a specific embodiment, the above-mentioned training of the initial sharpening recognition model based on the sharpening recognition loss to obtain the image sharpening recognition model may include: updating the model parameters of the initial sharpening recognition model based on the sharpening recognition loss, and repeating the above-mentioned training iterative steps of inputting the sample images in the second sample image set into the initial sharpening recognition model for sharpening recognition processing based on the updated initial sharpening recognition model to obtain the second sharpening index data corresponding to the sample images in the second sample image set, and updating the model parameters of the initial sharpening recognition model based on the sharpening recognition loss, until the second preset convergence condition is met.

[0137] In a specific embodiment, the second preset convergence condition can be that the sharpening recognition loss is less than or equal to the second preset loss threshold, or the number of training iteration steps reaches a second preset number, etc. Specifically, the second preset loss threshold and the second preset number can be set in combination with the model accuracy and training speed requirements in actual applications.

[0138] In the above embodiment, by inputting the sample images in the second sample image set into the initial sharpening recognition model for sharpening recognition processing, the second sharpening index data and the preset sharpening index data are obtained to determine the sharpening recognition loss, and then combined with the sharpening recognition loss, the initial sharpening recognition model is trained, which can better improve the sharpening recognition accuracy of the image sharpening recognition model.

[0139] In an optional embodiment, if Figure 3 As shown, the above method may further include the following steps:

[0140] In step S215, a pair of sample image blocks corresponding to each sharpened sample image pair is segmented from the same region corresponding to the two sharpened sample images in each sharpened sample image pair;

[0141] Correspondingly, in the above-mentioned step S207, at least one sharpened sample image pair corresponding to each sample image is input into the sharpening recognition model to be trained for sharpening recognition processing, and obtaining the first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair may include: inputting the sample image block pair corresponding to each sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing, and obtaining the first sharpening index data corresponding to each sample image block in each sample image block pair.

[0142] Determining the sorting penalty loss corresponding to each sharpened sample image pair according to the first sharpening index data corresponding to each sharpened sample image pair in step S209 may include: determining the sorting penalty loss corresponding to each sample image block pair according to the first sharpening index data corresponding to each sample image block pair;

[0143] In the above step S211, the sharpening recognition model to be trained is trained based on the sorting penalty loss corresponding to each sharpened sample image pair to obtain the initial sharpening recognition model, which may include: training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each sample image block pair to obtain the initial sharpening recognition model.

[0144] In practical applications, the sample images in the first sample image set may be large in size, and accordingly, the sharpened sample images corresponding to the sample images are also large in size. In order to reduce the consumption of computing resources during the training process, the same area corresponding to the two sharpened sample images in each sharpened sample image pair can be segmented to obtain the sample image block pair corresponding to each sharpened sample image pair as the input of the sharpening recognition model to be trained.

[0145] In one embodiment, the sharpening recognition model to be trained may be a pre-set deep learning model, and the specific model structure depth may be set in combination with actual application requirements.

[0146] In a specific embodiment, the specific detailed steps of determining the sorting penalty loss corresponding to each sample image block pair based on the first sharpening index data corresponding to each sample image block pair can be referred to the specific detailed steps of determining the sorting penalty loss corresponding to each sharpened sample image pair based on the first sharpening index data corresponding to each sharpened sample image pair, which will not be repeated here.

[0147] In a specific embodiment, the specific refinement steps of training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each sample image block to obtain the initial sharpening recognition model can be referred to the specific refinement steps of training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each sharpened sample image pair to obtain the initial sharpening recognition model, which will not be repeated here.

[0148] In the above embodiment, the sample image block pairs segmented from each sharpened sample image pair are used as the input of the model in the sharpening recognition training process, which can greatly reduce the consumption of computing resources in the model training process and thus improve the model training efficiency.

[0149] It can be seen from the technical solutions provided by the embodiments of the present specification that, in the process of training the image sharpening recognition model, the present specification first performs multi-level sharpening processing on the sample images in the first sample image set to obtain a sharpened image sequence containing sharpened sample images with different sharpening degrees, and then determines a sharpened sample image pair containing two sharpened sample images with different sharpening degrees from the sharpened image sequence, and inputs the sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing to obtain the first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair, and combines the first sharpening index data corresponding to each sharpened sample image pair to determine the ranking. The sequential penalty loss is used to train the sharpening recognition model to be trained, so that the model can learn the differences between different sharpening degrees. Even when the model annotation samples are reduced, the generalization ability of the model can be effectively guaranteed. At the same time, the model is trained based on the preset sharpening index data corresponding to some sample images in the first sample image set (the second sample image set). On the basis of effectively reducing the model annotation cost, the sharpening recognition accuracy of the model can be improved, thereby effectively ensuring the sharpening recognition accuracy of the image sharpening recognition model, improving the subsequent sharpening effect of multimedia content such as images and videos, reducing encoding costs, and improving system performance.

[0150] Figure 4 This is a flowchart of a sharpening recognition processing method according to an exemplary embodiment. The method can be used in electronic devices such as terminals and servers. Figure 4 As shown, the following steps may be included:

[0151] In step S401, at least one image to be recognized is obtained.

[0152] In a specific embodiment, at least one image to be identified may be an image that requires sharpening and recognition processing. Optionally, in some scenarios where sharpening and recognition processing is performed on a video, the at least one image to be identified may be at least one frame of a video. In some scenarios where sharpening and recognition processing is performed on individually captured images, each individually captured image may be used as an image to be identified based on actual scenario requirements.

[0153] In step S403, at least one image to be identified is input into the image sharpening recognition model for sharpening recognition processing to obtain third sharpening index data corresponding to each image to be identified.

[0154] In a specific embodiment, the above-mentioned third sharpening index data can represent the sharpness degree of each image to be identified; in a specific embodiment, the above-mentioned image sharpening recognition model is obtained by training the initial sharpening recognition model based on the second sample image set and the preset sharpening index data corresponding to the sample images in the second sample image set; the initial sharpening recognition model is obtained by training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to at least one sharpened sample image pair of each sample image in the first sample image set; each sharpened sample image pair of each sample image is two sharpened sample images after each sample image is sharpened with different degrees of sharpening; the second sample image set is a subset of the first sample image set.

[0155] In an optional embodiment, the above method may further include:

[0156] Each image to be identified is reduced to obtain a reduced image;

[0157] Performing image segmentation on the reduced image to obtain multiple image blocks corresponding to each image to be identified;

[0158] Accordingly, the step of inputting at least one image to be identified into the image sharpening recognition model for sharpening recognition processing to obtain the third sharpening index data corresponding to each image to be identified may include:

[0159] Inputting a plurality of image blocks corresponding to each image to be identified into an image sharpening recognition model for sharpening recognition processing to obtain fourth sharpening index data corresponding to each image block;

[0160] The plurality of fourth sharpening index data corresponding to each image to be identified are fused to obtain the third sharpening index data corresponding to each image to be identified; the plurality of fourth sharpening index data are the fourth sharpening index data corresponding to the corresponding plurality of image blocks.

[0161] In a specific embodiment, the image can be segmented after being reduced in size, in conjunction with the input image size requirements of the image sharpening recognition model, to obtain multiple image blocks corresponding to each image to be recognized. The fourth sharpening index data corresponding to each image block can represent the degree of sharpness corresponding to each image block.

[0162] In a specific embodiment, fusing the multiple fourth sharpening index data corresponding to each image to be identified to obtain the third sharpening index data corresponding to each image to be identified may include: summing and averaging the multiple fourth sharpening index data corresponding to each image to be identified to obtain the third sharpening index data corresponding to each image to be identified.

[0163] In the above embodiment, by reducing the size of each image to be identified and then dividing it into model inputs, the amount of calculation in the model sharpening recognition process can be reduced while ensuring that the sharpening information of each image to be identified is fully learned, thereby reducing computing resource consumption and improving system performance.

[0164] In an optional embodiment, when the at least one image to be identified is at least one frame of a video image in a video to be identified, the method further includes:

[0165] The third sharpening index data corresponding to at least one frame of video image is fused to obtain sharpening index data corresponding to the video to be identified.

[0166] In a specific embodiment, fusing the third sharpness index data corresponding to at least one frame of video image to obtain the sharpness index data corresponding to the video to be identified may include: summing and averaging the third sharpness index data corresponding to the at least one frame of video image to obtain the sharpness index data corresponding to the video to be identified. Specifically, the sharpness index data corresponding to the video to be identified may represent a degree of sharpness corresponding to the video to be identified.

[0167] In the above embodiment, by fusing the third sharpening index data corresponding to at least one frame of video image in the video to be identified, the sharpening index data corresponding to the video to be identified can be obtained quickly and accurately, thereby improving the sharpening effect of the video to be identified, reducing the encoding difficulty, and improving the system performance.

[0168] In another optional embodiment, the third sharpening index data corresponding to at least one frame of video image can be directly used as the sharpening index data corresponding to the video to be identified; accordingly, in the subsequent sharpening processing or encoding process of the video, the sharpening processing or encoding can be performed according to the third sharpening index data corresponding to each frame of video image in the video.

[0169] In a specific embodiment, in combination with 300 groups of videos selected by authorization, the image sharpening recognition model of the first model type (the image sharpening recognition model obtained by training only in combination with the sorting penalty loss corresponding to at least one sharp sample image pair of each sample image in the first sample image set), the image sharpening recognition model of the second model type (the image sharpening recognition model obtained by training only based on the preset sharpening index data corresponding to the sample images in the second sample image set and the second sample image set) and the image sharpening recognition model of the third model type of the present application (the image sharpening recognition model obtained by training based on the sorting penalty loss corresponding to at least one sharp sample image pair of each sample image in the first sample image set, combined with the preset sharpening index data corresponding to the sample images in the second sample image set and the second sample image set) are respectively used to verify the sharpening recognition effect. Specifically, the normalization threshold corresponding to the optimal effect of each model (the threshold of the normalized sharpening index data) can be combined to characterize the sharpening recognition effect from the perspective of F1Score (a measurement indicator for classification problems), precision rate and recall rate. Specifically, the results are shown in Table 2. It can be seen that the image sharpening recognition model of the third model type of the present application has the best effect, with the overall F1Scoer reaching more than 80%, and has good accuracy and recall capability.

[0170] Model Type Threshold (normalization) Precision Recall F1Score First model type 0.4 54.5％ 61.4％ 57.7％ Second model type 0.65 61.9％ 70.1％ 65.7％ The third model type 0.5 76.3％ 86.2％ 80.9％

[0171] Table 2

[0172] It can be seen from the technical solutions provided by the above embodiments of this specification that this specification obtains third sharpening index data that can characterize the sharpness degree of each image to be identified by inputting the acquired at least one image to be identified into the image sharpening recognition model for sharpening recognition processing; the image sharpening recognition model is obtained by training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to at least one sharp sample image pair of each sample image in the first sample image set, and further training is performed based on the preset sharpening index data corresponding to some sample images in the first sample image set. It can be obtained by at least one sharp sample image pair of each sample image in the first sample image set. The corresponding sorting penalty loss is used to train the model, allowing the model to learn the differences between different sharpening degrees. Even when the model annotation samples are reduced, the generalization ability of the model can be effectively guaranteed. At the same time, the model is trained based on the preset sharpening index data corresponding to some sample images in the first sample image set (the second sample image set). On the basis of effectively reducing the model annotation cost, the sharpening recognition accuracy of the model can be improved, and thus the accuracy of the sharpening index data of each image to be identified obtained by the sharpening recognition processing based on the image sharpening recognition model can be effectively guaranteed, thereby improving the subsequent sharpening effect of multimedia content such as images and videos, reducing encoding costs, and improving system performance.

[0173] Figure 5 FIG. 1 is a block diagram of an image sharpening recognition model training device according to an exemplary embodiment. Figure 5 , the device comprises:

[0174] The training data acquisition module 510 is configured to acquire preset sharpening index data corresponding to a first sample image set and a second sample image set, where the second sample image set is a subset of the first sample image set;

[0175] The multi-level sharpening processing module 520 is configured to perform multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image, wherein the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images after each sample image is sharpened at different degrees of sharpening;

[0176] The sharpened sample image pair determination module 530 is configured to determine at least one sharpened sample image pair for each sample image from the sharpened image sequence corresponding to each sample image; each sharpened sample image pair for each sample image is two sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening levels;

[0177] The first sharpening recognition processing module 540 is configured to input at least one sharpened sample image pair corresponding to each sample image into the sharpening recognition model to be trained to perform sharpening recognition processing, thereby obtaining first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair;

[0178] The sorting penalty loss determination module 550 is configured to determine the sorting penalty loss corresponding to each sharpened sample image pair based on the first sharpening index data corresponding to each sharpened sample image pair;

[0179] The first model training module 560 is configured to perform a ranking penalty loss corresponding to each sharpened sample image pair to train the sharpened recognition model to be trained, thereby obtaining an initial sharpened recognition model;

[0180] The second model training module 570 is configured to train the initial sharpening recognition model based on the second sample image set and preset sharpening index data to obtain an image sharpening recognition model.

[0181] In an optional embodiment, the multi-level sharpening processing module 520 includes:

[0182] a sharpening parameter acquiring unit, configured to acquire a plurality of sharpening parameters corresponding to different sharpening degrees;

[0183] The first sharpening processing unit is configured to perform sharpening processing on each sample image based on multiple sharpening parameters to obtain a sharpened image sequence corresponding to each sample image.

[0184] In an optional embodiment, the multi-level sharpening processing module 520 includes:

[0185] A second sharpening processing unit is configured to input each sample image into a preset sharpening processing model for sharpening processing, thereby obtaining a sharpened image sequence corresponding to each sample image;

[0186] Among them, the preset sharpening processing model includes multiple sharpening processing models connected in sequence; the multiple sharpened sample images in the sharpened image sequence corresponding to each sample image are the output images of the multiple sharpening processing models; the input of the first sharpening processing model in the multiple sharpening processing models is each sample image; the input of the non-first sharpening processing model in the multiple sharpening processing models is the output of the corresponding previous sharpening processing model.

[0187] In an optional embodiment, the above device further includes:

[0188] The first image segmentation module is configured to segment the same area corresponding to the two sharp sample images in each sharp sample image pair into a pair of sample image blocks corresponding to each sharp sample image pair;

[0189] The first sharpening recognition processing module 540 is further configured to input the sample image block pair corresponding to each sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing, and obtain first sharpening index data corresponding to each sample image block in each sample image block pair;

[0190] The sorting penalty loss determination module 550 is further configured to determine the sorting penalty loss corresponding to each sample image block pair according to the first sharpening index data corresponding to each sample image block pair;

[0191] The first training module is further configured to perform a ranking penalty loss based on each sample image block pair to train the sharpening recognition model to be trained and obtain an initial sharpening recognition model.

[0192] In an optional embodiment, the second model training module 570 includes:

[0193] The first sharpening recognition processing unit is configured to input the sample images in the second sample image set into the initial sharpening recognition model to perform sharpening recognition processing, and obtain second sharpening index data corresponding to the sample images in the second sample image set;

[0194] a sharpening recognition loss determining unit configured to determine a sharpening recognition loss based on the second sharpening index data and the preset sharpening index data;

[0195] The model training unit is configured to perform training on the initial sharpening recognition model based on the sharpening recognition loss to obtain an image sharpening recognition model.

[0196] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0197] Figure 6 FIG. 1 is a block diagram of a sharpening recognition processing device according to an exemplary embodiment. Figure 6 , the device comprises:

[0198] The to-be-recognized image acquisition module 610 is configured to acquire at least one to-be-recognized image;

[0199] The second sharpening recognition processing module 620 is configured to perform sharpening recognition processing by inputting at least one image to be recognized into the image sharpening recognition model to obtain third sharpening index data corresponding to each image to be recognized, and the third sharpening index data represents the sharpness degree of each image to be recognized.

[0200] In an optional embodiment, the apparatus further comprises:

[0201] A reduction processing module is configured to perform reduction processing on each image to be recognized to obtain a reduced image;

[0202] A second image segmentation module is configured to perform image segmentation on the reduced image to obtain a plurality of image blocks corresponding to each image to be identified;

[0203] The second sharpening identification processing module 620 includes:

[0204] The second sharpening recognition processing unit is configured to input a plurality of image blocks corresponding to each image to be recognized into the image sharpening recognition model for sharpening recognition processing, and obtain fourth sharpening index data corresponding to each image block;

[0205] The index fusion processing unit is configured to perform fusion processing on multiple fourth sharpening index data corresponding to each image to be identified to obtain third sharpening index data corresponding to each image to be identified; the multiple fourth sharpening index data are the fourth sharpening index data corresponding to the corresponding multiple image blocks.

[0206] In an optional embodiment, when the at least one image to be identified is at least one frame of a video image in a video to be identified, the apparatus further includes:

[0207] The index fusion processing module is configured to perform fusion processing on the third sharpening index data corresponding to at least one frame of video image to obtain the sharpening index data corresponding to the video to be identified.

[0208] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0209] Optional, Figure 7 This is a block diagram of an electronic device for image sharpening recognition model training according to an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7 As shown. The electronic device includes a processor, a memory and a network interface connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an image sharpening recognition model training method is implemented.

[0210] Optional, Figure 8 This is a block diagram of an electronic device for sharpening recognition processing according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a sharpening recognition processing method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0211] Those skilled in the art will understand that Figure 7 or Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0212] In an exemplary embodiment, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the image sharpening recognition model training method or the sharpening recognition processing method in the embodiment of the present disclosure.

[0213] In an exemplary embodiment, a computer-readable storage medium is also provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image sharpening recognition model training method or the sharpening recognition processing method in the embodiment of the present disclosure.

[0214] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when executed on a computer, enables the computer to execute the image sharpening recognition model training method or the sharpening recognition processing method in the embodiments of the present disclosure.

[0215] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0216] Other embodiments of the present disclosure will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, with the true scope and spirit of the present disclosure being indicated by the following claims.

[0217] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes can be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training an image sharpening recognition model, characterized in that: The method comprises: Obtaining preset sharpening index data corresponding to a first sample image set and a second sample image set, where the second sample image set is a subset of the first sample image set; performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image, wherein the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images after each sample image is subjected to sharpening processing at different sharpening degrees; Determining at least one sharpened sample image pair for each sample image from the sharpened image sequence corresponding to each sample image; each sharpened sample image pair for each sample image is two sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening degrees; Inputting at least one sharpened sample image pair corresponding to each sample image into the sharpening recognition model to be trained for sharpening recognition processing, and obtaining first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair; Determining a sorting penalty loss corresponding to each sharpened sample image pair according to the first sharpening index data corresponding to each sharpened sample image pair; Based on the sorting penalty loss corresponding to each pair of sharpened sample images, the sharpening recognition model to be trained is trained to obtain an initial sharpening recognition model; Based on the second sample image set and the preset sharpening index data, the initial sharpening recognition model is trained to obtain an image sharpening recognition model.

2. The image sharpening recognition model training method according to claim 1, characterized in that: The performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image includes: Get multiple sharpening parameters corresponding to different sharpening degrees; Based on the multiple sharpening parameters, each sample image is sharpened to obtain a sharpened image sequence corresponding to each sample image.

3. The image sharpening recognition model training method according to claim 1, characterized in that: The performing multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image includes: Inputting each sample image into a preset sharpening processing model for sharpening processing to obtain a sharpened image sequence corresponding to each sample image; In which, the preset sharpening processing model includes multiple sharpening processing models connected in sequence; the multiple sharpened sample images in the sharpened image sequence corresponding to each sample image are the output images of the multiple sharpening processing models; the input of the first sharpening processing model among the multiple sharpening processing models is each sample image; the input of the non-first sharpening processing model among the multiple sharpening processing models is the output of the corresponding previous sharpening processing model.

4. The image sharpening recognition model training method according to claim 1, characterized in that: The method further comprises: Segmenting a pair of sample image blocks corresponding to each sharpened sample image pair from a common area corresponding to two sharpened sample images in each sharpened sample image pair; Inputting at least one sharpened sample image pair corresponding to each sample image into the sharpening recognition model to be trained for sharpening recognition processing to obtain first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair includes: inputting the sample image block pair corresponding to each sharpened sample image pair into the sharpening recognition model to be trained for sharpening recognition processing to obtain first sharpening index data corresponding to each sample image block in each sample image block pair; Determining the sorting penalty loss corresponding to each sharpened sample image pair according to the first sharpening index data corresponding to each sharpened sample image pair includes: determining the sorting penalty loss corresponding to each sample image block pair according to the first sharpening index data corresponding to each sample image block pair; The training of the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each pair of sharpened sample images to obtain the initial sharpening recognition model includes: training the sharpening recognition model to be trained based on the sorting penalty loss corresponding to each pair of sample image blocks to obtain the initial sharpening recognition model.

5. The image sharpening recognition model training method according to any one of claims 1 to 4, characterized in that: The training of the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain the image sharpening recognition model includes: Inputting the sample images in the second sample image set into the initial sharpening recognition model to perform sharpening recognition processing, thereby obtaining second sharpening index data corresponding to the sample images in the second sample image set; determining a sharpening recognition loss according to the second sharpening index data and the preset sharpening index data; Based on the sharpening recognition loss, the initial sharpening recognition model is trained to obtain the image sharpening recognition model.

6. A sharpening recognition processing method, characterized in that: The method comprises: Acquire at least one image to be recognized; The at least one image to be identified is input into the image sharpening recognition model obtained by the image sharpening recognition model training method according to any one of claims 1 to 5 for sharpening recognition processing, and third sharpening index data corresponding to each image to be identified is obtained, and the third sharpening index data represents the sharpness degree of each image to be identified.

7. The sharpening recognition processing method according to claim 6, characterized in that: The method further comprises: Performing a reduction process on each image to be identified to obtain a reduced image; Performing image segmentation on the reduced image to obtain a plurality of image blocks corresponding to each image to be identified; Inputting the at least one image to be identified into the image sharpening recognition model for sharpening recognition processing to obtain third sharpening index data corresponding to each image to be identified includes: Inputting the plurality of image blocks corresponding to each image to be recognized into the image sharpening recognition model for sharpening recognition processing to obtain fourth sharpening index data corresponding to each image block; The plurality of fourth sharpening index data corresponding to each image to be identified are fused to obtain the third sharpening index data corresponding to each image to be identified; the plurality of fourth sharpening index data are the fourth sharpening index data corresponding to the corresponding plurality of image blocks.

8. The sharpening recognition processing method according to claim 6, characterized in that: In a case where the at least one image to be identified is at least one frame of a video image in a video to be identified, the method further includes: The third sharpening index data corresponding to the at least one frame of video image is fused to obtain the sharpening index data corresponding to the video to be identified.

9. An image sharpening recognition model training device, characterized in that: The device comprises: a training data acquisition module configured to acquire preset sharpening index data corresponding to a first sample image set and a second sample image set, where the second sample image set is a subset of the first sample image set; a multi-level sharpening processing module configured to perform multi-level sharpening processing on each sample image in the first sample image set to obtain a sharpened image sequence corresponding to each sample image, wherein the sharpened image sequence corresponding to each sample image is a plurality of sharpened sample images after each sample image is sharpened at different degrees of sharpening; a sharpened sample image pair determination module configured to determine at least one sharpened sample image pair for each sample image from a sharpened image sequence corresponding to each sample image; each sharpened sample image pair for each sample image is two sharpened sample images obtained by performing sharpening processing on each sample image at different sharpening degrees; A first sharpening recognition processing module is configured to input at least one sharpened sample image pair corresponding to each sample image into a sharpening recognition model to be trained to perform sharpening recognition processing, thereby obtaining first sharpening index data corresponding to each sharpened sample image in each sharpened sample image pair; a sorting penalty loss determination module configured to determine a sorting penalty loss corresponding to each sharpened sample image pair based on the first sharpening index data corresponding to each sharpened sample image pair; A first model training module is configured to perform training on the to-be-trained sharpening recognition model based on the sorting penalty loss corresponding to each pair of sharpened sample images to obtain an initial sharpening recognition model; The second model training module is configured to train the initial sharpening recognition model based on the second sample image set and the preset sharpening index data to obtain an image sharpening recognition model.

10. A sharpening recognition processing device, characterized in that: include: An image acquisition module to be identified is configured to acquire at least one image to be identified; The second sharpening recognition processing module is configured to execute sharpening recognition processing by inputting the at least one image to be recognized into the image sharpening recognition model obtained by the image sharpening recognition model training method according to any one of claims 1 to 5, and obtain third sharpening index data corresponding to each image to be recognized, wherein the third sharpening index data represents the sharpness degree of each image to be recognized.

11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the image sharpening recognition model training method according to any one of claims 1 to 5 or the sharpening recognition processing method according to any one of claims 6 to 8.

12. A computer-readable storage medium, characterized in that When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image sharpening recognition model training method as described in any one of claims 1 to 5 or the sharpening recognition processing method as described in any one of claims 6 to 8.

Images