Image processing effect evaluation method and device, equipment, medium and product

Abstract

The embodiment of the invention discloses an image processing effect evaluation method and device, equipment, a medium and a product. The method comprises the steps of extracting an object region image including a target object in a to-be-evaluated image; wherein the object area image is a local image of the to-be-evaluated image; determining a plurality of object edge key points associated with the target object in the object area image; inputting a to-be-evaluated neighborhood image corresponding to the object edge key point into an effect evaluation model obtained by pre-training, and outputting an effect category corresponding to the to-be-evaluated neighborhood image; and determining a target evaluation effect of the to-be-evaluated image according to the effect category corresponding to each to-be-evaluated neighborhood image. According to the technical scheme provided by the embodiment of the invention, the image processing effect of the to-be-evaluated image can be evaluated, so that the image processing effect of the to-be-evaluated image can be determined.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of image processing technology, and in particular to an image processing effect evaluation method, device, equipment, medium and product. Background Art

[0002] With the development of multimedia technology, images and videos have become widely used ways of information interaction. When interacting with information, the background of the screen can be changed to meet specific needs. At present, the most commonly used way to change the background of the screen is green screen cutout. Green screen cutout can be understood as placing a large enough green screen in the actual background, cutting out the area of ​​interest by using specific pixel values, and applying it to a specific scene. Optionally, the specific scene can be a video production or live broadcast scene, etc.

[0003] When implementing the technical solution based on the above method, the inventor found the following problems:

[0004] Green screen cutout is mainly based on pixel value (color), which is easily affected by the external environment, resulting in poor fusion effect of the image obtained by content fusion based on the cutout, causing poor viewing experience for users. Based on this, the evaluation of images fused with cutout content has become an urgent problem to be solved. Summary of the invention

[0005] The present invention provides an image processing effect evaluation method, device, equipment, medium and product to achieve the effect of effectively evaluating the processing effect of an image to be evaluated.

[0006] In a first aspect, an embodiment of the present invention provides an image processing effect evaluation method, the method comprising:

[0007] Determine the object edge key points corresponding to the target object in the image to be evaluated;

[0008] Inputting the neighborhood image to be evaluated corresponding to the edge key point of the object into the pre-trained effect evaluation model, and outputting the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes a plurality of pixel points centered on the edge key point of the object;

[0009] According to the effect category corresponding to each neighborhood image to be evaluated, the target evaluation effect of the image to be evaluated is determined.

[0010] Furthermore, determining the object edge key points corresponding to the target object in the image to be evaluated includes:

[0011] Extracting an object region image including a target object from the image to be evaluated;

[0012] The edge contour of the target object in the object region image is extracted to obtain a plurality of object edge key points corresponding to the edge contour.

[0013] Furthermore, extracting an object region image including the target object from the image to be evaluated includes:

[0014] Inputting the image to be evaluated into an object recognition model, determining a target object in the image to be evaluated, and outputting an image of the area to be adjusted with the target object marked;

[0015] The image in the area to be adjusted is expanded according to the size information of the image in the area to be adjusted to obtain the object area image.

[0016] Furthermore, the size information includes width information and height information, and the step of performing an outward expansion process on the image of the area to be adjusted according to the size information of the image of the area to be adjusted to obtain the object area image includes:

[0017] Determining the expansion size information of the image in the area to be adjusted according to the width information, height information and a preset expansion ratio of the image in the area to be adjusted;

[0018] Determining target pixel information in the expanded area corresponding to the expanded size information according to the size information of the image in the area to be adjusted, the expanded size information and the image to be evaluated;

[0019] The object area image is obtained according to the target pixel information and the pixel information of the area image to be adjusted.

[0020] Furthermore, in the case where the image to be evaluated includes a plurality of target objects, the method further includes:

[0021] An object region image corresponding to each target object is determined respectively.

[0022] Furthermore, the step of extracting the edge contour of the target object in the object region image to obtain a plurality of object edge key points corresponding to the edge contour includes:

[0023] Inputting the object area image into a pre-trained key point annotation model, and outputting an annotated image with the key points of the target object annotated;

[0024] The annotated image includes a plurality of annotated object edge key points.

[0025] Furthermore, the method further comprises:

[0026] The object edge key point is respectively taken as the center and a preset number of pixels is used as the pixel search range to determine the neighborhood image to be evaluated corresponding to each object edge key point.

[0027] Further, the effect category includes a first category corresponding to satisfying a preset condition, a second category not satisfying the preset condition, or a third category without a recognition result, and determining the target evaluation effect of the image to be evaluated according to the effect category corresponding to each neighborhood image to be evaluated includes:

[0028] According to the number of the first category and the second category in the effect category, a target evaluation effect of the image to be evaluated is determined.

[0029] Furthermore, the method further comprises:

[0030] Dividing a plurality of object edge key points in the standard image into at least two groups according to a preset division rule;

[0031] Accordingly, determining the target evaluation effect of the image to be evaluated according to the number of the first category and the second category in the effect category includes:

[0032] Determine the evaluation effect to be processed corresponding to each group of object edge key points according to the number of the first category and the second category corresponding to each group of object edge key points;

[0033] When the to-be-processed evaluation results of at least two groups of object edge key points satisfy the evaluation conditions, the target evaluation result is determined to be the first evaluation result.

[0034] Furthermore, the method further comprises:

[0035] In the case that there is a group of unprocessed evaluation results of the object edge key points that do not satisfy the evaluation condition, the target evaluation result is determined to be the second evaluation result.

[0036] Furthermore, when the target evaluation effect is the second evaluation effect, the method further includes:

[0037] Feedback prompt information that the image processing effect is a second evaluation effect is sent to a target platform corresponding to the image to be evaluated.

[0038] Furthermore, the image to be evaluated is at least an image including cutout content, and the cutout content corresponds to the target object.

[0039] In a second aspect, an embodiment of the present invention further provides an image processing effect evaluation device, the device comprising:

[0040] A key point determination module is used to determine the object edge key points corresponding to the target object in the image to be evaluated;

[0041] An effect category determination module is used to input the neighborhood image to be evaluated corresponding to the edge key point of the object into a pre-trained effect evaluation model, and output the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes a plurality of pixel points centered on the edge key point of the object;

[0042] The evaluation effect determination module is used to determine the target evaluation effect of the image to be evaluated according to the effect category corresponding to each neighborhood image to be evaluated.

[0043] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0044] one or more processors;

[0045] a storage device for storing one or more programs,

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing effect evaluation method as described in any one of the embodiments of the present invention.

[0047] In a fourth aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, wherein the computer executable instructions, when executed by a computer processor, are used to execute the image processing effect evaluation method as described in any one of the embodiments of the present invention.

[0048] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the computer program is used to execute the image processing effect evaluation method as described in any one of the embodiments of the present invention.

[0049] The technical solution of the embodiment of the present invention can first extract the object area image including the target object in the image to be evaluated, further determine the object edge key points corresponding to the target object in the object area image, and further input the to-be-processed neighborhood image corresponding to each object edge key point into the effect evaluation model, so as to obtain the effect category corresponding to each to-be-processed neighborhood image, and then determine the target processing effect of the image to be evaluated according to all effect categories, thereby solving the problem in the prior art that the image processing effect is not evaluated and effective feedback cannot be provided thereto, and realizes the use of corresponding technical means to determine the processing effect of the image to be evaluated, so that feedback can be provided thereto, so as to facilitate the re-processing of the image to be evaluated based on the feedback, thereby improving the image processing effect and the user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following briefly introduces the drawings required to describe the embodiments. Obviously, the drawings introduced are only drawings of a part of the embodiments to be described in the present invention, rather than all the drawings. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A schematic diagram of a flow chart of an image processing effect evaluation method provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a flow chart of an image processing effect evaluation method provided by an embodiment of the present invention;

[0053] Figure 3 A schematic diagram of obtaining an object area image provided by an embodiment of the present invention;

[0054] Figure 4 A schematic diagram including multiple object edge key points provided by an embodiment of the present invention;

[0055] Figure 5 A schematic diagram of a neighborhood image to be processed corresponding to each object edge key point provided by an embodiment of the present invention;

[0056] Figure 6 A schematic diagram of the structure of an image processing effect evaluation device provided by an embodiment of the present invention;

[0057] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0059] Before introducing the technical solution provided by the embodiment of the present invention, an application scenario may be exemplified first.

[0060] In the live broadcast scene, users can explain certain physical or virtual objects. In order to match the objects with the background, background boards that match different brands of objects or festivals can be made, so that users can broadcast live under the corresponding background boards. However, the above method requires the production of background boards corresponding to different objects or festivals, which has the problem of high cost. Based on this, a sufficiently large green screen can be placed in the actual background, and the green screen area can be cut out using specific pixel values. After the cutout, the background image is replaced to obtain each video frame in the live broadcast scene, and each video frame can be used as an image to be evaluated.

[0061] It should be noted that there is currently no unified standard for judging the image effect after cutting out the image; further, after the cut-out image is merged with the live broadcast screen, there are still problems such as stroking and green shadows at the junction of the user and the background in the live broadcast screen, which in turn causes the user to have a poor viewing experience. Since there is no solution to evaluate the cut-out effect, it is impossible to feedback the above problems to the target platform to solve the problems of poor cut-out effect and poor user viewing experience.

[0062] Of course, it can also be applied in image processing scenarios. For example, the image to be evaluated can be an image obtained by fusion after cutting out. The image processing effect corresponding to the image to be evaluated can be determined based on the technical solution provided in the embodiment of the present invention. At this time, the image processing effect can be understood as the fusion effect after cutting out.

[0063] It should also be noted that each image can be used as an image to be evaluated. Each video frame in the live broadcast scene can be used as an image to be evaluated, or each image can be used as an image to be evaluated. For this embodiment, whether it is an image to be evaluated with a cutout effect or an image without a cutout effect, the scheme provided by this embodiment can be used to determine its image processing effect. For images without a cutout effect, the final target evaluation effect is the conclusion of the first evaluation effect. For images with a cutout effect, the final target evaluation effect can be the first evaluation effect with a good image processing effect or the second evaluation effect with a poor image processing effect.

[0064] Figure 1 A flow chart of an image processing effect evaluation method provided in an embodiment of the present invention is provided. This embodiment is applicable to situations where the image processing effect of an image to be evaluated is determined. The method can be executed by an image processing effect evaluation device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, and the electronic device can be a mobile terminal or a PC, etc.

[0065] like Figure 1 As shown, the method provided by the embodiment of the present invention includes:

[0066] S110, extracting an object region image including a target object from the image to be evaluated; wherein the object region image is a local image of the image to be evaluated.

[0067] Among them, each image can be used as an image to be evaluated, that is, the image whose image processing effect needs to be determined is used as the image to be evaluated. The object area image can be understood as the image obtained after extracting the target object in the image to be evaluated. For the image to be evaluated, the target object can fill the entire screen or only be a part of the entire screen. Correspondingly, the object area image is a partial image of the image to be evaluated. The target object is a pre-set cutout object.

[0068] Specifically, when it is necessary to determine the image processing effect of a certain image, this image can be used as the image to be evaluated. According to the target object in the image to be evaluated, the object area image including the target object is determined. Exemplarily, it can be applied in a live broadcast scene. If it is necessary to analyze and process each video frame in the live broadcast stream, each video frame can be used as the image to be evaluated. For the image to be evaluated at this time, the target object can be an object, a user, etc. Optionally, the target object is a user. Furthermore, a rectangular frame including the target object in the image to be evaluated can be determined, and the image composed of all pixels in the rectangular frame can be used as the object area image.

[0069] In this embodiment, extracting an object area image including a target object in an image to be evaluated includes: inputting the image to be evaluated into an object recognition model, determining the target object in the image to be evaluated, and outputting an image of the area to be adjusted with the target object marked; and expanding the image of the area to be adjusted according to size information of the image of the area to be adjusted to obtain an object area image.

[0070] The object recognition model is pre-trained and used to mark the target object in the image to be evaluated, thereby obtaining a model of the object area image. The object recognition model can be a neural network model, which can use a target detection algorithm to identify the target object and mark it. Before marking the target object based on the object recognition model, a training sample can be obtained to train the object recognition model.

[0071] Optionally, multiple sample images including the target object are obtained. In this case, image frames can be extracted from the live playback, and the target object can be marked with a rectangular frame in the image frame. The annotated image obtained after the marking is completed and the unlabeled image as a whole are used as sample images. The unlabeled image is input into the object recognition model to be trained, and an output image in which a rectangular frame of the target object has been marked in the unlabeled image is output. According to the output image and the annotated image corresponding to the sample image, a loss value can be determined. According to the loss value, the model parameters in the object recognition model to be trained are corrected, and the model obtained when the loss function in the object recognition model to be trained converges is used as the object recognition model that can be used subsequently. The specific model training method can refer to the existing model training method, which will not be repeated in this embodiment.

[0072] If the image to be evaluated includes the target object, then after the image to be evaluated is input into the object recognition model, an image of the area to be adjusted with the rectangular frame of the target object can be output. Of course, if the image to be evaluated does not include the target object, the output result of the object recognition model is still the image to be evaluated.

[0073] Based on the above, it can be known that the image in the area to be adjusted is the image obtained after the object recognition model marks the target object in the image to be evaluated. In order to facilitate the subsequent analysis and processing of the object area image to determine its image processing effect, the image in the area to be adjusted can be further processed to obtain the object area image.

[0074] The size information of the image of the region to be adjusted includes the width information and the height information of the image of the region to be adjusted. The expansion ratio of the image of the region to be adjusted can be determined according to the size information of the image of the region to be adjusted, and then the image of the region to be adjusted is expanded according to the expansion ratio and the image to be evaluated to obtain the object region image. That is, the image of the region to be adjusted is a local image in the object region image, and the size of the local image is mainly adapted to the rectangular frame corresponding to the target object.

[0075] Next, it is explained in detail how to determine the object area image according to the size information of the image of the area to be adjusted: optionally, the size information includes width information and height information, and the image of the area to be adjusted is expanded according to the size information of the image of the area to be adjusted to obtain the object area image, including: determining the expanded size information of the image of the area to be adjusted according to the width information, height information and a preset expansion ratio of the image of the area to be adjusted; determining the target pixel information in the expanded area corresponding to the expanded size information according to the size information of the image of the area to be adjusted, the expanded size information and the image to be evaluated; and obtaining the object area image according to the target pixel information and the pixel information of the image of the area to be adjusted.

[0076] The width information corresponds to the first number of pixels of the image of the area to be adjusted in the first direction, and the height information corresponds to the second number of pixels of the image of the area to be adjusted in the second direction. The first direction and the second direction are perpendicular to each other. The preset expansion ratio is a preset ratio that needs to be expanded. Optionally, the preset expansion ratio can be 20%, that is, the first number of pixels is increased by 20% in the first direction, and the second number of pixels is increased by 20% in the second direction. That is, based on the preset expansion ratio and the size information of the image of the area to be adjusted, the expansion size information can be determined. The expansion size information corresponds to the number of pixels expanded in the first direction and the number of pixels expanded in the second direction.

[0077] After the number of expanded pixels is determined, the image of the area to be adjusted can be expanded based on the image of the area to be adjusted and combined with the image to be evaluated to obtain the image of the object area.

[0078] In practical applications, there may be a scene where the image to be evaluated includes multiple target objects. In this case, each target object can be treated as an independent individual and the object region image corresponding to it can be determined. Optionally, the object region image corresponding to each target object can be determined separately.

[0079] It can be understood that, when the image to be evaluated includes multiple target objects, the image to be evaluated can be processed once or multiple times to obtain an object region image adapted to each target object, and then the image processing effect can be detected based on the object region image.

[0080] S120: Determine a plurality of object edge key points associated with the target object in the object region image.

[0081] Usually, when cutting out an image, the information of interest is mostly cut out. For example, in a live broadcast scene, the information of interest is mostly the target object and the items explained by the target object, so the cut-out image is mostly the image corresponding to the target object. When evaluating the image processing effect (cut-out effect), it is necessary to pay attention to the cut-out edges corresponding to the target object. Therefore, the key points associated with the target object in the object area image can be determined. That is, the edge contour of the target object is annotated with key points to obtain multiple object edge key points. In other words, the object edge key points are the key points corresponding to the annotated edge contour of the target object.

[0082] In this embodiment, determining multiple object edge key points associated with a target object in an object area image includes: inputting the object area image into a pre-trained key point annotation model, and outputting an annotated image in which the key points of the target object are annotated; wherein the annotated image includes multiple annotated object edge key points.

[0083] It should be noted that different methods can be used to determine the object edge key points corresponding to the target object. For example, a key point recognition algorithm can be used to identify the key points corresponding to the target object and mark them. Alternatively, a key point marking model can be used to mark the key points of the target object.

[0084] The key point annotation model is a pre-trained model used to annotate key points of a target object. The input of the key point annotation model is an object region image, and the output is an image in which the object edge key points of the target object have been annotated in the object region image. When the image to be evaluated is a video frame determined based on a live broadcast scene, the target object may be a user, and correspondingly, the object edge key points may be points obtained after annotating the edge contour of the target object.

[0085] It should be noted that the annotated positions and quantities of the key points on the object edges correspond to the training data of the key point annotation model.

[0086] S130, inputting the neighborhood image to be evaluated corresponding to the edge key point of the object into the pre-trained effect evaluation model, and outputting the effect category corresponding to the neighborhood image to be evaluated.

[0087] Among them, the neighborhood image to be evaluated includes multiple pixels centered on the object edge key point. Optionally, the neighborhood image to be evaluated is a rectangle, and the number of pixels contained therein is nine or 16, etc., and the specific number of pixels can be set according to actual needs. The central pixel of the neighborhood image to be evaluated is the object edge key point. The effect evaluation model is pre-trained and can be a single-input single-output model. The input of the effect evaluation model can be each neighborhood image to be evaluated, and the output can be the effect category corresponding to the neighborhood image to be evaluated. In the present embodiment, the effect category can include a first category corresponding to the image processing effect naturally, a second category corresponding to the poor image processing effect, and a third category corresponding to the unrecognizable image processing effect.

[0088] Specifically, after obtaining the neighborhood image to be evaluated corresponding to each object edge key point, the neighborhood image to be evaluated can be input into the effect evaluation model, and the effect evaluation model can output the effect category corresponding to the neighborhood image to be evaluated. Optionally, the first category, the second category, or the third category can be output.

[0089] In this embodiment, determining the neighborhood image to be evaluated corresponding to each object edge key point can be done by: taking the object edge key point as the center and a preset number of pixels as the pixel search range, determining the neighborhood image to be evaluated corresponding to each object edge key point; wherein the preset neighborhood includes at least one pixel.

[0090] It can be understood that the rectangular area corresponding to each object edge key point can be determined with the object edge key point as the center and the preset number of pixels as the outer range, and the image corresponding to the rectangular area is used as the neighborhood image to be evaluated.

[0091] S140: Determine a target evaluation effect of the image to be evaluated according to the effect category corresponding to each neighborhood image to be evaluated.

[0092] The target evaluation effect is the effect finally determined after analyzing and processing the image to be evaluated. Optionally, the target evaluation effect includes a first evaluation effect and a second evaluation effect. The first evaluation effect corresponds to the evaluation effect of natural image processing, that is, natural cutout; the second evaluation effect corresponds to the evaluation effect of unnatural image processing, that is, poor edge processing after cutout, which is easy to be exposed.

[0093] Specifically, after obtaining the effect category corresponding to each neighborhood image to be evaluated, all effect categories can be analyzed and processed to obtain the target evaluation effect of the image to be evaluated. Alternatively, all effect categories can be grouped and processed to obtain the evaluation effect to be processed corresponding to each group, and then the target evaluation effect can be determined based on the evaluation effect corresponding to each group.

[0094] The technical solution of the embodiment of the present invention can first extract the object area image including the target object in the image to be evaluated, further determine the object edge key points corresponding to the target object in the object area image, and further input the to-be-processed neighborhood image corresponding to each object edge key point into the effect evaluation model, so as to obtain the effect category corresponding to each to-be-processed neighborhood image, and then determine the target processing effect of the image to be evaluated according to all effect categories, thereby solving the problem in the prior art that the image processing effect is not evaluated and effective feedback cannot be provided thereto, and realizes the use of corresponding technical means to determine the processing effect of the image to be evaluated, so that feedback can be provided thereto, so as to facilitate the re-processing of the image to be evaluated based on the feedback, thereby improving the image processing effect and the user viewing experience.

[0095] Figure 2 A flow chart of an image processing effect evaluation method provided in an embodiment of the present invention, based on the aforementioned embodiment, the object edge key points can be divided into at least two groups according to preset rules, and the evaluation effects to be processed corresponding to each group of object edge key points can be determined respectively, thereby determining the target evaluation effect of the image to be evaluated. The specific implementation method can refer to the detailed description of this embodiment, wherein the technical terms that are the same as or corresponding to the above-mentioned embodiment will not be repeated in this embodiment.

[0096] like Figure 2 As shown, the method provided by the embodiment of the present invention includes:

[0097] S210 , extracting an object region image including a target object from the image to be evaluated; wherein the object region image is a local image of the image to be evaluated.

[0098] For example, during the live broadcast process, it is possible to determine in real time or periodically whether the processing effect of each picture frame is natural. In this case, each picture frame can be used as an image to be evaluated. Figure 3 After obtaining the image to be evaluated corresponding to the live screen, the image to be evaluated can be input into the object recognition model so that the object recognition model can mark the target object (user) in the image to be evaluated to obtain the image of the area to be adjusted. The marking schematic diagram of the image of the area to be adjusted can be seen in Figure 3 The schematic diagram shown on the left. Next, the corresponding external expansion size information based on the image of the area to be adjusted can be determined according to the size information of the image of the area to be adjusted and the preset external expansion ratio of 20%. Based on the external expansion size information, the image to be evaluated and the image of the area to be adjusted, the object area image is determined. The object area image can be cut out from the image to be evaluated to obtain the following Figure 3 The purpose of cutting out is to determine whether the edge processing effect corresponding to the target object in the image to be evaluated meets the corresponding conditions.

[0099] S220: Determine a plurality of object edge key points associated with the target object in the object region image.

[0100] Based on the above examples, it can be known that the target object mainly refers to the user. The key points of the human body contour can be located for the target object in the object area image to obtain an image in which the key points of the object edge are marked in the object area image.

[0101] For example, HRNet can be used as a key point positioning algorithm. Figure 3 The corresponding object area image on the right is annotated with key points, and the following is obtained: Figure 4 The effect diagram is shown. It should be noted that HRNet can be used as a key point positioning algorithm to determine the key point labeling model. The training data corresponding to the key point labeling model is an image in which key points in the object area image are labeled. Specifically, the upper body of the target object can be used as the labeling range, and multiple object edge key points can be labeled on the left and right sides of the user. The multiple object edge key points are evenly distributed on the outermost side of the upper body, starting at the neck and ending at the lowest position of the outermost contour line of the upper body in the image, such as Figure 4 See Figure 4 , multiple object edge key points are divided into two groups, one group is multiple object edge key points on the left outer edge contour, and the other group is multiple object edge key points on the right outer edge contour.

[0102] S230, inputting the neighborhood image to be evaluated corresponding to the edge key point of the object into the pre-trained effect evaluation model, and outputting the effect category corresponding to the neighborhood image to be evaluated.

[0103] For each object edge keypoint, the neighborhood image to be evaluated can be cut out with the object edge keypoint as the center. For example, see Figure 5 , a 16×16 square neighborhood is cut out with the object edge key point as the center as the neighborhood image to be evaluated. Based on the above example, each group of object edge key points includes five object edge key points. After cutting out the neighborhood image to be evaluated with each object edge key point as the center, 10 neighborhood images to be evaluated can be obtained.

[0104] The neighborhood image to be evaluated corresponding to each object edge key point is input into the pre-trained effect evaluation model, and the effect category corresponding to the neighborhood image to be evaluated can be output. The effect category may include a first category corresponding to a natural cutout effect, a second category corresponding to a poor cutout effect, or a third category corresponding to an invalid category.

[0105] S240: When the object edge key points are divided into at least two groups, determine the evaluation results to be processed corresponding to each group of object edge key points according to the number of first categories and second categories corresponding to each group of object edge key points.

[0106] Among them, the object edge key points can be divided into two groups according to their position information on the target object. The number of the first category and the second category corresponding to the multiple object edge key points in each group can be counted. The evaluation effect to be processed can be the effect determined after processing each group of object edge key points. The evaluation effect to be processed can be the overall processing effect corresponding to the multiple object edge key points. All the evaluation effects to be processed can be integrated to determine the target evaluation effect corresponding to the image to be evaluated.

[0107] Exemplarily, the 10 object edge key points are divided into two groups, left and right, with five object edge key points in each group. For the five object edge key points in each group, the number of effect categories of the first category and the second category can be counted. The ratio between the first number of the first category and the total number is calculated to obtain the evaluation effect to be processed corresponding to the group of object edge key points. The total number can be the sum of the number of the first category and the second category in the group.

[0108] For example, the calculation formula for the pending evaluation effect is:

[0109]

[0110] Among them, num goodis the number of natural cutouts (first category) after the discrimination processing of the neighboring images corresponding to the edge key points of the five objects, num bad is the number of images to be evaluated that are judged to have poor cutout effects (the second category) after the five neighboring images corresponding to the edge key points of the five objects are discriminated and processed, num bad +num good +num invalid =5,num invalid After the discrimination processing of the neighborhood images to be processed by the gamble of the edge key points of the five objects, the discrimination effect is the number of invalid categories (the third category).

[0111] S250: When the to-be-processed evaluation results of at least two groups of object edge key points satisfy the evaluation conditions, determine the target evaluation result as the first evaluation result.

[0112] The first evaluation effect is an evaluation effect with a good image processing effect. The evaluation condition is a judgment condition for evaluating whether the evaluation effect to be processed is the first evaluation effect or the second evaluation effect. The evaluation effect to be processed can be characterized by an evaluation value, and optionally, the evaluation value can be a numerical value corresponding to the image processing effect.

[0113] Specifically, if the values ​​of the pending evaluation effects corresponding to at least two groups of object edge key points are greater than the preset numerical threshold, it means that the pending evaluation effects are all the first evaluation effects. Accordingly, the target evaluation effect is the first evaluation effect. In practical applications, when there is a group of pending evaluation effects of object edge key points that do not meet the evaluation conditions, the target evaluation effect is determined to be the second evaluation effect.

[0114] It can be understood that: if there is a group of object edge key points whose evaluation value of the to-be-processed evaluation effect is less than the preset numerical threshold, it means that the image processing effect is not good, and at this time, the target evaluation effect is determined to be the second evaluation effect. In other words, the second evaluation effect is the evaluation result of the poor processing effect.

[0115] On the basis of the above technical solution, when the target evaluation effect is the second evaluation effect, the method further includes: feeding back prompt information that the image processing effect is the second evaluation effect to the target platform corresponding to the image to be evaluated.

[0116] The target platform may be a platform that provides or displays the image to be evaluated. The prompt information may be a problem that the processing effect of the image to be evaluated is not good and may affect the user's viewing experience. The benefit of feedback prompt information is that it can facilitate the improvement of the subsequent image processing precision.

[0117] It can be understood that: if after analyzing and processing the image to be evaluated, it is determined that the processing effect of the image to be evaluated is not good, at this time, a prompt message that the image processing effect is the second evaluation effect can be fed back to the target platform displaying the image to be evaluated, so that the background user corresponding to the target platform can improve the processing effect of subsequent images based on the prompt information.

[0118] Exemplarily, the image to be evaluated is a live broadcast picture. If the target evaluation effect is the second evaluation effect, it indicates that the processing effect of the image to be evaluated, such as the cutout effect, is not well processed. At this time, a prompt message indicating that the image processing effect is not good can be fed back to the live broadcast platform (target platform), so that the staff corresponding to the live broadcast platform can improve the precision of subsequent live broadcast picture processing, and thereby obtain the image to be evaluated whose target evaluation effect is the first evaluation effect.

[0119] Based on the above example, if the ratio of the evaluation effects to be processed on both sides is greater than 0.5, the target evaluation effect of the image to be evaluated is considered to be the first evaluation effect. If there are any group of rounds in the evaluation effects to be processed where the contour cutout effect is poor, then the cutout effect of the entire image is judged to be poor, that is, the evaluation effect to be processed is the second evaluation effect. As long as there is a conclusion that the evaluation effect to be processed is the second evaluation effect, it means that the target evaluation effect is the second evaluation effect.

[0120] The technical solution of the embodiment of the present invention can first extract the object area image including the target object in the image to be evaluated, further determine the object edge key points corresponding to the target object in the object area image, and further input the to-be-processed neighborhood image corresponding to each object edge key point into the effect evaluation model, so as to obtain the effect category corresponding to each to-be-processed neighborhood image, and then determine the target processing effect of the image to be evaluated according to all effect categories, thereby solving the problem in the prior art that the image processing effect is not evaluated and effective feedback cannot be provided thereto, and realizes the use of corresponding technical means to determine the processing effect of the image to be evaluated, so that feedback can be provided thereto, so as to facilitate the re-processing of the image to be evaluated based on the feedback, thereby improving the image processing effect and the user viewing experience.

[0121] Figure 6 A schematic diagram of the structure of an image processing effect evaluation device provided in an embodiment of the present invention, the device comprises: a regional image extraction module 310, a key point determination module 320, an effect category determination module 330 and an evaluation effect determination module 340.

[0122] Among them, the regional image extraction module 310 is used to extract the object region image including the target object in the image to be evaluated; wherein the object region image is a local image of the image to be evaluated; the key point determination module 320 is used to determine multiple object edge key points associated with the target object in the object region image; the effect category determination module 330 is used to input the neighborhood image to be evaluated corresponding to the object edge key points into the pre-trained effect evaluation model, and output the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes multiple pixel points centered on the object edge key points; the evaluation effect determination module 340 is used to determine the target evaluation effect of the image to be evaluated based on the effect category corresponding to each neighborhood image to be evaluated.

[0123] The technical solution of the embodiment of the present invention can first extract the object area image including the target object in the image to be evaluated, further determine the object edge key points corresponding to the target object in the object area image, and further input the to-be-processed neighborhood image corresponding to each object edge key point into the effect evaluation model, so as to obtain the effect category corresponding to each to-be-processed neighborhood image, and then determine the target processing effect of the image to be evaluated according to all effect categories, thereby solving the problem in the prior art that the image processing effect is not evaluated and effective feedback cannot be provided thereto, and realizes the use of corresponding technical means to determine the processing effect of the image to be evaluated, so that feedback can be provided thereto, so as to facilitate the re-processing of the image to be evaluated based on the feedback, thereby improving the image processing effect and the user viewing experience.

[0124] Based on the above technical solution, the regional image extraction module includes:

[0125] The regional image output unit is used to input the image to be evaluated into the object recognition model, determine the target object in the image to be evaluated, and output the regional image to be adjusted with the target object marked; the regional image determination unit includes expanding the regional image to be adjusted according to the size information of the regional image to be adjusted to obtain the object regional image.

[0126] Based on the above technical solution, the size information includes width information and height information, and the area image determination unit includes:

[0127] The expansion size determination subunit is used to determine the expansion size information of the image in the area to be adjusted according to the width information, height information and a preset expansion ratio of the image in the area to be adjusted;

[0128] A pixel information determination subunit, used to determine target pixel information in the expanded area corresponding to the expanded size information according to the size information of the image in the area to be adjusted, the expanded size information and the image to be evaluated;

[0129] The area image determination subunit is used to obtain the object area image according to the target pixel information and the pixel information of the area image to be adjusted.

[0130] On the basis of the above technical solutions, the object area image determination module is further used to respectively determine the object area image corresponding to each target object.

[0131] On the basis of the above technical solutions, the edge key point determination module includes:

[0132] The annotated image determining unit is used to input the object area image into a pre-trained key point annotation model, and output an annotated image with the key points of the target object annotated;

[0133] The annotated image includes a plurality of annotated object edge key points.

[0134] On the basis of the above technical solutions, the device further includes: a neighborhood image determination module, which is used to determine the neighborhood image to be evaluated corresponding to each object edge key point, with the object edge key point as the center and a preset number of pixels as the pixel search range. On the basis of the above technical solutions, the effect category includes a first category corresponding to meeting the preset conditions, a second category not meeting the preset conditions, or a third category with no recognition results, and the target evaluation effect determination module is further used to determine the target evaluation effect of the image to be evaluated according to the number of the first category and the second category in the effect category.

[0135] On the basis of the above technical solutions, the device further comprises: a key point division module, which is used to divide a plurality of edge key points in the standard image into at least two groups according to a preset division rule;

[0136] Accordingly, the target evaluation effect determination module includes:

[0137] A pending evaluation effect determination unit, used to determine the pending evaluation effect corresponding to each group of object edge key points according to the number of first categories and second categories corresponding to each group of object edge key points;

[0138] The target evaluation effect determination unit is used to determine the target evaluation effect as the first evaluation effect when the to-be-processed evaluation effects of at least two groups of object edge key points meet preset conditions.

[0139] On the basis of the above-mentioned technical solutions, the target evaluation effect determination unit is also used to determine the target evaluation effect as the second evaluation effect when there is a group of object edge key points whose to-be-processed evaluation effects do not meet the preset conditions.

[0140] On the basis of the above technical solutions, the device further includes: a feedback module, which is used to feed back prompt information that the image processing effect is a second evaluation effect to a target platform corresponding to the image to be evaluated.

[0141] On the basis of the above technical solutions, the image to be evaluated is at least an image including cutout content, and the cutout content corresponds to the target object.

[0142] The image processing effect evaluation device provided in the embodiment of the present invention can execute the image processing effect evaluation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present invention.

[0144] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 7 A block diagram of an exemplary electronic device 40 suitable for implementing exemplary embodiments of the present invention is shown. Figure 7 The electronic device 40 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0145] like Figure 7 As shown, the electronic device 40 is in the form of a general computing device. The components of the electronic device 40 may include but are not limited to: one or more processors or processing units 401, a system memory 402, and a bus 403 connecting different system components (including the system memory 402 and the processing unit 401).

[0146] Bus 403 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus structures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0147] The electronic device 40 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 40, including volatile and non-volatile media, removable and non-removable media.

[0148] System memory 402 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 404 and / or cache memory 405. Electronic device 40 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 406 may be used to read and write non-removable, non-volatile magnetic media ( Figure 7 not shown, usually called a "hard drive"). Although Figure 7 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 403 via one or more data medium interfaces. Memory 402 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of various embodiments of the present invention.

[0149] A program / utility 408 having a set (at least one) of program modules 407 may be stored, for example, in memory 402, such program modules 407 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 407 generally perform the functions and / or methods of the embodiments described herein.

[0150] The electronic device 40 may also communicate with one or more external electronic devices 409 (e.g., keyboards, pointing devices, displays 410, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 40, and / or communicate with any device that enables the electronic device 40 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 411. Furthermore, the electronic device 40 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 412. As shown, the network adapter 412 communicates with other modules of the electronic device 40 via the bus 403. It should be understood that although Figure 7 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 40, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0151] The processing unit 401 executes various functional applications and data processing by running the programs stored in the system memory 402, such as implementing the image processing effect evaluation method provided in the embodiment of the present invention.

[0152] An embodiment of the present invention further provides a storage medium containing computer executable instructions, and the computer executable instructions are used to execute the image processing effect evaluation method when executed by a computer processor.

[0153] The method includes:

[0154] Determine the object edge key points corresponding to the target object in the image to be evaluated;

[0155] Inputting the neighborhood image to be evaluated corresponding to the edge key point of the object into the pre-trained effect evaluation model, and outputting the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes a plurality of pixel points centered on the edge key point of the object;

[0156] According to the effect category corresponding to each neighborhood image to be evaluated, the target evaluation effect of the image to be evaluated is determined.

[0157] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0158] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0159] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0160] Computer program code for performing the operation of embodiments of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0161] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for evaluating image processing effects, characterized in that: include: Determine the object edge key points corresponding to the target object in the image to be evaluated; Inputting the neighborhood image to be evaluated corresponding to the edge key point of the object into the pre-trained effect evaluation model, and outputting the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes a plurality of pixel points centered on the edge key point of the object; According to the effect category corresponding to each neighborhood image to be evaluated, the target evaluation effect of the image to be evaluated is determined.

2. The method according to claim 1, characterized in that The step of determining the object edge key points corresponding to the target object in the image to be evaluated includes: Extracting an object region image including a target object from the image to be evaluated; The edge contour of the target object in the object region image is extracted to obtain a plurality of object edge key points corresponding to the edge contour.

3. The method according to claim 2, characterized in that The step of extracting an object region image including a target object from the image to be evaluated comprises: Inputting the image to be evaluated into an object recognition model, determining a target object in the image to be evaluated, and outputting an image of the area to be adjusted with the target object marked; The image in the area to be adjusted is expanded according to the size information of the image in the area to be adjusted to obtain the object area image.

4. The method according to claim 3, characterized in that: The size information includes width information and height information, and the step of performing an outward expansion process on the image of the area to be adjusted according to the size information of the image of the area to be adjusted to obtain the object area image includes: Determining the expansion size information of the image in the area to be adjusted according to the width information, height information and a preset expansion ratio of the image in the area to be adjusted; Determining target pixel information in the expanded area corresponding to the expanded size information according to the size information of the image in the area to be adjusted, the expanded size information and the image to be evaluated; The object area image is obtained according to the target pixel information and the pixel information of the area image to be adjusted.

5. The method according to claim 2, characterized in that: In the case where the image to be evaluated includes a plurality of target objects, the method further includes: An object region image corresponding to each target object is determined respectively.

6. The method according to claim 2, characterized in that The step of extracting the edge contour of the target object in the object region image to obtain a plurality of object edge key points corresponding to the edge contour comprises: Inputting the object area image into a pre-trained key point annotation model, and outputting an annotated image with the key points of the target object annotated; The annotated image includes a plurality of annotated object edge key points.

7. The method according to claim 1, characterized in that The method further comprises: The object edge key point is respectively taken as the center and a preset number of pixels is used as the pixel search range to determine the neighborhood image to be evaluated corresponding to each object edge key point.

8. The method according to claim 1, characterized in that: The effect categories include a first category corresponding to satisfying a preset condition, a second category not satisfying the preset condition, or a third category without a recognition result, and determining a target evaluation effect of the image to be evaluated according to the effect category corresponding to each neighborhood image to be evaluated includes: According to the number of the first category and the second category in the effect category, a target evaluation effect of the image to be evaluated is determined.

9. The method according to claim 8, characterized in that The method further comprises: Dividing multiple object edge key points into at least two groups according to a preset division rule; Accordingly, determining the target evaluation effect of the image to be evaluated according to the number of the first category and the second category in the effect category includes: Determine the evaluation effect to be processed corresponding to each group of object edge key points according to the number of the first category and the second category corresponding to each group of object edge key points; When the to-be-processed evaluation results of at least two groups of object edge key points satisfy the evaluation conditions, the target evaluation result is determined to be the first evaluation result.

10. The method according to claim 9, characterized in that The method further comprises: In the case that there is a group of unprocessed evaluation results of the object edge key points that do not satisfy the evaluation condition, the target evaluation result is determined to be the second evaluation result.

11. The method according to claim 1 or 10, characterized in that: When the target evaluation effect is the second evaluation effect, the method further includes: Feedback prompt information that the image processing effect is a second evaluation effect is sent to a target platform corresponding to the image to be evaluated.

12. The method according to claim 1, characterized in that The image to be evaluated is at least an image including cutout content, and the cutout content corresponds to the target object.

13. An image processing effect evaluation device, characterized in that: include: A key point determination module is used to determine the object edge key points corresponding to the target object in the image to be evaluated; An effect category determination module is used to input the neighborhood image to be evaluated corresponding to the edge key point of the object into a pre-trained effect evaluation model, and output the effect category corresponding to the neighborhood image to be evaluated; wherein the neighborhood image to be evaluated includes a plurality of pixel points centered on the edge key point of the object; The evaluation effect determination module is used to determine the target evaluation effect of the image to be evaluated according to the effect category corresponding to each neighborhood image to be evaluated.

14. An electronic device, characterized in that: The electronic device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the image processing effect evaluation method as described in any one of claims 1-12.

15. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the image processing effect evaluation method according to any one of claims 1 to 12 when executed by a computer processor.

16. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program is used to execute the image processing effect evaluation method according to any one of claims 1 to 12.

Images