Method, apparatus, electronic device and medium for obtaining video subjective evaluation data

Abstract

The present application relates to the field of evaluation technologies, and discloses a method, apparatus, electronic device and medium for obtaining video subjective evaluation data. Among them, the method for obtaining video subjective evaluation data includes: obtaining marking data of at least one video frame in the video to be evaluated during the playback of the video to be evaluated; the marking data includes at least one of: marking time, marking type, marking times, marking position, and marking duration from the display start time of the video frame to the marking time when the user performs the marking operation; determining the subjective evaluation data corresponding to the video to be evaluated according to the marking data of the at least one video frame. In the solution provided by the present application, the subjective evaluation data is data at the video frame level. Therefore, the obtained subjective evaluation data is more detailed, and the obtained subjective evaluation data can more accurately and detailedly characterize the characteristics of the video to be evaluated, and the obtained subjective evaluation data is more accurate and reliable.

Description

Technical Field

[0001] This application relates to the field of evaluation technologies. Specifically, this application relates to a method, apparatus, electronic device, and medium for obtaining video subjective evaluation data. Background Art

[0002] With the development of video services, people's requirements for video image quality are getting higher and higher. Since video image data will inevitably affect the image quality during the processes of acquisition, compression, transmission, and processing, the accurate evaluation of video image quality has become one of the key technologies in the field of video image research. The evaluation methods of video image quality can be divided into subjective evaluation and objective evaluation. The subjective evaluation method is relatively direct, and the results are accurate and reliable. Moreover, accurate and reliable subjective evaluation results can also be used to guide the research of objective evaluation algorithms.

[0003] The existing subjective evaluation method refers to selecting a group of non-professional subjects, watching a set of test sequences, scoring the video quality in this set of sequences, and finally obtaining the Mean Opinion Score (MOS). Currently, the subjective evaluation tools in the industry mainly follow the standard test methods of the International Telecommunication Union (ITU). This standard mainly proposes four subjective evaluation methods: 1) The tester first watches the original video and then watches the distorted video, and scores the distorted video; 2) The playing order of the original video and the distorted video is random, and both the original video and the distorted video are scored; 3) Only the distorted video is played, and the distorted video is scored; 4) Only the distorted video is played, and the sequence duration is relatively long. The score not only considers the magnitude but also the scoring time. However, the score finally given by this evaluation method can only give an overall conclusion, and the evaluation result is not detailed enough to determine the video quality details based on the obtained evaluation result. Summary of the Invention

[0004] The purpose of this application is to at least solve one of the above technical defects, and the following technical solutions are specifically proposed:

[0005] In one aspect of this application, a method for obtaining video subjective evaluation data is provided, including:

[0006] Obtaining the marking data of at least one video frame in the to-be-evaluated video during the playing process of the to-be-evaluated video; the marking data includes at least one of the marking moment, marking type, marking times, marking position, and marking duration from the display start moment of the video frame to the marking moment when the user performs the marking operation;

[0007] Determining the subjective evaluation data of the corresponding to-be-evaluated video according to the marking data of the at least one video frame.

[0008] Another aspect of the present application provides an apparatus for obtaining video subjective evaluation data, which includes:

[0009] A module for obtaining marked data, configured to obtain marked data of at least one video frame in the video to be evaluated by a user during the playback of the video to be evaluated; the marked data includes at least one of: a marked time, a marked type, a marked count, a marked position, and a marked duration from the start time of display of the video frame to the marked time when the user performs a marking operation.

[0010] A module for determining subjective evaluation data, configured to determine subjective evaluation data corresponding to the video to be evaluated according to the marked data of the at least one video frame.

[0011] Another aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method for obtaining video subjective evaluation data shown in the first aspect of the present application.

[0012] Another aspect of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for obtaining video subjective evaluation data shown in the first aspect of the present application.

[0013] The beneficial effects brought by the technical solution provided by the present application are:

[0014] The method for obtaining video subjective evaluation data provided by the present application receives marked data of video frames in the video to be evaluated by a user during the playback of the video to be evaluated, and obtains subjective evaluation data of the video to be evaluated according to the marked data. Since the subjective evaluation data is data at the video frame level, rather than the evaluation data at the overall video level in the prior art, the obtained subjective evaluation data is more detailed, and the obtained subjective evaluation data can more accurately and detailedly characterize the characteristics of the video to be evaluated, and the obtained subjective evaluation data is more accurate and reliable.

[0015] Additional aspects and advantages of the present application will be given in part in the following description, which will become apparent from the following description, or can be understood through the practice of the present application. Description of the Drawings

[0016] The above and / or additional aspects and advantages of the present application will become apparent and easy to understand from the following description of the embodiments in conjunction with the drawings, where:

[0017] Figure 1 is a flowchart of a method for obtaining video subjective evaluation data provided by an embodiment of the present application;

[0018] Figures 2-1 to 2-2Schematic diagram showing that different marking operations performed by a user correspond to different marking types provided by an embodiment of the present application;

[0019] Figures 3-1 to 3-2 Schematic diagram showing the evaluation of a video to be evaluated in the form of single video playback provided by another embodiment of the present application;

[0020] Figure 4 Schematic diagram showing the evaluation of a video to be evaluated in the form of dual video playback provided by an embodiment of the present application;

[0021] Figures 5-1 to 5-2 Display interface diagram of video frame images corresponding to different marking times performed on video frames provided by another embodiment of the present application;

[0022] Figure 6 Interface schematic diagram showing the evaluation result of a video to be evaluated displayed on a corresponding display interface provided by an embodiment of the present application;

[0023] Figure 7 Schematic diagram of a user interface provided by an embodiment of the present application;

[0024] Figure 8 Logic schematic diagram of a method for obtaining video subjective evaluation data provided by an embodiment of the present application;

[0025] Figure 9 Service architecture diagram of a method for obtaining video subjective evaluation data provided by an embodiment of the present application;

[0026] Figure 10 Structural schematic diagram of a device for obtaining video subjective evaluation data provided by an embodiment of the present application;

[0027] Figure 11 Structural schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0028] The embodiments of the present application will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be construed as a limitation to the present application.

[0029] Those skilled in the art can understand that, unless specifically stated otherwise, the singular forms "a", "an", "the" and "said" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of this application means the presence of features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. It should be understood that when we say that an element is "connected" or "coupled" to another element, it can be directly connected or coupled to other elements, or there may also be intermediate elements. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The phrase "and / or" used herein includes all or any unit and all combinations of one or more related listed items.

[0030] Those skilled in the art can understand that, unless otherwise defined, all terms used herein (including technical terms and scientific terms) have the same meaning as the general understanding of those of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in a general dictionary should be understood to have a meaning consistent with the meaning in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as here.

[0031] User "attitude": The user expresses an attitude of approval or disapproval of the current video frame by performing a marking operation.

[0032] Microservice: A software architecture based on services with single responsibility and small functions as basic modules, and the modules communicate with each other using language-independent APIs.

[0033] Mini program: An application that can be used without downloading and installing.

[0034] Regarding the technical problems existing in the prior art, the method, device, electronic device and medium for obtaining video subjective evaluation data provided by this application are intended to solve at least one of the above technical problems in the prior art.

[0035] The following uses specific embodiments to elaborate in detail on the technical solution of this application and how the technical solution of this application solves the above technical problems. These several specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this application will be described below in conjunction with the accompanying drawings.

[0036] In the application scenario of the method for obtaining video subjective evaluation data provided by an embodiment of the present application, it includes: a background processor and a human-computer interaction interface. The human-computer interaction interface is on the client side. The background processor can be the processor of the client side or the server side. When the client receives an evaluation task of a video to be evaluated, in response to this evaluation task, it plays the video to be evaluated on the human-computer interaction interface and receives the marking operation of the user on the video to be evaluated on the human-computer interaction interface. The client converts the marking operation received through the human-computer interaction interface into marking data and sends the marking data of at least one video frame to the background processor. The background processor determines the subjective evaluation data of the video to be evaluated based on the marking data of at least one video frame.

[0037] An embodiment of the present application provides a possible implementation manner. As Figure 1 shown, a flowchart of a method for obtaining video subjective evaluation data is provided. This solution can be executed on a background processor, and the background processor can be set on the server side, including the following steps:

[0038] Step S110, obtain the marking data of at least one video frame in the video to be evaluated by the user during the playback of the video to be evaluated; wherein, the marking data includes at least one of: marking time, marking type, marking times, marking position, and marking duration from the start time of display to the marking time of the user's marking operation for the video frame.

[0039] Step S120, determine the subjective evaluation data corresponding to the video to be evaluated according to the marking data of at least one video frame.

[0040] Play the video to be evaluated on the human-computer interaction interface. The video to be evaluated includes at least one target video frame. During this process, each video frame of the video to be evaluated can receive the marking operation of the user on this video frame, that is, the evaluation entrance is open to the user from the first frame to the last frame of the playback of the video to be evaluated. When the user performs a marking operation on a certain video frame, there is no need to close the current playback interface, and the marking operation can be performed on the display interface of the current video to be evaluated. The human-computer interaction interface receives the marking operation of the user on this video frame and determines the marking data of this video frame based on the marking operation.

[0041] The user can perform marking operations on each video frame of the video to be evaluated, or can perform marking operations on several video frames of the video to be evaluated. For example: marking operations can be performed on the video frames with obvious video quality changes in the video to be evaluated, or marking operations can be performed on the video frames with good or bad quality, reducing the amount of subjective evaluation data while obtaining accurate evaluation data for the video to be evaluated.

[0042] Before playing the video to be evaluated on the client, it also includes: displaying a marking rule on the current display interface, where the marking rule includes: the marking type corresponding to the marking operation, the meaning of the number of markings, etc. For example, clicking on the target video frame indicates that the user approves of the target video frame, and drawing a line on the target video frame indicates that the user does not approve of the target video frame. That is, the marking operation represents the user's attitude towards the target video frame. That is, by the display interface, it is informed to the user that when the viewing is comfortable and the video is clear, etc., click on the target video frame to express approval of the target video frame, where the click includes operations such as single-click and double-click. The number of markings represents the intensity of the user's attitude towards the target video frame. Before the user performs the marking operation, understanding the meaning of the marking operation according to the marking rule displayed on the display interface is beneficial to accurately annotating the video to be evaluated.

[0043] Combined with Figures 2-1 to 2-2 As shown, different marking operations performed by the user correspond to different marking types. For the video frame image shown in the figure, the marking operation of the user on this video frame image is received to express the user's attitude towards this video frame image. Figure 2-1 In [a certain situation], after the user performs the marking operation, a heart-shaped pattern appears, which can represent that the user approves of this video frame. Figure 2-2 In [a certain situation], after the user performs the marking operation, an exclamation mark appears, representing that the user does not approve of this video frame.

[0044] Optionally, the marking operation performed by the user on the video frame can display the marking position, marking type, etc. on the current interface, so that the user can clarify the object and type of the marking operation and avoid marking errors.

[0045] Obtain the marking data corresponding to the video frame on which the marking operation is performed in the video to be evaluated, and determine the subjective evaluation data of this video to be evaluated according to the marking data of at least one frame of video frame.

[0046] The method for obtaining the subjective evaluation data of the video provided by this application receives the marking data of any video frame in the video to be evaluated during the playback of the video to be evaluated, and obtains the subjective evaluation data of this video to be evaluated according to the marking data. Since this subjective evaluation data is data at the video frame level, rather than the evaluation data at the overall video level in the prior art, therefore, the obtained subjective evaluation data is more detailed, and the obtained subjective evaluation data can more accurately and detailedly represent the characteristics of the video to be evaluated, and the obtained subjective evaluation data is more accurate and reliable.

[0047] In order to more clearly understand the solution for obtaining the subjective evaluation data of the video provided by this application and its technical effects, next, multiple embodiments will be used to elaborate on its specific implementation in detail.

[0048] In one embodiment, obtaining the marking data of at least one video frame in the video to be evaluated during the playback of the video to be evaluated in step S120 includes:

[0049] During the playback of the video to be evaluated, obtain the marking data of at least one video frame in the video to be evaluated in sequence. The marking data includes at least one of the following: marking time, marking type, number of markings, marking position, and marking duration from the start time of display to the marking time when the user performs the marking operation for the video frame.

[0050] Receive the marking time when the user marks the video frame. This marking time is calculated starting from the playback of the first frame of the video to be evaluated. Based on the marking time in the marking data, the video quality can be determined, such as the time point when the video quality changes.

[0051] According to the pre-set correspondence between the marking operation and the marking type, determine the marking type based on the marking operation performed by the current user. For example, clicking on the target video frame indicates that the user approves of the target video frame, and drawing a line on the target video frame indicates that the user does not approve of the target video frame. That is, the marking operation represents the user's attitude towards the target video frame; the number of markings represents the intensity of the user's attitude towards the target video frame. Before the user performs the marking operation, the meaning of the marking operation is clarified according to the marking rules displayed on the display interface, which is beneficial to accurately annotating the video to be evaluated.

[0052] The number of markings represents the intensity of the marking type. For example, when the marking type is approval, during the playback of the target video frame, if multiple clicks of the same marking type are received for the target video frame, it indicates that the user has a high degree of approval of the quality of the target video frame. The more the number of markings, the higher the intensity.

[0053] Among them, the intensity of the marking type can also be represented by the marking strength. The marking strength can be determined by the size of the contact area on the human-computer interaction interface. A large contact area represents a high intensity of the marking type.

[0054] The marking position represents the position where the marking operation occurs in the target video frame, further refining the subjective evaluation data of the video to be evaluated. According to the marking position, the specific position with good video quality or the specific position with poor video quality can be determined. Based on the marking position, the subjective evaluation data of the video to be evaluated can be further refined, improving the comprehensiveness and fineness of the representation of the subjective evaluation data.

[0055] The marking duration is the duration from the start moment of video frame display to the marking moment when the user performs the marking operation. That is, it is the duration from the start of display of this video frame to the moment when this video frame receives the user's marking. This marking duration can characterize whether the image quality of this video frame is easy to distinguish. For example, the shorter the marking duration, the easier it is to distinguish the quality of the video frame. The quality of this video frame includes good quality and bad quality. The longer the marking duration, the more difficult it is to distinguish the video frame quality.

[0056] The user's evaluation of the video to be evaluated can be in the form of single - video playback, that is, the video frames are sequentially displayed on the current display interface in the order of the video frames; or in the form of dual - video or multi - video playback, that is, at least two synchronized video frames are simultaneously displayed on the current display interface, but the video quality of each video frame is different.

[0057] Figures 3-1 to 3-2 For the evaluation of the video to be evaluated in the form of single - video playback, Figure 3-1 is the image at the T - th second, Figure 3-2 is the image at the (T + 1) - th second. The two images belong to the same video to be evaluated. By comparing the video frame images at different times, it can be seen that: in Figure 3 - 1, the stripes of the hand and the object are clear, but in Figure 3 - 2, the details of the hand are lost. The quality of the two video frame images has changed significantly. The user can mark this situation, mark that the video frame at the (T + 1) - th second has changed significantly in image quality, and can also, through the corresponding marking operation, mark that the video frame of the video to be evaluated at the (T + 1) - th second has become significantly worse in image quality. By comparing and marking the video through a single video, that is, by comparing the later - played video frame with the previously - played video frame vertically, the change in video quality can be obtained.

[0058] The evaluation of the video to be evaluated can also be in the form of multi - video playback. For example, Figure 4 As shown, when using the form of dual - video playback to evaluate the video to be evaluated, on the current display interface, two videos to be evaluated with different video qualities are displayed. When the two videos to be evaluated are played, due to different video qualities, there are differences in the display quality of static images and dynamic images. For example, when the video with higher clarity displays static items, the display quality is better, but when displaying dynamic items, when the action amplitude or movement frequency of the dynamic object is large, jitter will occur, resulting in a poor user experience. According to this situation, when the user notices a large difference in video quality between the two display screens in the same display interface, the user can mark the video to be evaluated with good perception, and / or mark the video to be evaluated with poor perception. For example, Figure 4 As shown, in this application, the left - hand image with better video quality is marked, and a heart - shaped pattern is displayed on the current display interface. The number of heart - shaped patterns can be proportional to the number of marking times. Figure 4The comparison of the two images shows that the video quality of the left image with a heart pattern is better.

[0059] When evaluating the video to be evaluated in the form of single-video playback, marking can be performed by clicking on the current display interface. When evaluating the video to be evaluated in the form of multi-video playback, where multi-video includes dual-video, marking of video frames can also be performed by comparing video frames in different display areas.

[0060] Figures 5-1 to 5-2 It is a display interface diagram of video frame images corresponding to different marking times for video frames. If an identification mark appears each time it is marked, as Figures 5-1 to 5-2 shown, when marked once, a heart pattern is displayed on the current display interface. The user marks the Figure 5-1 video frame image once, and the user marks the Figure 5-2 video frame image multiple times. The number of markings represents the intensity of the user's attitude towards the video frame. If the marking operation performed by the current user represents approval of the video frame, the more times of marking, the higher the degree of approval of the user for the video frame.

[0061] Combined with Figures 2-1 to 5-2 any of the legends shown, when the user marks the target video frame, the marking position can also be recorded, and the specific position of approval or disapproval of the target video frame can be determined according to the marking position, further improving the refinement of subjective evaluation data.

[0062] In an optional embodiment, after step S130, it further includes: determining the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated, and displaying the evaluation result in the corresponding display interface of the video to be evaluated.

[0063] Determining the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated. The target video frame to be marked can be determined according to the marking time, the target position of the marking can be determined according to the marking position, and the degree of approval / disapproval can be determined according to the number of markings and the marking duration. The evaluation results are such as: the target video frames that are approved, the target video frames that are not approved, the target positions that are approved, the target positions that are not approved, etc.

[0064] After obtaining the evaluation result of the video to be evaluated, the evaluation result is displayed in the corresponding display interface to visually and automatically display the evaluation result. For example: displaying the approved target video frames and target positions in one display area, and displaying the unapproved target video frames and target positions in another display area. Or displaying both the approved target video frames and target positions and the unapproved target video frames and target positions in one display area, and performing text annotation on the target video frames and target positions to indicate whether they are approved and the approved / unapproved target positions.

[0065] In an alternative embodiment, the interface schematic diagram for displaying the evaluation result of the video to be evaluated on the corresponding display interface is as Figure 6 shown, including 3 videos to be evaluated, namely: evaluation video 1, evaluation video 2, and evaluation video 3. On the display interface corresponding to evaluation video 1, the marked target video frames and target positions in the video to be evaluated are displayed on the corresponding display interface, and the target positions are marked on the picture in text form, as Figure 6 shown. 3 target positions are displayed on the display interface and are shown on the picture in text form. The target positions include the text and the house structure in the picture.

[0066] This application determines the evaluation result based on subjective evaluation data, realizes automatically obtaining the evaluation result, and displays the evaluation result through the display interface, which can visually display the evaluation result and realize the visual display of the evaluation result.

[0067] Optionally, displaying the evaluation result in the display interface corresponding to the video to be evaluated includes:

[0068] Displaying the picture to be displayed with a preset duration including the marked moment on the display interface, and / or, displaying the picture to be displayed within a preset range including the marked position on the display interface.

[0069] Obtaining the video frame image corresponding to the marked moment, and displaying the picture to be displayed with a preset duration including the marked moment on the display interface. Wherein, the marked moment is any moment within the preset duration.

[0070] Optionally, the marked moment can be the central moment of the preset duration, that is, taking the marked moment as the central moment, obtaining the video frame images of the second preset duration before and after the marked moment, and the second preset duration is half of the above-mentioned preset duration. The preset duration and the second preset duration can be determined and adjusted according to the actual situation.

[0071] The picture to be displayed obtained in this application is a picture with a preset duration including the marked moment. If the preset duration is short, the picture corresponding to the preset duration including the marked moment can be a single video frame image. If the preset duration is long, the picture corresponding to the preset duration including the marked moment can be multiple video frame images. Adopting this method can verify whether there is a marking error in the target video frame corresponding to the marked moment.

[0072] Display the pictures to be displayed within a preset range including the marked position on the display interface. During the display, the display range is expanded, and the pictures to be displayed within the preset range including the marked position are displayed on the display interface to clarify the marked position. The preset range can be confirmed and adjusted according to the actual situation. For example, the target position can be compared with the surrounding positions to clarify the mark type; when there are multiple identical objects in the target video, the marked position of the object can be clarified after expanding the display range, realizing the confirmation of the mark type and the marked position.

[0073] In an alternative embodiment, when the video to be evaluated includes multiple pictures to be displayed, the evaluation results can be displayed in the corresponding display interface of the video to be evaluated, which can be achieved in the following ways, including:

[0074] A1. Obtain the density of the marking operations performed by the user on the pictures to be displayed, and determine the display weight of each picture to be displayed according to the density.

[0075] A2. Determine the display information of the pictures to be displayed according to the display weight, where the display information includes at least one of the display position and the display area.

[0076] A3. Display multiple pictures to be displayed on the corresponding display interface according to the display information.

[0077] When there are multiple pictures to be displayed, the display position, display area, etc. of the pictures to be displayed need to be determined by a display strategy. In an alternative display strategy, obtain the density of the marking operations performed by the user on the pictures to be displayed. The pictures to be displayed include the target video frame at the marking moment or the target video frame including the marked position. The density of the marking operations performed on the pictures to be displayed is also the density of the marking operations on the target video frame corresponding to the marking moment or the target video frame including the marked position.

[0078] The density of the marking operations can be calculated by the number of marking operations received per unit time. The higher the density of the marking operations, the stronger the degree of approval / disapproval. If the current marking operation is an approval, the higher the density of the marking operations, the higher the degree of approval of the user for the picture to be displayed.

[0079] Determine the display weight of each picture to be displayed according to the density of the marking operations. It can be set that the higher the density, the greater the display weight of the picture to be displayed, and the greater the display weight can correspond to better display resources.

[0080] Determine the display information of the to-be-displayed picture according to the display weight of the to-be-displayed picture. The display information includes at least one of the display position and the display area. The greater the display weight, the closer the display position of the to-be-displayed picture can be to the display center, or the corresponding display area can be larger, so as to enhance the prominence of the to-be-displayed picture.

[0081] Determine the display information of each to-be-displayed picture in the above manner, and display the to-be-displayed pictures on the corresponding display interface according to the display information.

[0082] The solution provided by this application determines the display position, display area, etc. of the to-be-displayed picture according to the density of the marking operations received by the to-be-displayed picture, and can display the to-be-displayed picture that the user most approves or least approves in the most prominent way on the corresponding display interface, intuitively reflecting the key quality manifestation points of the to-be-evaluated video.

[0083] In an alternative embodiment, the above display information further includes a display form. Determining the display information of the to-be-displayed picture according to the display weight includes:

[0084] A21, when the to-be-displayed picture receives multiple marking operations, obtain the duration from the first mark to the last mark of the to-be-displayed picture;

[0085] A22, determine the display form of the to-be-displayed picture according to the duration. The display form includes: a dynamic picture or a static picture.

[0086] When any to-be-displayed picture receives multiple marking operations, perform the following operations on the to-be-displayed picture: obtain the first marking moment when the to-be-displayed picture receives the first mark, and the second marking moment when the to-be-displayed picture receives the last mark, and obtain the duration between the first marking moment and the second marking moment.

[0087] Compare the duration with a preset duration. The preset duration can be the maximum duration that a static image can represent. If it exceeds the preset duration, display the to-be-displayed picture as a dynamic picture. If it does not exceed the preset duration, display the to-be-displayed picture as a static picture.

[0088] In an alternative embodiment, determine the evaluation result of the to-be-evaluated video according to the subjective evaluation data of the to-be-evaluated video, including:

[0089] Determine the quality key points of the to-be-evaluated video according to the subjective evaluation data. Among them, the quality key points include at least one of the quality key time points and the quality key positions.

[0090] After obtaining the subjective evaluation data, the quality key points of the video to be evaluated can be determined based on the subjective evaluation data, which can be done in the following way: The quality key time points of the video to be evaluated can be determined according to the marked times corresponding to the marked target video frames, and the quality key positions can be determined according to the marked target positions. For example, Figure 7 as shown in the schematic diagram of the user interface Figure 7 Marking operations have been performed on the 3 video frames in. Therefore, the marked times corresponding to these 3 video frames are the quality key time points of the video to be evaluated. The marked positions of each video frame are different. From left to right, the marked positions are: "Tencent", building structure, "tencent". Therefore, the quality key points of the video to be evaluated containing these 3 video frames are the positions corresponding to "Tencent", building structure, and "tencent".

[0091] Combined with Figure 8 as shown in the logical schematic diagram to elaborate the implementation logic of this application: The mobile phone stores the marking data of the video frames of the video to be evaluated by the user. The marking data includes: marked time, number of markings, marking density, marking intensity, marking type, and marking position. Based on the marking data, the subjective evaluation data of the video to be evaluated is determined and displayed in the display interface corresponding to the video to be evaluated. Among them, the marking position can be determined by its horizontal and vertical coordinates in the coordinate system.

[0092] The service architecture corresponding to this application is as Figure 9 shown. This architecture includes a mobile phone applet end, a background microservice end, and a PC web page end. The PC web page end creates an evaluation task and sends the evaluation task to the mobile phone applet end via the background service end. The mobile phone applet end responds to the task, obtains the marking data of the video to be evaluated by the user through the human-computer interaction interface of the mobile phone applet end, and uploads it to the background microservice end. The background microservice end obtains the evaluation results of the video to be evaluated through services such as task service, video service, report service, user management service, notification service, and gateway management service, and sends them to the PC web page for display.

[0093] Among them, the configurations of the front end and the background can adopt the following configuration information to quickly and accurately obtain the subjective evaluation data of the video to be evaluated.

[0094] Background: TafGo, tRPC-Go are microservice architectures;

[0095] Front end: Node, React, WebPack, Tea UI;

[0096] Applet end: WeChat applet.

[0097] This application obtains the subjective evaluation data of the video to be evaluated through the above-mentioned method for obtaining subjective evaluation data of the video. On this basis, a subjective evaluation database can be constructed. The subjective evaluation data in the subjective evaluation database are used to train a video quality assessment algorithm based on deep learning. Finally, the trained video quality assessment algorithm is deployed to the business line to observe and control possible quality problems in a closed-loop manner.

[0098] The method for obtaining subjective video evaluation data provided by this application realizes an automated comprehensive evaluation service that records the user's subjective "attitude" at any time, and can quickly and intuitively locate the core capabilities of a certain video transcoding method or a certain video algorithm for improving video quality. For example, if it is determined based on subjective evaluation data that a certain video transcoding can achieve lossless compression for people, but the transcoding loss rate for other objects is high, it means that the core capability of the video transcoding lies in the processing of people. The characteristics of the image quality of a certain type of video can be clarified, and the corresponding characteristics can be quantified to specific time points and locations. Video evaluation can be completed faster, more comprehensively, and more automatically, and can be used to enhance the development of video algorithms.

[0099] In addition, the method for obtaining subjective video evaluation data provided by this application can record real-time user attitude data that is ignored in traditional evaluation schemes, achieving the effect of "frame-by-frame data" for the video, which is used to support video evaluation conclusions and quickly locate problems. Similarly, based on frame-by-frame video data, evaluation standards for video transcoding gears and video algorithms can be formulated to evaluate videos. After standardized mining of frame-by-frame data, it can be used as input for automated video clarity algorithms to build better and newer frame-by-frame evaluation models.

[0100] Based on the same principle as the method provided in the embodiment of the present application, the embodiment of the present application also provides a device 100 for obtaining video subjective evaluation data, such as Figure 10 As shown, the device may include: obtaining a tag data module 110, determining subjective evaluation data 120, wherein:

[0101] The module for obtaining marking data 110 is used to obtain marking data of at least one frame of the video to be evaluated by the user during the playback of the video to be evaluated; wherein the marking data includes at least one of the marking time, marking type, marking number, marking position, and marking duration of the video frame from the start time of display to the marking time when the user performs the marking operation;

[0102] Determine subjective evaluation data 120, which is used to determine the subjective evaluation data corresponding to the video to be evaluated according to the marking data of at least one video frame.

[0103] The device for obtaining subjective evaluation data of a video provided by this application receives the marking data of video frames in the video to be evaluated during the playback of the video to be evaluated, and obtains the subjective evaluation data of the video to be evaluated according to the marking data. Since the subjective evaluation data is data at the video frame level, rather than the evaluation data at the overall video level in the prior art, the obtained subjective evaluation data is more detailed, and the obtained subjective evaluation data can more accurately and detailedly characterize the characteristics of the video to be evaluated, and the obtained subjective evaluation data is more accurate and reliable.

[0104] Optionally, the device 100 for obtaining subjective evaluation data of a video further includes:

[0105] A display module, configured to determine the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated, and display the evaluation result in the corresponding display interface of the video to be evaluated.

[0106] Optionally, the display module is further configured to:

[0107] Display the to-be-displayed picture with a preset duration including the marked moment on the display interface, and / or display the to-be-displayed picture within a preset range including the marked position on the display interface.

[0108] Optionally, when there are multiple to-be-displayed pictures in the video to be evaluated, the display module is configured to:

[0109] Obtain the density of the marking operations performed by the user on the multiple to-be-displayed pictures, and determine the display weight of each to-be-displayed picture according to the density;

[0110] Determine the display information of the to-be-displayed pictures according to the display weight, where the display information includes at least one of a display position and a display area;

[0111] Display the multiple to-be-displayed pictures on the corresponding display interface according to the display information.

[0112] Optionally, the display information further includes a display form, and the display module is configured to:

[0113] When the to-be-displayed picture receives multiple marking operations, obtain the duration from the first marking to the last marking of the to-be-displayed picture;

[0114] Determine the display form of the to-be-displayed picture according to the duration, where the display form includes: a dynamic picture or a static picture.

[0115] Optionally, the display module is further configured to:

[0116] Determine the quality key points of the video to be evaluated according to the subjective evaluation data; the quality key points include at least one of a quality key time point and a quality key position.

[0117] The device for obtaining video subjective evaluation data in the embodiments of the present application can execute the method for obtaining video subjective evaluation data provided in the embodiments of the present application, and their implementation principles are similar. The actions performed by each module and unit in the device for obtaining video subjective evaluation data in each embodiment of the present application correspond to the steps in the method for obtaining video subjective evaluation data in each embodiment of the present application. For the detailed function descriptions of each module of the device for obtaining video subjective evaluation data, reference can specifically be made to the descriptions in the corresponding method for obtaining video subjective evaluation data shown above, and details are not described herein again.

[0118] Based on the same principle as the method shown in the embodiments of the present application, an electronic device is further provided in the embodiments of the present application. The electronic device may include, but is not limited to: a processor and a memory; the memory is used to store a computer program; the processor is used to execute the method for obtaining video subjective evaluation data shown in any optional embodiment of the present application by calling the computer program. Compared with the prior art, in the present application, during the playback of the video to be evaluated, the user's marking data for the video frames in the video to be evaluated is received, and the subjective evaluation data of the video to be evaluated is obtained according to the marking data. Since the subjective evaluation data is data at the video frame level, the obtained subjective evaluation data is more detailed, and the obtained subjective evaluation data can more accurately and detailedly characterize the characteristics of the video to be evaluated, and the obtained subjective evaluation data is more accurate and reliable.

[0119] In an optional embodiment, an electronic device is provided, as Figure 11 shown. Figure 11 As shown, the electronic device 4000 may be a server, including: a processor 4001 and a memory 4003. Among them, the processor 4001 and the memory 4003 are connected, such as connected through a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation to the embodiments of the present application.

[0120] The processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute various exemplary logical blocks, modules, and circuits described in connection with the disclosure of this application. The processor 4001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0121] The bus 4002 may include a path for transmitting information between the above components. The bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus 4002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 only a thick line is used to represent it in the figure, but it does not mean that there is only one bus or one type of bus.

[0122] The memory 4003 may be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, or it may also be an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or other optical disc storage, optical disc storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0123] The memory 4003 is used to store the application program code for executing the solution of this application, and is controlled by the processor 4001 to execute. The processor 4001 is used to execute the application program code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0124] Among them, the electronic device includes but is not limited to: mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 11 The illustrated electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of this application.

[0125] The embodiments of this application provide a computer-readable storage medium, on which a computer program is stored. When it runs on a computer, it enables the computer to execute the corresponding content in the foregoing method embodiments.

[0126] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same moment, but can be executed at different moments, and their execution order is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0127] It should be noted that the above-mentioned computer-readable medium in the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. And in the present application, the computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0128] The above-mentioned computer-readable medium can be included in the above-mentioned electronic device; or it can exist separately and not be assembled into the electronic device.

[0129] The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is caused to execute the method shown in the above-mentioned embodiments.

[0130] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0131] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0132] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation on the module itself in some cases. For example, the module for obtaining marker data can also be described as "the module for obtaining marker data of at least one video frame in the video to be evaluated".

[0133] The above description is only for the preferred embodiments of this application and the explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in this application is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features with similar functions disclosed in this application.

Claims

1. A method for obtaining subjective evaluation data of video, characterized in that, it includes: responding to a marking operation performed on at least one video frame of a video to be evaluated in a display interface, obtaining marking data corresponding to the at least one video frame; the marking operation is performed frame by frame; the display interface includes at least two regions, each region displays a video to be evaluated with a certain video quality, and the video qualities displayed in different regions are different while the video content is the same; the marking operation is used to mark the video to be evaluated with a better visual experience and / or mark the video to be evaluated with a worse visual experience when there is a large difference in video quality between two display pictures in the same display interface; the marking data includes at least one of: marking time, different marking types corresponding to different marking operations, marking times, marking positions, and the marking duration from the start time of display to the marking time when the user performs the marking operation for the video frame; determining the subjective evaluation data of the video to be evaluated according to the marking data corresponding to the at least one video frame.

2. The method according to claim 1, characterized in that, it further includes: determining the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated, and displaying the evaluation result in the display interface corresponding to the video to be evaluated; or, in response to the marking operation, also displaying the marking effect corresponding to the marking operation.

3. The method according to claim 2, characterized in that, the displaying the evaluation result in the display interface corresponding to the video to be evaluated includes: displaying a to-be-displayed picture with a preset duration including the marking time on the display interface, and / or, displaying a to-be-displayed picture within a preset range including the marking position on the display interface.

4. The method according to claim 3, characterized in that, when there are multiple to-be-displayed pictures in the evaluation result, the displaying the evaluation result in the display interface corresponding to the video to be evaluated includes: obtaining the density of the marking operations performed by the user on the multiple to-be-displayed pictures, and determining the display weight of each to-be-displayed picture according to the density; determining the display information of the to-be-displayed pictures according to the display weight, the display information includes at least one of the display position and display area in the display interface; displaying the multiple to-be-displayed pictures on the corresponding display interface according to the display information.

5. The method according to claim 4, characterized in that, the display information further includes a display form, and the determining the display information of the to-be-displayed pictures according to the display weight includes: when the to-be-displayed picture receives multiple marking operations, obtaining the duration from the first marking to the last marking of the to-be-displayed picture; determining the display form for displaying the to-be-displayed picture according to the duration, the display form includes: dynamic picture or static picture.

6. The method according to claim 2, characterized in that, the determining the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated includes: Determine the quality key points of the video to be evaluated according to the subjective evaluation data; the quality key points include at least one of a quality key time point and a quality key position; the quality key position indicates the marked positions in each video frame.

7. An apparatus for obtaining subjective evaluation data of a video, characterized in that it includes: A marking data acquisition module, configured to acquire marking data corresponding to at least one video frame in the video to be evaluated in response to a marking operation performed on at least one video frame in the display interface; the marking operation is performed on a frame-by-frame basis; the display interface includes at least two regions, and each region displays a video to be evaluated with a certain video quality, and the video qualities displayed in different regions are different while the video content is the same; the marking operation is used to mark the video to be evaluated with a better visual experience and / or the video to be evaluated with a worse visual experience when there is a large difference in video quality between two display pictures in the same display interface; the marking data includes at least one of a marking moment, different marking types corresponding to different marking operations, the number of marking times, a marking position, and the marking duration from the start moment of display to the marking moment when the user performs the marking operation for the video frame. Determine subjective evaluation data, configured to determine the subjective evaluation data of the video to be evaluated according to the marking data corresponding to at least one video frame.

8. The apparatus according to claim 7, characterized in that the apparatus for obtaining subjective evaluation data of the video further includes: A display module, configured to determine the evaluation result of the video to be evaluated according to the subjective evaluation data of the video to be evaluated, and display the evaluation result in the display interface corresponding to the video to be evaluated.

9. An electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the program, it implements the method for obtaining subjective evaluation data of a video according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the program is executed by a processor, it implements the method for obtaining subjective evaluation data of a video according to any one of claims 1-6.

Images