High light video processing method, device, storage medium and electronic device

Abstract

The present invention discloses a highlight video processing method, device, storage medium and electronic device, the method comprising: performing identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; calculating the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtaining the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; determining the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; adjusting the comprehensive scoring result according to the out-of-frame rate; and screening the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain the highlight video that meets the conditions. The method greatly reduces the amount of calculation and the calculation time, and improves the quality of the highlight video of the target person.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a highlight video processing method, device, storage medium and electronic equipment. Background Art

[0002] With the development of the Internet, people's demand for watching videos and socializing through online platforms is increasing. Among them, the audience of video websites is becoming more and more extensive. Users can watch various videos and interact with anchors or others in real time. In this process, users hope to understand the highlight video clips (such as wonderful clips) in the video, and when watching real-time live broadcasts or other videos, they can choose the video clips they are more concerned about. However, the quality of the highlight video clips obtained by the existing technology is poor. For example, the user appears in the obtained highlight video clips for a short time, resulting in a poor user experience. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a highlight video processing method, device, storage medium and electronic device to solve the technical problem of poor quality of highlight video clips obtained in the prior art.

[0004] The technical solution proposed by the present invention is as follows:

[0005] A first aspect of an embodiment of the present invention provides a highlight video processing method, which includes: performing identity matching on the acquired highlight moment video to obtain a highlight moment video of a target person; calculating the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to preset evaluation indexes of multiple dimensions, and obtaining weights corresponding to the preset evaluation indexes of different dimensions to perform a comprehensive score on the highlight moment video; determining the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; adjusting the comprehensive scoring result according to the out-of-frame rate; and screening the images in the highlight moment video according to the adjusted comprehensive scoring result and a preset threshold to obtain a highlight video that meets the conditions.

[0006] Optionally, matching the identities of users in the acquired highlight moment video to obtain the highlight moment video of the target person includes: acquiring a video image of a person appearing in the acquired highlight moment video; determining the video image of the target person in the person video image according to a method of image search; and determining the highlight moment video of the target person based on the video image.

[0007] Optionally, determining the video image of the target person in the person video image according to the image-based image search method includes: determining the target person according to a face re-recognition method.

[0008] Optionally, the preset evaluation index includes: any one of: confidence of potential highlights, proportion of image search mode, change in position of center point of head, change in pixel value of head and image quality.

[0009] A second aspect of an embodiment of the present invention provides a highlight video processing device, which includes: a matching module, used to perform identity matching on the acquired highlight moment video to obtain a highlight moment video of a target person; a processing module, used to calculate the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtain the weights corresponding to the preset evaluation index of different dimensions to comprehensively score the highlight moment video; a determination module, used to determine the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; an adjustment module, used to adjust the comprehensive scoring result according to the out-of-frame rate; a screening module, used to screen the images in the highlight moment video according to the adjusted comprehensive scoring result and a preset threshold to obtain a highlight video that meets the conditions.

[0010] Optionally, the matching module includes: an acquisition module, used to acquire the video image of the person appearing in the highlight moment video; a first determination module, used to determine the video image of the target person in the person video image according to the image search method; and a second determination module, used to determine the highlight moment video of the target person based on the video image.

[0011] Optionally, the first determination module includes: a third determination module, configured to determine the target person according to a face re-recognition method.

[0012] Optionally, the preset evaluation index includes: any one of: confidence of potential highlights, proportion of image search mode, change in position of center point of head, change in pixel value of head and image quality.

[0013] A third aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the highlight video processing method as described in the first aspect of the embodiment of the present invention and any one of the first aspects.

[0014] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the highlight video processing method as described in the first aspect of the embodiment of the present invention and any one of the first aspects by executing the computer instructions.

[0015] The technical solution provided by the present invention has the following effects:

[0016] The highlight video processing method provided by the embodiment of the present invention performs identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; calculates the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtains the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; determines the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; adjusts the comprehensive scoring result according to the out-of-frame rate; and filters the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain a highlight video that meets the conditions. The method sets evaluation indicators for highlight moment videos, and comprehensively scores the highlight moment videos according to the weights of the evaluation indicators, which greatly reduces the amount of calculation and calculation time; adjusts the comprehensive scoring result according to the out-of-frame rate, and improves the quality of the highlight video of the target person. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flowchart of a highlight video processing method according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of a highlight video image according to an embodiment of the present invention;

[0020] Figure 3 is a structural block diagram of a highlight video processing device according to an embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of the structure of a computer-readable storage medium provided according to an embodiment of the present invention;

[0022] Figure 5 is a schematic structural diagram of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0024] The embodiment of the present invention provides a method for processing a highlight video. Figure 1 As shown, the method comprises the following steps:

[0025] Step S101: Perform identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person. Specifically, the highlight moment video is acquired in the camera video, and the identity of the target person in the highlight moment video is matched, and then the highlight moment video of the target person is obtained. For example, when the acquired video is a video captured by a camera in a kindergarten, the target person may be a kindergarten child, etc.; the acquired highlight moment video may refer to wonderful video clips of the kindergarten child's daily activities, including a series of interesting scenes such as being happy, funny, dancing, playing games, etc.

[0026] Step S102: Calculate the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtain the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video. Specifically, after obtaining the highlight moment video of the target person, set evaluation indicators for multiple dimensions of the highlight moment video, and calculate the score of each image in the highlight moment video under the preset evaluation index of each dimension according to the set evaluation indicators, and then obtain the weights corresponding to the preset evaluation indicators of different dimensions, and calculate the comprehensive score of the highlight moment video in combination with the weights. Among them, the weights corresponding to the preset evaluation indicators of different dimensions can be assigned different values ​​according to the effect of the obtained highlight moment video; the sum of the weights corresponding to the preset evaluation indicators of different dimensions is 100; the embodiment of the present application does not specifically limit the weights corresponding to the preset evaluation indicators of different dimensions, as long as they can meet the needs.

[0027] Step S103: Determine the out-of-frame rate of the target person in each video image according to the position of the target person in each video image. Specifically, after obtaining the comprehensive score of the highlight moment video, determine the out-of-frame rate of the target person in each video image according to the position of the target person in each video image to ensure the appearance rate of the target person in the highlight moment video finally obtained. Among them, out-of-frame rate = out-of-frame size / actual size, specifically, the actual size refers to the size of the target person image; the out-of-frame size refers to the image size of the target person when it is not in the current video image.

[0028] In one embodiment, if Figure 2 As shown, the highlight moment video image a of the target person, and the target person image b. Among them, d is the out-of-frame image of the target person in the video image, and the out-of-frame rate of the target person at the current moment is: the size of c / the size of b.

[0029] Step S104: Adjust the comprehensive scoring result according to the out-of-frame rate. Specifically, after obtaining the out-of-frame rate of the target person in each video image, a subtraction operation is performed on the obtained comprehensive scoring result according to the out-of-frame rate based on the comprehensive scoring result according to a preset threshold, and the final comprehensive scoring result of the target person's highlight moment video is adjusted and obtained.

[0030] In one embodiment, the comprehensive scoring result can be adjusted according to the frame rate: when the frame rate is greater than 50%, 40 points are directly subtracted from the comprehensive scoring result; when the frame rate is less than 50%, it is calculated according to the following formula:

[0031] cut_score=10 2x ×4

[0032] In the formula, cut_score represents the score that needs to be subtracted; x represents the frame rate.

[0033] Specifically, when x=10%, approximately 6.3 points are deducted; when x=20%, approximately 10 points are deducted; when x=30%, approximately 15.9 points are deducted; when x=40%, approximately 25.2 points are deducted; when x=0, no deduction operation is performed. The embodiment of the present application does not limit the above score adjustment method, and those skilled in the art can determine it according to actual needs.

[0034] Step S105: Screen the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain a highlight video that meets the conditions. Specifically, after adjusting the comprehensive scoring result, screen the comprehensive scoring result according to the preset threshold, and the highlight moment video obtained after screening is a high-quality highlight video that meets the conditions. The preset threshold is greater than or equal to 60 and less than or equal to 100. Specifically, the higher the preset threshold, the higher the quality of the highlight video obtained.

[0035] In one embodiment, the preset threshold is set to 80, and the video images with a comprehensive score greater than or equal to 70 are screened out to obtain the highlight video of the target person.

[0036] The highlight video processing method provided by the embodiment of the present invention performs identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; calculates the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtains the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; determines the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; adjusts the comprehensive scoring result according to the out-of-frame rate; and filters the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain a highlight video that meets the conditions. The method sets evaluation indicators for the highlight moment video, and comprehensively scores the highlight moment video according to the weights of the evaluation indicators, which greatly reduces the amount of calculation and the calculation time; adjusts the comprehensive scoring result according to the out-of-frame rate, and improves the quality of the highlight video of the target person.

[0037] As an optional implementation of the embodiment of the present invention, when matching the identity of the user in the acquired highlight moment video to obtain the highlight moment video of the target person, first obtain the video image of the person appearing in the highlight moment video, and then determine the video image of the target person in the highlight moment video according to the face re-recognition method and the image search method in combination with the original image of the target person. The original image of the target person refers to the image of the person itself in any state.

[0038] Specifically, after obtaining all the video images of the characters appearing in the highlight moment video, the original image of the target character is combined and the image search method is used to search in all the video images of the character, and the video image of the target character can be determined in all the video images of the character. At this time, the video image of the target character obtained is static and one-sided. Therefore, after determining the video image of the target character in the highlight moment video, the identity of the target task can be further determined by using the face re-recognition method, and the video image of the target character in the highlight moment video can be finally determined based on the successfully matched identity combined with the video image of the target character obtained before, and then the highlight moment video of the target character can be determined based on the video image. Among them, the video image of the target character finally obtained is the video image of the target character in any state and at any time.

[0039] In one embodiment, after combining the original image of the target person and searching all the person's video images using the image search method, a frontal video image of the target person A facing the video is obtained. After the identity of the target task is determined by the face re-recognition method, a video image of the target person in any state in the highlight moment video can be obtained. Finally, the highlight moment video of the target person is determined based on the video image. Among them. Face re-recognition is a technology that uses computer vision technology to determine whether there is a specific person in an image or video sequence, which solves the visual limitations of a fixed camera. Specifically, given any image of a target person, images of the target person in any scene and any state can be retrieved.

[0040] As an optional implementation of the embodiment of the present invention, the preset evaluation indicators include: any of the following: confidence of potential highlights, proportion of image search mode, change in the position of the center point of the head, change in the pixel value of the head, and image quality. Specifically, the preset evaluation indicators are normalized:

[0041] Confidence of potential highlights = the average confidence of all highlight attributes in the detection frame;

[0042] Image search mode ratio = image search mode / number of image searches;

[0043] Change of the center position of the head = (change of the center position of a single head - mean change of the center position of the head) / variance of the center position change of the head;

[0044] Change of head pixel value = (change of single head pixel value - mean change of head pixel value) / variance of head pixel value change;

[0045] Image quality = (single image quality - image quality mean) / image quality variance.

[0046] As an optional implementation of the embodiment of the present invention, step S102 specifically performs a comprehensive score on the highlight moment video:

[0047] Highlight video score = N1 * confidence + N2 * head pixel change + N3 * image search majority ratio + N4 * image quality + N5 * head center point change.

[0048] Among them, N1+N2+N3+N4+N5=100, that is, the full score of the video is 100 points.

[0049] The embodiment of the present invention also provides a highlight video processing device, such as Figure 3 As shown, the device comprises:

[0050] The matching module 401 is used to perform identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; for details, please refer to the relevant description of step S101 in the above method embodiment.

[0051] Processing module 402 is used to calculate the score of each image in the highlight moment video of the target person under the preset evaluation indicators of each dimension according to the preset evaluation indicators of multiple dimensions, and obtain the weights corresponding to the preset evaluation indicators of different dimensions to comprehensively score the highlight moment video; for details, please refer to the relevant description of step S102 in the above method embodiment.

[0052] The determination module 403 is used to determine the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; for details, please refer to the relevant description of step S103 in the above method embodiment.

[0053] The adjustment module 404 is used to adjust the comprehensive scoring result according to the frame rate; for details, please refer to the relevant description of step S104 in the above method embodiment.

[0054] The screening module 405 is used to screen the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain the highlight video that meets the conditions; for details, please refer to the relevant description of step S105 in the above method embodiment.

[0055] The highlight video processing device provided by the embodiment of the present invention performs identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; calculates the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtains the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; determines the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; adjusts the comprehensive scoring result according to the out-of-frame rate; and filters the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain a highlight video that meets the conditions. The method sets evaluation indicators for highlight moment videos, and comprehensively scores the highlight moment videos according to the weights of the evaluation indicators, which greatly reduces the amount of calculation and the calculation time; adjusts the comprehensive scoring result according to the out-of-frame rate, and improves the quality of the highlight video of the target person.

[0056] As an optional implementation of an embodiment of the present invention, the matching module includes: an acquisition module, used to acquire a video image of a person appearing in a highlight moment video; a first determination module, used to determine a video image of a target person in the person video image according to a method of searching by image; and a second determination module, used to determine the highlight moment video of the target person based on the video image.

[0057] As an optional implementation of the embodiment of the present invention, the first determination module includes: a third determination module, used to determine the target person according to the face re-recognition method.

[0058] As an optional implementation of an embodiment of the present invention, the preset evaluation indicators include: any multiple of: confidence of potential highlights, proportion of image search majority, change in position of the center point of the head, change in pixel value of the head, and image quality.

[0059] For a detailed description of the functions of the highlight video processing device provided in the embodiment of the present invention, please refer to the description of the highlight video processing method in the above embodiment.

[0060] The embodiment of the present invention also provides a storage medium, such as Figure 4 As shown, a computer program 601 is stored thereon, and when the instruction is executed by the processor, the steps of the highlight video processing method in the above embodiment are implemented. The storage medium also stores audio and video stream data, feature frame data, interaction request signaling, encrypted data, and preset data size, etc. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0061] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0062] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, the electronic device may include a processor 51 and a memory 52, wherein the processor 51 and the memory 52 may be connected via a bus or other means. Figure 5 The example of connecting through bus is taken in the following.

[0063] The processor 51 may be a central processing unit (CPU). The processor 51 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0064] The memory 52 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the corresponding program instructions / modules in the embodiment of the present invention. The processor 51 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 52, that is, the highlight video processing method in the above method embodiment is implemented.

[0065] The memory 52 may include a program storage area and a data storage area, wherein the program storage area may store an application required for operating the device and at least one function; the data storage area may store data created by the processor 51, etc. In addition, the memory 52 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 52 may optionally include a memory remotely arranged relative to the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0066] The one or more modules are stored in the memory 52 and when executed by the processor 51, perform the following steps: Figure 1 -2 is a highlight video processing method in the embodiment shown in FIG.

[0067] For details of the above electronic equipment, please refer to Figure 1 to Figure 2 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0068] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A highlight video processing method, It is characterized in that The steps include: Perform identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person, wherein the highlight moment video represents a wonderful video clip of the target person; Calculate the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation index of multiple dimensions, and obtain the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; Determining the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; Adjusting the comprehensive scoring result according to the frame rate; Screening the images in the highlight moment video according to the adjusted comprehensive scoring result and the preset threshold to obtain a highlight video that meets the conditions; Among them, identity matching is performed on the acquired highlight moment video to obtain the highlight moment video of the target person, including: Acquire an original image of the target person and a video image of the person appearing in the highlight moment video; Based on the original image, searching and determining a first video image of the target person in the person video image by using a method of searching by image, wherein the first video image is static and one-sided; Processing the first video image through a face re-recognition method and determining the identity of the target person; Determine a second video image of the target person in the highlight moment video based on the identity of the target person and the first video image, where the second video image is a video image of the target person in any state and at any time; Determine the highlight moment video of the target person according to the second video image.

2. The method according to claim 1, It is characterized in that The preset evaluation indicators include: any number of confidence of potential highlights, proportion of image search majority, change in the position of the center point of the head, change in the pixel value of the head, and image quality.

3. A highlight video processing device, It is characterized in that include: A matching module is used to perform identity matching on the acquired highlight moment video to obtain the highlight moment video of the target person; A processing module, used to calculate the score of each image in the highlight moment video of the target person under the preset evaluation index of each dimension according to the preset evaluation indexes of multiple dimensions, and obtain the weights corresponding to the preset evaluation indexes of different dimensions to comprehensively score the highlight moment video; A determination module, used to determine the out-of-frame rate of the target person in each video image according to the position of the target person in each video image; An adjustment module, used for adjusting the comprehensive scoring result according to the frame rate; A screening module, used for screening images in the highlight moment video according to the adjusted comprehensive scoring result and a preset threshold value to obtain a highlight video that meets the conditions; Wherein, the matching module is specifically used for: Acquire an original image of the target person and a video image of the person appearing in the highlight moment video; based on the original image, use an image-based search method to search for and determine a first video image of the target person in the person video image, wherein the first video image is static and one-sided; process the first video image through a face re-recognition method and determine the identity of the target person; based on the identity of the target person and the first video image, determine a second video image of the target person in the highlight moment video, wherein the second video image is a video image of the target person in any state and at any time; determine the highlight moment video of the target person based on the second video image.

4. The device according to claim 3, It is characterized in that The preset evaluation indicators include: any of the following: confidence of potential highlights, proportion of image search majority, change in position of the center point of the head, change in pixel value of the head, and image quality.

5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the highlight video processing method as claimed in claim 1 or 2.

6. An electronic device, It is characterized in that include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the highlight video processing method as claimed in claim 1 or 2 by executing the computer instructions.

Images