A video title and ending recognition method, device, equipment and readable storage medium

Abstract

The present invention provides a method, device, equipment and readable storage medium for identifying video titles and endings, and relates to the field of video recognition technology. The method comprises: obtaining a first time range in which a scene switch occurs in a target video according to recognition information of a target video; wherein the recognition information comprises at least one of sound, image and text recognition information; matching multiple frames of first images within a first duration before a first moment in the first time range with multiple frames of second images within a first duration after the first moment, and obtaining a first matching result; matching first sound data within a first duration before a first moment in the first time range with second sound data within a first duration after the first moment, and obtaining a second matching result; obtaining the title ending moment or the ending starting moment of the target video according to the first matching result and the second matching result. The scheme of the present invention can efficiently and accurately identify the title and ending of a video.

Description

Technical Field

[0001] The present invention belongs to the technical field of video recognition, and in particular, relates to a method, device, equipment and readable storage medium for recognizing video titles and endings. Background Art

[0002] Existing video playback software provides users with the option of skipping the opening and ending of a video to improve the user experience and save viewing time. However, current methods for identifying the opening and ending of a video generally use manual annotation methods or video fingerprint technology.

[0003] Among them, the manual labeling method manually edits the opening and ending time points, which requires a lot of manpower to watch the video and label it, resulting in low recognition efficiency and high cost. Video fingerprint technology only performs video fingerprint detection in the set detection time area, and then determines the opening and ending time points based on the detection results. It relies on video fingerprints, and the detection results of video fingerprints greatly affect the recognition accuracy. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, equipment and readable storage medium for identifying video opening and ending, so as to solve the problems of low efficiency and low accuracy in the prior art of identifying video opening and ending.

[0005] In order to achieve the above object, an embodiment of the present invention provides a video title and ending recognition method, comprising:

[0006] According to the identification information of the target video, a first time range in which a scene switching occurs in the target video is obtained; wherein the identification information includes at least one of sound, image and text identification information;

[0007] Matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment to obtain a first matching result;

[0008] Matching first sound data within a first duration before a first moment in the first time range with second sound data within a first duration after the first moment to obtain a second matching result;

[0009] According to the first matching result and the second matching result, the beginning end time or the ending start time of the target video is obtained.

[0010] Optionally, the method for identifying a video title and an ending, obtaining a first time range in which a scene switching occurs in the target video according to the identification information of the target video, includes:

[0011] Divide the third sound data within the second time length before the target video into a plurality of sub-sound data, or divide the third sound data within the second time length after the target video into a plurality of sub-sound data;

[0012] Acquire multiple frames of images in a video segment within a second duration before the target video, or acquire multiple frames of images in a video segment within a second duration after the target video, according to a preset interval frame number;

[0013] Based on at least one of the following, a first time range in which a scene switch occurs in the target video is obtained:

[0014] A scene matching degree between the sound recognition information of the plurality of sub-sound data;

[0015] A scene matching degree between the image recognition information of the multiple frames of images;

[0016] The scene matching degree between the text recognition information of the multiple frames of images.

[0017] Optionally, the method for identifying the video title and ending, matching a plurality of first image frames within a first duration before a first moment in the first time range with a plurality of second image frames within a first duration after the first moment to obtain a first matching result, includes:

[0018] For the multiple frames of first images, perform target segmentation, and obtain positioning information and image feature information of the segmented first target;

[0019] Establishing a motion trajectory model of the first target according to the positioning information and image feature information of the first target in the multiple frames of first images;

[0020] According to the motion trajectory model, obtaining a predicted position of the first target in the plurality of frames of second images;

[0021] The location of the first target in the multiple frames of second images is matched with the predicted location to obtain the first matching result.

[0022] Optionally, the video title and ending recognition method, after performing object segmentation on the multiple frames of first images, further comprises:

[0023] Using the preset shape feature information, in the multiple frames of first images, targets whose color difference from the background is less than a preset threshold are obtained.

[0024] Optionally, the method for identifying the video title and ending, matching the first sound data within a first duration before a first moment in the first time range with the second sound data within a first duration after the first moment to obtain a second matching result, includes:

[0025] Acquire a first scene corresponding to the first sound data and a second scene corresponding to the second sound data by identifying acoustic information and semantic information in the first sound data and the second sound data;

[0026] The first scene is matched with the second scene to obtain the second matching result.

[0027] Optionally, the video title and ending recognition method, obtaining the title ending time or the ending starting time of the target video according to the first matching result and the second matching result, includes:

[0028] Acquire a second time range corresponding to at least one frame of the second image for which the first matching result is smaller than a first matching threshold;

[0029] Acquire a third time range corresponding to at least one second sound data whose second matching result is smaller than a second matching threshold;

[0030] Based on the second time range and the third time range, the end time of the header or the start time of the tail is obtained.

[0031] Optionally, the video title and ending recognition method, obtaining the title ending time or the ending starting time based on the second time range and the third time range, includes:

[0032] According to the overlapping moment of the second time range and the third time range, the ending moment of the title or the starting moment of the tail is obtained.

[0033] In order to achieve the above object, an embodiment of the present invention further provides a video title and ending recognition device, comprising:

[0034] A first acquisition module, configured to acquire a first time range in which a scene switching of the target video occurs according to identification information of the target video; wherein the identification information includes at least one of sound, image and text identification information;

[0035] A second acquisition module, used for matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment, to obtain a first matching result;

[0036] A third acquisition module is used to match the first sound data within a first time period before the first moment in the first time range with the second sound data within a first time period after the first moment to obtain a second matching result;

[0037] The fourth acquisition module is used to obtain the end time of the beginning or the start time of the ending of the target video according to the first matching result and the second matching result.

[0038] In order to achieve the above-mentioned purpose, an embodiment of the present invention also provides a video opening and ending recognition device, including a transceiver, a processor, a memory, and a program or instruction stored on the memory and executable on the processor; when the processor executes the program or instruction, the steps in the video opening and ending recognition method as described in any one of the above items are implemented.

[0039] In order to achieve the above-mentioned purpose, an embodiment of the present invention further provides a readable storage medium for storing a computer program, and when the computer program is executed by a processor, the steps in the video header and end recognition method as described above are implemented.

[0040] The above technical solution of the present invention has at least the following beneficial effects:

[0041] In the above scheme of the embodiment of the present invention, based on the identification information of the target video, the first time range in which the scene switching occurs in the target video is obtained, and multiple frames of first images within a first time length before a first moment in the first time range are matched with multiple frames of second images within a first time length after the first moment to obtain a first matching result, and first sound data within a first time length before a first moment in the first time range are matched with second sound data within a first time length after the first moment to obtain a second matching result, so as to obtain the end moment of the title or the start moment of the end of the target video based on the first matching result and the second matching result, that is, the end moment of the title or the start moment of the end of the end is analyzed and identified through image recognition technology and sound recognition technology, which has the advantages of high recognition efficiency and high recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a video title and ending recognition method provided by an embodiment of the present invention;

[0043] Figure 2 A schematic diagram of an application of a video title and ending recognition method provided by an embodiment of the present invention;

[0044] Figure 3 A structural diagram of a video title and ending recognition device provided by an embodiment of the present invention;

[0045] Figure 4 A structural diagram of a video title and ending recognition device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present invention. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0047] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0048] Additionally, the terms "system" and "network" are often used interchangeably herein.

[0049] In the embodiments provided in the present application, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0050] See also Figure 1 , Figure 1 is a flow chart of a video title and ending recognition method provided by an embodiment of the present invention. Figure 1 As shown, the following steps are included:

[0051] Step 101, obtaining a first time range in which a scene switch occurs in the target video according to identification information of the target video; wherein the identification information includes at least one of sound, image and text identification information;

[0052] In this step, feature values ​​can be formed based on the input sound recognition information, image recognition information, and text recognition information, and then the feature values ​​are input into a supervised deep learning model to determine the scene of the target video, and the first time range of the scene switching of the target video is output. Among them, the sound recognition information may include music, character dialogue, and narration, etc.; the image recognition information may include characters, places, and environments, etc. The deep learning model can automatically identify the first time range of the scene switching of the target video, thereby improving the recognition efficiency. The deep learning model is a pre-trained model.

[0053] It should be noted that, through this step, two first time ranges for scene switching of the target video can be obtained, which correspond to the beginning and the end of the video, respectively. Of course, the first time range of the beginning is before the first time range of the end of the video.

[0054] Step 102, matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment to obtain a first matching result;

[0055] In this step, image processing is performed on the first time range according to the first time range obtained in step 101 to obtain matching results of frame images before and after the first moment in the first time range, thereby obtaining the moment corresponding to the frame image where the before and after images do not match.

[0056] Step 103, matching the first sound data within a first time period before the first moment in the first time range with the second sound data within a first time period after the first moment to obtain a second matching result;

[0057] In this step, according to the first time range obtained in step 101, sound processing is performed on the first time range to obtain matching results of sound data before and after the first moment in the first time range, thereby obtaining the moment corresponding to the sound data where the before and after sounds do not match.

[0058] Step 104: Obtain the ending time of the beginning or the beginning time of the ending of the target video according to the first matching result and the second matching result.

[0059] It should be noted that the embodiments of the present invention can obtain scene recognition results obtained by performing image processing on multiple frames of first images and multiple frames of second images, and obtain scene recognition results obtained by performing sound processing on first sound data and second sound data based on the first matching result and the second matching result, and then input the scene recognition results of image processing and sound processing into the comparison model respectively, so as to output the end time of the beginning or the start time of the ending.

[0060] In an embodiment of the present invention, based on identification information of a target video, a first time range in which a scene switch occurs in the target video is obtained, and a plurality of frames of first images within a first duration before a first moment in the first time range are matched with a plurality of frames of second images within a first duration after the first moment to obtain a first matching result, and first sound data within a first duration before a first moment in the first time range are matched with second sound data within a first duration after the first moment to obtain a second matching result, thereby obtaining an end moment of the title or a start moment of the end of the target video based on the first matching result and the second matching result, that is, the end moment of the title or the start moment of the end of the end of the title is analyzed and identified through image recognition technology and sound recognition technology, which has the advantages of high recognition efficiency and high recognition accuracy.

[0061] In an optional embodiment of the present invention, step 101 includes:

[0062] Divide the third sound data within the second time length before the target video into a plurality of sub-sound data, or divide the third sound data within the second time length after the target video into a plurality of sub-sound data;

[0063] Acquire multiple frames of images in a video segment within a second duration before the target video, or acquire multiple frames of images in a video segment within a second duration after the target video, according to a preset interval frame number;

[0064] Based on at least one of the following, a first time range in which a scene switch occurs in the target video is obtained:

[0065] A scene matching degree between the sound recognition information of the plurality of sub-sound data;

[0066] A scene matching degree between the image recognition information of the multiple frames of images;

[0067] The scene matching degree between the text recognition information of the multiple frames of images.

[0068] The second duration is generally determined according to the duration of the beginning or the end of the video. The third sound data within the second duration before the target video and the video clip within the second duration before the target video are both used to identify the first time range of the beginning of the target video; the third sound data within the second duration after the target video and the video clip within the second duration after the target video are both used to identify the first time range of the end of the target video.

[0069] Multiple sub-sound data or multiple frame images are divided into multiple comparison groups, each group includes two sub-sound data or two frame images, and then each comparison group is input into the deep learning model to output the scene matching degree.

[0070] Based on the scene matching degree of at least one of the sound recognition information, the image recognition information and the text recognition information, a time range corresponding to at least two sub-sound data or at least two frames of images whose scene matching degree is less than a preset matching threshold is determined as a first time range.

[0071] It should be noted that the text recognition information is obtained based on text recognition of the frame image.

[0072] That is to say, this step 101 can be understood as obtaining the first time range through the video fingerprint technology.

[0073] In an optional embodiment of the present invention, step 102 includes:

[0074] For the multiple frames of first images, perform target segmentation, and obtain positioning information and image feature information of the segmented first target;

[0075] Establishing a motion trajectory model of the first target according to the positioning information and image feature information of the first target in the multiple frames of first images;

[0076] According to the motion trajectory model, obtaining a predicted position of the first target in the plurality of frames of second images;

[0077] The location of the first target in the multiple frames of second images is matched with the predicted location to obtain the first matching result.

[0078] It should be noted that image segmentation is to extract partial pixels in an image to express a known target. Here, since the goal is to identify the beginning or end of a video, the target that may appear at the beginning or end of each frame of the multiple first images, that is, partial pixels corresponding to the first target, are segmented out.

[0079] Then, the position of the first target is detected on the first image after target segmentation to determine the position of the first target in the first image and the size of the first target, that is, to determine the positioning information and image feature information of the first target, thereby establishing a motion trajectory model of the first target.

[0080] Furthermore, based on the motion trajectory model, the position of the first target in multiple frames of the second image is predicted, and the position of the first target in multiple frames of the second image is identified, and the identified position is matched with the predicted position to obtain a matching value, that is, a first matching result.

[0081] In an optional embodiment of the present invention, after performing object segmentation on the multiple frames of first images, the method further includes:

[0082] Using the preset shape feature information, in the multiple frames of first images, targets whose color difference from the background is less than a preset threshold are obtained.

[0083] It should be noted that since the target segmented may not be the target in the opening or ending of interest, that is, not the first target, or because the color difference between the target and the background is less than a preset threshold, resulting in inaccurate segmented targets, it is necessary to use preset shape feature information to further obtain targets whose color difference with the background is less than a preset threshold in the multiple frames of the first image after target segmentation, and improve the target segmentation result of the image, thereby selecting the first target from multiple segmented targets.

[0084] In an optional embodiment of the present invention, step 103 includes:

[0085] Acquire a first scene corresponding to the first sound data and a second scene corresponding to the second sound data by identifying acoustic information and semantic information in the first sound data and the second sound data;

[0086] The first scene is matched with the second scene to obtain the second matching result.

[0087] It should be noted that by identifying the acoustic information and semantic information in the sound data, the human voice and accompaniment are converted into a text analysis context, thereby obtaining the scene corresponding to the sound data.

[0088] The first scene corresponding to the first sound data is matched with the second scene corresponding to the second sound data to obtain a matching value, that is, a second matching result.

[0089] In an optional embodiment of the present invention, step 104 includes:

[0090] Acquire a second time range corresponding to at least one frame of the second image for which the first matching result is smaller than a first matching threshold;

[0091] Acquire a third time range corresponding to at least one second sound data whose second matching result is smaller than a second matching threshold;

[0092] Based on the second time range and the third time range, the end time of the header or the start time of the tail is obtained.

[0093] The first matching threshold and the second matching threshold are determined based on empirical values ​​or measured values.

[0094] The overlap time of the second time range and the third time range is analyzed and compared, so as to obtain the end time of the title or the start time of the end. Moreover, the start time of the title is the start time of the target video, and the end time of the end is the end time of the target video, so as to determine the title time range and the end time range.

[0095] It should be noted that the embodiments of the present invention can also obtain scene recognition results obtained by performing image processing on multiple frames of first images and multiple frames of second images, and obtain scene recognition results obtained by performing sound processing on first sound data and second sound data based on the first matching result and the second matching result, and then input the scene recognition results of image processing and sound processing into the comparison model respectively, so as to output the end time of the beginning or the start time of the ending.

[0096] Figure 2 Schematic diagram of the application of the video title and ending recognition method provided by the embodiment of the present invention. Figure 2 The application process of the video opening and ending recognition method is specifically described.

[0097] Step 201, input the target video.

[0098] Step 202, using video fingerprint technology to perform video retrieval and determine the time range of the beginning or the end of the video, that is, the first time range.

[0099] Step 203: extract multiple frames of first images and multiple frames of second images in a first time range of the target video.

[0100] Step 204: extract first sound data and second sound data in a first time range of the target video.

[0101] Step 205, object segmentation, segmenting the objects in each frame of the first image and each frame of the second image.

[0102] Step 206, target detection, performs position and size detection on the targets in the first image and the second image respectively, and determines the positioning information and image feature information of the targets in the first image and the second image.

[0103] Step 207, target recognition, qualitatively recognizes the target in the second image based on the image feature information, and matches the target in the first image with the target in the second image.

[0104] Step 208, target tracking, constructs a motion trajectory model based on the positioning information of the first target in the first image, and predicts the position of the first target in the second image based on the model, and performs scene matching with the identified position of the first target in the second image.

[0105] Step 209: Obtain the first matching result.

[0106] Step 210: Perform acoustic information recognition on the first sound data and the second sound data respectively.

[0107] Step 211: perform semantic information recognition on the first sound data and the second sound data respectively.

[0108] Step 212: Obtain a second matching result based on the identified scene.

[0109] Step 213: input the first matching result and the second matching result into a comparison model for training.

[0110] Step 214, output the beginning end time or the ending start time of the film.

[0111] In summary, the video opening and ending recognition method of the embodiment of the present invention combines image recognition technology and sound recognition technology to improve the efficiency and accuracy of video opening and ending recognition, while saving a lot of manual labeling work and reducing costs.

[0112] like Figure 3 As shown, the embodiment of the present invention further provides a video title and ending recognition device, comprising:

[0113] A first acquisition module 301 is used to acquire a first time range in which a scene switching of the target video occurs according to identification information of the target video; wherein the identification information includes at least one of sound, image and text identification information;

[0114] A second acquisition module 302 is used to match multiple frames of first images within a first time period before a first moment in the first time range with multiple frames of second images within a first time period after the first moment to obtain a first matching result;

[0115] A third acquisition module 303 is used to match the first sound data within a first duration before the first moment in the first time range with the second sound data within a first duration after the first moment to obtain a second matching result;

[0116] The fourth acquisition module 304 is used to acquire the beginning end time or the end start time of the target video according to the first matching result and the second matching result.

[0117] In an embodiment of the present invention, based on the identification information of the target video, a first time range in which a scene switch occurs in the target video is obtained, and a plurality of frames of first images within a first duration before a first moment in the first time range are matched with a plurality of frames of second images within a first duration after the first moment to obtain a first matching result, and first sound data within a first duration before a first moment in the first time range are matched with second sound data within a first duration after the first moment to obtain a second matching result, thereby obtaining the end moment of the title or the start moment of the end of the target video based on the first matching result and the second matching result, that is, the end moment of the title or the start moment of the end of the end of the title is analyzed and identified through image recognition and sound recognition technology, which has the advantages of high recognition efficiency and high recognition accuracy.

[0118] Optionally, in the video title and ending recognition device, the first acquisition module 301 is specifically used for:

[0119] Divide the third sound data within the second time length before the target video into a plurality of sub-sound data, or divide the third sound data within the second time length after the target video into a plurality of sub-sound data;

[0120] Acquire multiple frames of images in a video segment within a second duration before the target video, or acquire multiple frames of images in a video segment within a second duration after the target video, according to a preset interval frame number;

[0121] Based on at least one of the following, a first time range in which a scene switch occurs in the target video is obtained:

[0122] A scene matching degree between the sound recognition information of the plurality of sub-sound data;

[0123] A scene matching degree between the image recognition information of the multiple frames of images;

[0124] The scene matching degree between the text recognition information of the multiple frames of images.

[0125] Optionally, in the video title and ending recognition device, the second acquisition module 302 is specifically used to:

[0126] For the multiple frames of first images, perform target segmentation, and obtain positioning information and image feature information of the segmented first target;

[0127] Establishing a motion trajectory model of the first target according to the positioning information and image feature information of the first target in the multiple frames of first images;

[0128] According to the motion trajectory model, obtaining a predicted position of the first target in the plurality of frames of second images;

[0129] The location of the first target in the multiple frames of second images is matched with the predicted location to obtain the first matching result.

[0130] Optionally, the video title and ending recognition device further comprises:

[0131] The fifth acquisition module is used to use the preset shape feature information to acquire, in the multiple frames of the first image, a target whose color difference from the background is less than a preset threshold.

[0132] Optionally, in the video title and ending recognition device, the third acquisition module 303 is specifically used for:

[0133] Acquire a first scene corresponding to the first sound data and a second scene corresponding to the second sound data by identifying acoustic information and semantic information in the first sound data and the second sound data;

[0134] The first scene is matched with the second scene to obtain the second matching result.

[0135] Optionally, in the video title and ending recognition device, the fourth acquisition module 304 includes:

[0136] A first acquisition unit, configured to acquire a second time range corresponding to at least one frame of a second image whose first matching result is smaller than a first matching threshold;

[0137] A second acquisition unit, configured to acquire a third time range corresponding to at least one second sound data whose second matching result is smaller than a second matching threshold;

[0138] The third acquisition unit is used to acquire the end time of the header or the start time of the tail based on the second time range and the third time range.

[0139] Optionally, in the video title and ending recognition device, the third acquisition unit is specifically used to:

[0140] According to the overlapping moment of the second time range and the third time range, the ending moment of the title or the starting moment of the tail is obtained.

[0141] It should be noted that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned video title and ending recognition method embodiment, and can achieve the same technical effect. The parts and beneficial effects that are the same as the method embodiment in this embodiment will not be described in detail here.

[0142] The embodiment of the present invention also provides a video title and ending recognition device, such as Figure 4 As shown, it includes: a processor 401; and a memory 402 connected to the processor 401 through a bus interface, the memory 402 is used to store programs and data used by the processor 401 when performing operations, and the processor 401 calls and executes the programs and data stored in the memory 402.

[0143] The processor 401 is used to read the program in the memory 402 and execute the following process:

[0144] According to the identification information of the target video, a first time range in which a scene switching occurs in the target video is obtained; wherein the identification information includes at least one of sound, image and text identification information;

[0145] Matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment to obtain a first matching result;

[0146] Matching first sound data within a first duration before a first moment in the first time range with second sound data within a first duration after the first moment to obtain a second matching result;

[0147] According to the first matching result and the second matching result, the beginning end time or the ending start time of the target video is obtained.

[0148] The transceiver 403 is connected to the bus interface and is used to receive and send data under the control of the processor 401 .

[0149] Among them, Figure 4 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically one or more processors represented by processor 401 and various circuits of memory represented by memory 402 are linked together. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 403 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 404 may also be an interface capable of externally connecting or internally connecting required devices, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, and the like.

[0150] The processor 401 is responsible for managing the bus architecture and general processing, and the memory 402 can store data used by the processor 401 when performing operations.

[0151] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0152] Divide the third sound data within the second time length before the target video into a plurality of sub-sound data, or divide the third sound data within the second time length after the target video into a plurality of sub-sound data;

[0153] Acquire multiple frames of images in a video segment within a second duration before the target video, or acquire multiple frames of images in a video segment within a second duration after the target video, according to a preset interval frame number;

[0154] Based on at least one of the following, a first time range in which a scene switch occurs in the target video is obtained:

[0155] A scene matching degree between the sound recognition information of the plurality of sub-sound data;

[0156] A scene matching degree between the image recognition information of the multiple frames of images;

[0157] The scene matching degree between the text recognition information of the multiple frames of images.

[0158] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0159] For the multiple frames of first images, perform target segmentation, and obtain positioning information and image feature information of the segmented first target;

[0160] Establishing a motion trajectory model of the first target according to the positioning information and image feature information of the first target in the multiple frames of first images;

[0161] According to the motion trajectory model, obtaining a predicted position of the first target in the plurality of frames of second images;

[0162] The location of the first target in the multiple frames of second images is matched with the predicted location to obtain the first matching result.

[0163] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0164] Using the preset shape feature information, in the multiple frames of first images, targets whose color difference from the background is less than a preset threshold are obtained.

[0165] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0166] Acquire a first scene corresponding to the first sound data and a second scene corresponding to the second sound data by identifying acoustic information and semantic information in the first sound data and the second sound data;

[0167] The first scene is matched with the second scene to obtain the second matching result.

[0168] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0169] Acquire a second time range corresponding to at least one frame of the second image for which the first matching result is smaller than a first matching threshold;

[0170] Acquire a third time range corresponding to at least one second sound data whose second matching result is smaller than a second matching threshold;

[0171] Based on the second time range and the third time range, the end time of the header or the start time of the tail is obtained.

[0172] Optionally, the processor 401 is further configured to read the computer program and execute the following steps:

[0173] According to the overlapping moment of the second time range and the third time range, the ending moment of the title or the starting moment of the tail is obtained.

[0174] It should be noted that the above-mentioned device provided by the embodiment of the present invention can implement all the method steps implemented by the above-mentioned video title and ending recognition method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0175] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by instructing related hardware through a program, wherein the program includes instructions for executing part or all of the steps of the above method; and the program may be stored in a readable storage medium, which may be any form of storage medium.

[0176] An embodiment of the present invention further provides a readable storage medium, wherein a program is stored on the readable storage medium, and when the program is executed by a processor, the video header and ending recognition method as described in any one of the above items is implemented.

[0177] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0178] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may be physically included separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0179] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform some steps of the sending and receiving methods described in various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store program codes.

[0180] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary personnel in this technical field, several improvements and modifications can be made without departing from the principles described in the present invention. These improvements and modifications are also within the scope of protection of the present invention.

Claims

1. A video title and ending recognition method, characterized in that: include: According to the identification information of the target video, a first time range in which a scene switching occurs in the target video is obtained; wherein the identification information includes at least one of sound, image and text identification information; Matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment to obtain a first matching result; Matching first sound data within a first duration before a first moment in the first time range with second sound data within a first duration after the first moment to obtain a second matching result; According to the first matching result and the second matching result, the beginning end time or the ending start time of the target video is obtained.

2. The method according to claim 1, characterized in that The step of obtaining, according to the identification information of the target video, a first time range in which a scene switching of the target video occurs includes: Divide the third sound data within the second time length before the target video into a plurality of sub-sound data, or divide the third sound data within the second time length after the target video into a plurality of sub-sound data; Acquire multiple frames of images in a video segment within a second duration before the target video, or acquire multiple frames of images in a video segment within a second duration after the target video, according to a preset interval frame number; Based on at least one of the following, a first time range in which a scene switch occurs in the target video is obtained: A scene matching degree between the sound recognition information of the plurality of sub-sound data; A scene matching degree between the image recognition information of the multiple frames of images; The scene matching degree between the text recognition information of the multiple frames of images.

3. The method according to claim 1, characterized in that The step of matching a plurality of first image frames within a first time period before a first moment in the first time range with a plurality of second image frames within a first time period after the first moment to obtain a first matching result includes: For the multiple frames of first images, perform target segmentation, and obtain positioning information and image feature information of the segmented first target; Establishing a motion trajectory model of the first target according to the positioning information and image feature information of the first target in the multiple frames of first images; According to the motion trajectory model, obtaining a predicted position of the first target in the plurality of frames of second images; The location of the first target in the multiple frames of second images is matched with the predicted location to obtain the first matching result.

4. The method according to claim 3, characterized in that After performing object segmentation on the multiple frames of first images, the method further includes: Using the preset shape feature information, in the multiple frames of first images, targets whose color difference from the background is less than a preset threshold are obtained.

5. The method according to claim 1, characterized in that The matching of the first sound data within a first duration before a first moment in the first time range with the second sound data within a first duration after the first moment to obtain a second matching result includes: Acquire a first scene corresponding to the first sound data and a second scene corresponding to the second sound data by identifying acoustic information and semantic information in the first sound data and the second sound data; The first scene is matched with the second scene to obtain the second matching result.

6. The method according to claim 1, characterized in that The obtaining, according to the first matching result and the second matching result, the beginning end time or the ending start time of the target video includes: Acquire a second time range corresponding to at least one frame of the second image for which the first matching result is smaller than a first matching threshold; Acquire a third time range corresponding to at least one second sound data whose second matching result is smaller than a second matching threshold; Based on the second time range and the third time range, the end time of the header or the start time of the tail is obtained.

7. The method according to claim 6, characterized in that The acquiring the end time of the title or the start time of the end of the title based on the second time range and the third time range includes: According to the overlapping moment of the second time range and the third time range, the ending moment of the title card or the starting moment of the tail card is obtained.

8. A video title and end recognition device, characterized in that: include: A first acquisition module, configured to acquire a first time range in which a scene switching of the target video occurs according to identification information of the target video; wherein the identification information includes at least one of sound, image and text identification information; A second acquisition module, used for matching a plurality of frames of first images within a first time period before a first moment in the first time range with a plurality of frames of second images within a first time period after the first moment, to obtain a first matching result; A third acquisition module is used to match the first sound data within a first time period before the first moment in the first time range with the second sound data within a first time period after the first moment to obtain a second matching result; The fourth acquisition module is used to obtain the end time of the beginning or the start time of the ending of the target video according to the first matching result and the second matching result.

9. A video title and ending recognition device, comprising a transceiver, a processor, a memory, and a program or instruction stored in the memory and executable on the processor; characterized in that: When the processor executes the program or instruction, the steps in the video header and end recognition method as described in any one of claims 1 to 7 are implemented.

10. A readable storage medium for storing a computer program, characterized in that: When the computer program is executed by a processor, the steps in the video header and end recognition method according to any one of claims 1 to 7 are implemented.

Images