Method, system, device and storage medium for generating point of interest display video

Abstract

The present invention provides a method, system, device and storage medium for generating a point of interest display video, the method comprising: performing scene segmentation on a point of interest video to obtain a number of point of interest sub-videos, establishing a sub-video set and removing duplicates; establishing a picture set containing a large number of pictures based on the same point of interest, classifying each picture set according to a preset scene and establishing a picture sub-set, obtaining a number of pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establishing a reference picture set; performing similarity calculation on the deduplicated point of interest sub-video and each reference picture, so as to match the preset scene corresponding to the reference picture to each point of interest sub-video; splicing the point of interest sub-videos according to a preset scene sequence to generate a point of interest display video. The present invention can automatically edit video materials that meet the current marketing scene from existing video materials, thereby significantly reducing the cost of video marketing and improving the user viewing experience.

Description

Background Art

[0002] POI (Point of Interest) is a term in Geographic Information System (GIS), which refers to a specific geographical location with some significance or attraction. For example, a hotel is a POI, a scenic spot is a POI, a restaurant is a POI, etc.

[0003] In recent years, with the rapid rise and development of short video platforms, more and more companies have begun to try to use video materials for marketing. Compared with traditional graphic marketing, video materials can use visual, audio and text to convey information at the same time, helping users understand complex concepts faster and more accurately, and can quickly capture users' attention. Videos are easier to share on social media, especially high-quality videos, which are far more effective than graphics in terms of dissemination.

[0004] The demand for POI video marketing is personalized and differentiated. For example, in hotel marketing, the interest points of the same hotel may be different each time. Some marketing scenes will highlight the hotel's parent-child features, while others will highlight the catering features. If new video materials need to be shot and manually edited for each marketing, it will bring high costs and low efficiency, making it difficult to keep up with the rapid iteration of marketing.

[0005] Although the existing AI model can generate a hotel display video based on multiple hotel introduction videos, the order of the scenes in the generated video is random. In particular, video clips of the same scene are used multiple times or video content of the same scene appears repeatedly. For users who browse various hotels on an OTA platform and compare hotels horizontally by watching, the disorder of the scene order greatly affects the viewing experience and the sense of control in subjective comparison, and the user experience is very poor.

[0006] Since many POIs have existing video materials that have been shot, how to use the existing video materials to produce videos that meet the needs of marketing delivery is a problem that this invention solves.

[0007] Therefore, the present invention provides a method, system, device and storage medium for generating a point of interest display video. Summary of the invention

[0008] In response to the problems in the prior art, the purpose of the present invention is to provide a method, system, device and storage medium for generating point of interest display videos, which overcome the difficulties of the prior art and can automatically edit video materials that meet the current marketing scenarios from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and enhance the user viewing experience.

[0009] An embodiment of the present invention provides a method for generating a point of interest display video, comprising the following steps:

[0010] S110, performing scene segmentation on the interest point video to obtain a plurality of interest point sub-videos, establishing a sub-video set of the interest point sub-videos and performing video deduplication;

[0011] S120, establishing a picture set including a large number of pictures based on the same interest point, classifying each picture set according to a preset scene, establishing a picture sub-set for each preset scene, obtaining a number of pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establishing a reference picture set;

[0012] S130, calculating similarity between the deduplicated interest point sub-video and each of the reference pictures, so as to match the preset scene corresponding to the reference picture to each of the interest point sub-videos;

[0013] S140: splicing the interest point sub-videos according to a preset scene sequence order to generate an interest point display video.

[0014] Preferably, the step S110 includes the following steps:

[0015] S111, establishing a video collection including a large number of videos of a point of interest based on a point of interest;

[0016] S112, performing scene segmentation on each interest point video to obtain a plurality of interest point sub-videos;

[0017] S113, establishing a sub-video set of all the interest point sub-videos;

[0018] S114: Deduplication of videos in the sub-video set.

[0019] Preferably, the step S120 includes the following steps:

[0020] S121, establishing a picture collection containing a large number of pictures based on the same point of interest, and classifying each picture collection according to a preset scene;

[0021] S122. For each preset scene, a picture subset is established. Several pictures with the lowest similarity to other pictures in the picture subset are obtained as reference pictures. The feature vector of each picture in the picture subset is obtained through the Clip model. Then, the similarity between different pictures is calculated using the cosine similarity. For several pictures with high similarity, only one of them is retained as the reference picture.

[0022] Preferably, the step S130 includes the following steps:

[0023] S131, calculating the similarity between the interest point sub-video after deduplication and each of the reference pictures;

[0024] S132: Obtain a reference picture with the highest similarity for each interest point sub-video, and use a preset scene of the reference picture as a label of the interest point sub-video.

[0025] Preferably, the step S140 includes the following steps:

[0026] S141, selecting at least one interest point sub-video as a backup sub-video from the interest point sub-videos with a unified label based at least on a preset video duration range and video quality;

[0027] S142: splicing the spare sub-videos according to a preset sequence of scenes to generate a point of interest display video.

[0028] Preferably, in step S142, the preset scene sequence is hotel exterior, hotel lobby, hotel room, hotel restaurant, hotel gym and hotel swimming pool.

[0029] Preferably, after step S140, the following steps are further included:

[0030] S150: performing face recognition on the interest point display video, and deleting video frames containing portraits.

[0031] An embodiment of the present invention further provides a system for generating a point of interest display video, which is used to implement the above-mentioned method for generating a point of interest display video. The system for generating a point of interest display video includes:

[0032] The sub-video collection module performs scene segmentation on the interest point video to obtain a number of interest point sub-videos, establishes a sub-video collection of the interest point sub-videos and performs video deduplication;

[0033] A reference picture set module is used to establish a picture set containing a large number of pictures based on the same point of interest, classify each picture set according to a preset scene, establish a picture sub-set for each preset scene, obtain several pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establish a reference picture set;

[0034] A similarity calculation module is used to calculate the similarity between the interest point sub-video after deduplication and each of the reference pictures, so as to match the preset scene corresponding to the reference picture to each of the interest point sub-videos;

[0035] The display video generation module splices the interest point sub-videos according to the order of a preset scene sequence to generate an interest point display video.

[0036] An embodiment of the present invention further provides a device for generating a point of interest display video, comprising:

[0037] processor;

[0038] a memory storing executable instructions of the processor;

[0039] Wherein, the processor is configured to execute the steps of the above-mentioned method for generating the point of interest display video by executing the executable instructions.

[0040] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the method for generating the above-mentioned point of interest display video.

[0041] The purpose of the present invention is to provide a method, system, device and storage medium for generating point of interest display videos, which can automatically edit video materials that meet the current marketing scenarios from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and enhance the user viewing experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0043] Figure 1 It is a flow chart of the method for generating a point of interest display video of the present invention.

[0044] Figure 2 It is a schematic diagram of the steps of implementing the method for generating a point of interest display video of the present invention.

[0045] Figure 3 It is a schematic diagram of the steps of the similar video deduplication process in the method for generating a point of interest display video of the present invention.

[0046] Figure 4 It is a structural schematic diagram of a system for generating a point of interest display video according to the present invention.

[0047] Figure 5 It is a structural schematic diagram of a device for generating a point of interest display video according to the present invention.

[0048] Figure 6 It is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present application. The present application can also be implemented or applied through other different specific implementation methods, and the details in the present application can also be modified or changed in various ways according to different viewpoints and application systems without departing from the spirit of the present application. It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0050] The following is a detailed description of the embodiments of the present application with reference to the accompanying drawings so that those skilled in the art can easily implement the present application. The present application can be embodied in a variety of different forms and is not limited to the embodiments described herein.

[0051] In the representations of the present application, the representations with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" etc. mean that the specific features, structures, materials or characteristics represented in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the represented specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples represented in the present application and the features of the different embodiments or examples, unless they contradict each other.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the representation of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0053] In order to clearly describe the present application, components irrelevant to the description are omitted, and the same reference symbols are given to the same or similar components throughout the specification.

[0054] Throughout the specification, when a device is said to be "connected" to another device, this includes not only the case of "direct connection" but also the case of "indirect connection" by placing other elements therebetween. In addition, when a device is said to "include" a certain component, unless otherwise stated, it does not exclude other components, but means that other components may be included.

[0055] When a device is said to be "on" another device, it may be directly on the other device, but there may also be other devices between it. In contrast, when a device is said to be "directly" on another device, there are no other devices between it.

[0056] Although the terms first, second, etc. are used to represent various elements in some examples, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, the first interface and the second interface, etc. are represented. Moreover, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising" and "including" indicate the existence of features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Only when the combination of elements, functions, steps or operations is inherently mutually exclusive in some way, will there be an exception to this definition.

[0057] The technical terms used herein are only used to refer to specific embodiments and are not intended to limit the present application. The singular form used herein also includes the plural form unless the sentence clearly indicates the contrary meaning. The meaning of "including" used in the specification is to specify specific characteristics, regions, integers, steps, operations, elements and / or components, and does not exclude the existence or addition of other characteristics, regions, integers, steps, operations, elements and / or components.

[0058] Although not defined differently, all terms, including technical and scientific terms used herein, have the same meaning as those generally understood by those skilled in the art to which this application belongs. Terms defined in commonly used dictionaries are additionally interpreted as having meanings consistent with the relevant technical literature and the contents of the current disclosure, and unless defined, shall not be overly interpreted as ideal or very formal meanings.

[0059] Figure 1 : is a flow chart of the method for generating a point of interest display video of the present invention. Figure 1 As shown, the method for generating a point of interest display video of the present invention includes:

[0060] S110, performing scene segmentation on the interest point video to obtain a plurality of interest point sub-videos, establishing a sub-video set of the interest point sub-videos and performing video deduplication;

[0061] S120, establishing a picture set including a large number of pictures based on the same interest point, classifying each picture set according to a preset scene, establishing a picture sub-set for each preset scene, obtaining a number of pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establishing a reference picture set;

[0062] S130, calculating similarity between the deduplicated interest point sub-video and each reference picture, so as to match the preset scene corresponding to the reference picture to each interest point sub-video;

[0063] S140: splicing the interest point sub-videos according to a preset scene sequence order to generate an interest point display video.

[0064] The purpose of this invention is to develop relevant algorithmic processes, automatically edit video materials that meet the current marketing scenario from existing video materials, so as to significantly reduce the cost of video marketing, provide different perspectives to show more specific and detailed hotel scenes and enhance the user viewing experience. To automatically edit relevant videos from existing videos, it is necessary to first segment the original video into scenes, and then splice them according to the given scene description based on the scene semantics of each video clip. Among them, the core function is to label the original video clips with scenes. For example, for a hotel, it is necessary to label the scene information such as the appearance, lobby, room, restaurant, etc. Due to the high content complexity of the original material, the present invention proposes to use POI images to assist the scene label recognition of video clips. Through offline evaluation, the accuracy of the video clip scene label reaches the available standard, so as to significantly reduce the cost of video marketing, provide different perspectives to show more specific and detailed hotel scenes and enhance the user viewing experience.

[0065] In a preferred embodiment, step S110 includes the following steps:

[0066] S111, establishing a video collection including a large number of videos of a point of interest based on a point of interest;

[0067] S112, performing scene segmentation on each interest point video to obtain a plurality of interest point sub-videos;

[0068] S113, creating a sub-video set of all interest point sub-videos;

[0069] S114. Deduplication of videos is performed in the sub-video set to avoid the occurrence of sub-videos of interest points with repeated contents, but the present invention is not limited thereto.

[0070] In a preferred embodiment, step S120 includes the following steps:

[0071] S121, establishing a picture collection containing a large number of pictures based on the same point of interest, and classifying each picture collection according to a preset scene;

[0072] S122. For each preset scene, a picture subset is established, and several pictures with the lowest similarity to other pictures are obtained in the picture subset as reference pictures. In this embodiment, the feature vector of each picture in the picture subset can be obtained through the Clip model, and then the similarity between different pictures is calculated using cosine similarity. For several pictures with high similarity, only one is retained as a reference picture, and by finding some pictures with the greatest contrast with other pictures in the picture subset, more pictures with different perspectives and different contents are obtained, thereby avoiding content homogeneity (a shortcoming of the prior art) when splicing videos later, but the present invention is not limited thereto.

[0073] In a preferred embodiment, step S130 includes the following steps:

[0074] S131, calculating the similarity between the interest point sub-video after deduplication and each reference picture;

[0075] S132: Obtain a reference picture with the highest similarity for each interest point sub-video, and use a preset scene of the reference picture as a label of the interest point sub-video, thereby obtaining a corresponding label based on the preset scene for each interest point sub-video, but the present invention is not limited thereto.

[0076] In a preferred embodiment, step S140 includes the following steps:

[0077] S141, selecting at least one interest point sub-video as a backup sub-video from the interest point sub-videos with a unified label based at least on a preset video duration range and video quality;

[0078] S142. Splicing the spare sub-videos according to the order of a preset scene sequence to generate a point of interest display video. The preset scene sequence is hotel exterior, hotel lobby, hotel room, hotel restaurant, hotel gym and hotel swimming pool. The present invention can unify the introduction videos of all hotels on the OTA platform into one viewing order through the preset scene sequence, which is conducive to improving the convenience of user horizontal comparison and significantly enhancing the user experience, but is not limited to this.

[0079] In a preferred embodiment, after step S140, the following steps are further included:

[0080] S150: Perform face recognition on the point of interest display video and delete video frames containing portraits, but the present invention is not limited thereto.

[0081] The specific implementation of the present invention is as follows:

[0082] Figure 2 It is a schematic diagram of the steps of implementing the method for generating a point of interest display video of the present invention. Figure 3Schematic diagram of the steps of the similar video deduplication process in the method for generating the point of interest display video of the present invention. Figure 2 and 3 As shown, the algorithm and process proposed in the present invention can automatically edit videos that meet the required scenes from videos related to the POI by giving a POI and scene description, greatly reducing the cost of video marketing. The present invention uses technical means such as video scene segmentation algorithm, similar video deduplication algorithm, similar image deduplication algorithm, video and image similarity algorithm, video face detection algorithm, video splicing, etc. to generate video materials that can be directly delivered by giving a POI and scene description.

[0083] The present invention is based on a given POI and scene description, and goes through 8 steps (see Figure 2 ) is used to splice and synthesize videos, wherein the scene description is mainly the expected splicing order of the videos. For example, taking a hotel as an example, it is hoped that the video sequence will be spliced ​​in the order of hotel exterior, lobby, room, restaurant, etc. Since the scenes in POI videos are relatively complex, if the scenes in the videos are directly classified, the accuracy will be relatively low, and the usability of the generated videos will be relatively poor. However, the image scenes of POIs are relatively simple and are manually maintained. Therefore, in this invention, by calculating the similarity between videos and pictures, the scene labels of the pictures are used to label the videos, thereby solving the problem of low accuracy of video scene labels.

[0084] Continue to refer Figure 2 and 3 As shown, the key technical solutions in the present invention are introduced in detail:

[0085] (1) Video scene segmentation

[0086] Video scene segmentation refers to the automatic division of a video into different scenes or shots according to content changes. Each scene usually represents a continuous segment in the video, and the boundaries of the scene are marked by significant content changes (such as scene switching, color changes, object movement, etc.). Video scene segmentation has a wide range of applications in film and television production, video analysis, content management, etc.

[0087] In the present invention, a scene segmentation algorithm is used to perform scene segmentation on the POI video material, and the POI video material is divided into different scene segments for use in subsequent processes.

[0088] Common techniques and algorithms for scene segmentation include detection based on inter-frame differences, detection based on motion information, detection based on content (feature extraction), gradient detection, machine learning methods, semantic analysis, etc. The present invention uses a detection method based on inter-frame differences to perform video scene segmentation.

[0089] The core of the frame difference detection method is to calculate the difference between consecutive frames. It is usually achieved through the following steps:

[0090] Convert to Grayscale: To simplify computation, frames are usually converted to grayscale images so that color information is ignored and the focus is on changes in brightness and structure.

[0091] Calculate frame difference: Calculate the difference of each pixel between the current frame and the previous frame. A common calculation method is to use the absolute value of the frame difference (for example, the difference in the grayscale value of each pixel).

[0092]

[0093] Among them, I t is the current frame, I t-1 is the previous frame, i,j are pixel coordinates.

[0094] Normalized difference: To cope with different resolutions or changes in video frame size, the difference values ​​are normalized (for example, by the total number of pixels in the frame).

[0095] Set threshold: The algorithm sets a threshold for the difference between frames. When the frame difference exceeds this preset threshold, it is considered that a scene switch has occurred. If the difference is less than the threshold, the consecutive frames are considered to belong to the same scene. The threshold can be fixed or dynamically adjusted according to the changes in the content of the frame to meet the sensitivity requirements of different scene switches in the video.

[0096] When the difference between two consecutive frames exceeds a preset threshold, it is considered to indicate a scene change. For example, if a video suddenly switches from an indoor scene to an outdoor scene, the change in video content is usually obvious, and the difference between frames will increase significantly, and the algorithm will determine the scene change.

[0097] (2) Deduplication of similar videos

[0098] Since different videos of the same POI may have repeated scenes, there will be repeated videos in the video clips cut by the scene segmentation algorithm. It is necessary to deduplicate the videos to prevent repeated scene clips in the later synthesized videos.

[0099] In the present invention, the feature vector of each frame image in the video is obtained by the Clip model (see Figure 3 ), then the feature vectors of all frames in each scene video are averaged as the feature vector of the video, and finally the similarity between different videos is calculated by cosine similarity. The overall calculation process is as follows Figure 2 As shown:

[0100] (3) Similar image deduplication algorithm

[0101] At present, there are a large number of pictures in each POI gallery. If you want to calculate the similarity between each video clip and all pictures, it will greatly increase the computing time cost. Therefore, it is necessary to deduplicate similar pictures and select the most representative TOP N pictures in each scene. The present invention uses the Clip model to obtain the feature vector of the picture, and then uses the cosine similarity to calculate the similarity between different pictures. For pictures with high similarity, only one is retained for subsequent processes.

[0102] (4) Image and video similarity matching

[0103] The core function of this invention is to label the video clips after the video scenes are segmented. For example, for a hotel, the scenes are divided into exterior, lobby, room, restaurant, gym, swimming pool, etc. After labeling, the videos can be spliced ​​in order according to needs.

[0104] The present invention calculates the similarity between the video clip and the picture of the POI, and takes the scene label of the picture with the highest similarity as the scene label of the video when the similarity threshold is met. This has two advantages:

[0105] Since the scene classification of each POI is different, for example, if Hotel A has a swimming pool, then Hotel A must have a swimming pool tag; while Hotel B may not have a swimming pool, so the scene tags of Hotel A and Hotel B will be different. If you tag the video with the hotel's pictures, you don't need to define the scene tags of each POI in advance.

[0106] Since the video content of each POI is relatively complex, some video scenes do not belong to the content of the POI itself. Therefore, if the scene labels are defined in advance, the scene labels of the video are judged by the similarity between the text and the video, which may lead to misjudgment. However, the present invention uses the similarity between the picture and the video to label, and the probability of misjudgment is greatly reduced.

[0107] The calculation logic of the image and video similarity in the present invention is as follows:

[0108] Extract image frames from video material and use Clip model to obtain feature vectors of each frame.

[0109] Use the Clip model to obtain the feature vector of the POI image.

[0110] Calculate the cosine similarity between the POI image and each frame of the video, and take the maximum value as the similarity between the image and the video.

[0111] If the similarity between the POI image and the video meets the threshold, the video is labeled with the scene to which the image belongs.

[0112] (5) Video face detection

[0113] Since there are some human face clips in the original video material, if the video clips with human faces are used, it may involve the issue of portrait rights. Therefore, the human face clips in the video need to be removed.

[0114] The present invention uses the MTCNN algorithm to detect face segments in videos. The MTCNN algorithm is a multi-task cascade CNN face detection deep learning model that can provide relatively accurate detection results in different complex scenes (such as light changes, occlusion, etc.). In MTCNN, three convolutional networks are cascaded:

[0115] The first stage of the network outputs candidate windows of faces

[0116] In the second stage, the candidate serial ports generated in the first stage are corrected to remove the candidate windows that do not meet the requirements.

[0117] The third stage further modifies the results based on the second stage and gives the final landmarks of the five faces.

[0118] The present invention proposes an algorithm solution for automatic segmentation and splicing of POI videos. Through a given POI and scene description, video scene segmentation algorithm, similar video deduplication algorithm, similar image deduplication algorithm, video and image similarity algorithm, video face detection algorithm, video splicing and other technical means are used to generate video materials that can be used for direct delivery, which greatly reduces the cost of manual material search and editing and improves marketing efficiency.

[0119] Figure 4 Schematic diagram of the structure of the system for generating the point of interest display video of the present invention. Figure 4 As shown, an embodiment of the present invention further provides a system for generating a point of interest display video, which is used to implement the above-mentioned method for generating a point of interest display video. The system 5 for generating a point of interest display video includes:

[0120] The sub-video collection module 51 performs scene segmentation on the interest point video to obtain a number of interest point sub-videos, establishes a sub-video collection of the interest point sub-videos and performs video deduplication.

[0121] The reference picture collection module 52 establishes a picture collection containing a large number of pictures based on the same interest point, classifies each picture collection according to a preset scene, establishes a picture sub-collection for each preset scene, obtains several pictures with the lowest similarity to other pictures in the picture sub-collection as reference pictures, and establishes a reference picture collection.

[0122] The similarity calculation module 53 performs similarity calculation on the interest point sub-video after deduplication and each reference picture, so as to match the preset scene corresponding to the reference picture to each interest point sub-video.

[0123] The display video generation module 54 generates an interest point display video by splicing the interest point sub-videos according to the order of a preset scene sequence.

[0124] In a preferred embodiment, the sub-video set module 51 is configured to establish a video set containing a large number of interest point videos based on an interest point, perform scene segmentation on each interest point video to obtain a number of interest point sub-videos, establish a sub-video set of all interest point sub-videos, and perform video deduplication in the sub-video set.

[0125] In a preferred embodiment, the reference picture set module 52 is configured to establish a picture set containing a large number of pictures based on the same point of interest, and classify each picture set according to a preset scene. For each preset scene, a picture subset is established, and several pictures with the lowest similarity to other pictures in the picture subset are obtained as reference pictures, and the feature vector of each picture in the picture subset is obtained through the Clip model, and then the cosine similarity is used to calculate the similarity between different pictures. For several pictures with high similarity, only one is retained as a reference picture.

[0126] In a preferred embodiment, the similarity calculation module 53 is configured to calculate the similarity between the deduplicated interest point sub-video and each reference picture, obtain the reference picture with the highest similarity for each interest point sub-video, and use the preset scene of the reference picture as the label of the interest point sub-video.

[0127] In a preferred embodiment, the display video generation module 54 is configured to select at least one interest point sub-video as a spare sub-video from the interest point sub-videos with a unified label based at least on a preset video duration range and video quality, and to splice the spare sub-videos according to a preset sequence of scenes to generate an interest point display video.

[0128] In a preferred embodiment, the display video generation module 54 is also configured to preset a sequence of scenes, namely, hotel exterior, hotel lobby, hotel room, hotel restaurant, hotel gymnasium and hotel swimming pool.

[0129] In a preferred embodiment, it also includes: a portrait deletion module 55, which performs face recognition on the interest point display video and deletes the video frames with portraits.

[0130] The point of interest display video generation system of the present invention can automatically edit video materials that meet the current marketing scenario from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and improving the user viewing experience.

[0131] The embodiment of the present invention further provides a device for generating a point of interest display video, comprising a processor and a memory in which executable instructions of the processor are stored, wherein the processor is configured to execute the steps of the method for generating a point of interest display video by executing the executable instructions.

[0132] As shown above, the device for generating point of interest display videos of this embodiment of the present invention can automatically edit video materials that meet the current marketing scenarios from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and enhancing the user viewing experience.

[0133] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0134] Figure 5 Schematic diagram of the structure of the device for generating the point of interest display video of the present invention. Figure 5 The electronic device 600 according to this embodiment of the present invention is described. Figure 5 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0135] like Figure 5 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0136] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention described in the above method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0137] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0138] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0139] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0140] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0141] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, and the steps of the method for generating a point of interest display video are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above method section of this specification.

[0142] As shown above, the system for generating point of interest display videos of this embodiment of the present invention can automatically edit video materials that meet the current marketing scenarios from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and enhancing the user viewing experience.

[0143] Figure 6 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Figure 6As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0144] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0145] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0146] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0147] In summary, the purpose of the present invention is to provide a method, system, device and storage medium for generating point of interest display videos, which can automatically edit video materials that meet the current marketing scenarios from existing video materials, thereby significantly reducing the cost of video marketing, providing different perspectives to display more specific and detailed hotel scenes and enhance the user viewing experience.

[0148] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for generating a point of interest display video, characterized in that: The following steps are involved: S110, performing scene segmentation on the interest point video to obtain a plurality of interest point sub-videos, establishing a sub-video set of the interest point sub-videos and performing video deduplication; S120, establishing a picture set including a large number of pictures based on the same interest point, classifying each picture set according to a preset scene, establishing a picture sub-set for each preset scene, obtaining a number of pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establishing a reference picture set; S130, calculating similarity between the deduplicated interest point sub-video and each of the reference pictures, so as to match the preset scene corresponding to the reference picture to each of the interest point sub-videos; S140: splicing the interest point sub-videos according to a preset scene sequence order to generate an interest point display video.

2. The method for generating a point of interest display video according to claim 1, characterized in that: The step S110 includes the following steps: S111, establishing a video collection including a large number of videos of a point of interest based on a point of interest; S112, performing scene segmentation on each interest point video to obtain a plurality of interest point sub-videos; S113, establishing a sub-video set of all the interest point sub-videos; S114: Deduplication of videos in the sub-video set.

3. The method for generating a point of interest display video according to claim 1, characterized in that: The step S120 includes the following steps: S121, establishing a picture collection containing a large number of pictures based on the same point of interest, and classifying each picture collection according to a preset scene; S122. For each preset scene, a picture subset is established. Several pictures with the lowest similarity to other pictures in the picture subset are obtained as reference pictures. The feature vector of each picture in the picture subset is obtained through the Clip model. Then, the similarity between different pictures is calculated using the cosine similarity. For several pictures with high similarity, only one of them is retained as the reference picture.

4. The method for generating a point of interest display video according to claim 1, characterized in that: The step S130 includes the following steps: S131, calculating the similarity between the interest point sub-video after deduplication and each of the reference pictures; S132: Obtain a reference picture with the highest similarity for each interest point sub-video, and use a preset scene of the reference picture as a label of the interest point sub-video.

5. The method for generating a point of interest display video as claimed in claim 4, characterized in that: The step S140 includes the following steps: S141, selecting at least one interest point sub-video as a backup sub-video from the interest point sub-videos with a unified label based at least on a preset video duration range and video quality; S142: splicing the spare sub-videos according to a preset scene sequence to generate an interest point display video.

6. The method for generating a point of interest display video according to claim 1, characterized in that: In step S142, the preset scene sequence is hotel exterior, hotel lobby, hotel room, hotel restaurant, hotel gym and hotel swimming pool.

7. The method for generating a point of interest display video according to claim 1, characterized in that: After step S140, the following steps are also included: S150: Perform face recognition on the interest point display video, and delete video frames containing portraits.

8. A system for generating a point of interest display video, used to implement the method for generating a point of interest display video according to claim 1, characterized in that: include: A sub-video collection module, which performs scene segmentation on the interest point video to obtain a number of interest point sub-videos, establishes a sub-video collection of the interest point sub-videos and performs video deduplication; A reference picture set module is used to establish a picture set containing a large number of pictures based on the same point of interest, classify each picture set according to a preset scene, establish a picture sub-set for each preset scene, obtain several pictures with the lowest similarity to other pictures in the picture sub-set as reference pictures, and establish a reference picture set; A similarity calculation module is used to calculate the similarity between the interest point sub-video after deduplication and each of the reference pictures, so as to match the preset scene corresponding to the reference picture to each of the interest point sub-videos; The display video generation module splices the interest point sub-videos according to the order of a preset scene sequence to generate an interest point display video.

9. A device for generating a point of interest display video, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the method for generating a point of interest display video according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the method for generating a point of interest display video as described in any one of claims 1 to 7 are implemented.

Images