Image display method and apparatus therefor

Abstract

The application discloses an image display method and device, and belongs to the technical field of image processing. The method comprises the following steps: in a plurality of videos comprising the same scene, a reference video is determined, and a video segment in a non-reference video which is aligned with the scene of the reference video; all image frames in the reference video and the video segment aligned with the scene are extracted according to a first time unit in time sequence, so as to obtain a plurality of image groups, wherein the image frames in the same time sequence are in the same image group; and the images in the plurality of image groups which match a display strategy are displayed.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image display method and apparatus. Background Technology

[0002] With the development of technologies such as smart cameras, drones, and mobile devices, the video data generated by people recording and sharing various experiences in life, work, and travel is experiencing explosive growth. Therefore, based on the massive amount of video data generated, effective processing and analysis are needed to continuously improve its quality and meet user needs.

[0003] Currently, there is a lack of effective means for analyzing and processing video data, making it difficult to accurately match information from multiple videos. For example, factors such as complex shooting scenes, numerous moving objects in the scenes, different resolutions, styles, and colors in different videos make it challenging to match and display key images across multiple videos during data analysis and processing. Summary of the Invention

[0004] The purpose of this application is to provide an image display method that can solve the problem in the prior art that key images in multiple videos cannot be accurately matched and displayed.

[0005] In a first aspect, embodiments of this application provide an image display method, which includes: determining a reference video and a video segment in a non-reference video that is aligned with the scene of the reference video from among multiple videos including the same scene; extracting all image frames from the reference video and the scene-aligned video segment in chronological order according to a first time unit to obtain multiple image groups, wherein image frames in the same chronological order are in the same image group; and displaying images in the multiple image groups that match a display strategy.

[0006] Secondly, embodiments of this application provide an image display device, comprising: a determining module, configured to determine a reference video and a video segment in a non-reference video aligned with the scene of the reference video from among multiple videos including the same scene; an extraction module, configured to extract all image frames from the reference video and the scene-aligned video segment in chronological order according to a first time unit, to obtain multiple image groups, wherein image frames in the same chronological order are in the same image group; and a display module, configured to display images in the multiple image groups that match a display strategy.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In the embodiments of this application, multiple videos captured in the same scene are used as the processing objects. First, a reference video is determined, and video segments aligned with the scene of the reference video are found in other videos (i.e., non-reference videos). After alignment, all image frames in the reference video and the scene-aligned video segments are extracted in chronological order according to a first time unit. Image frames with the same chronological order are in the same image group, thus obtaining multiple image groups. Finally, images from the multiple image groups that match the display strategy are displayed. Based on the embodiments of this application, images matching the information of multiple videos including the same scene can be extracted and displayed, thereby improving the processing and analysis capabilities of video data. Attached Figure Description

[0012] Figure 1 This is one of the flowcharts of the image display method according to an embodiment of this application;

[0013] Figure 2 This is a second flowchart of the image display method according to an embodiment of this application;

[0014] Figure 3 This is the third flowchart of the image display method according to an embodiment of this application;

[0015] Figure 4 This is the fourth flowchart of the image display method according to an embodiment of this application;

[0016] Figure 5 This is the fifth flowchart of the image display method according to an embodiment of this application;

[0017] Figure 6 This is the sixth flowchart of the image display method according to an embodiment of this application;

[0018] Figure 7 This is a block diagram of an image display device according to an embodiment of this application;

[0019] Figure 8 This is one of the hardware structure diagrams of the electronic device according to an embodiment of this application;

[0020] Figure 9 This is the second schematic diagram of the hardware structure of the electronic device according to an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0022] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0023] The image display method provided in this application embodiment can be executed by the image display device provided in this application embodiment, or by an electronic device integrating the image display device, wherein the image display device can be implemented in hardware or software.

[0024] The image display method provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0025] Figure 1 A flowchart illustrating an embodiment of an image display method of this application is shown, applied to an electronic device, the method comprising:

[0026] S110: Among multiple videos containing the same scene, identify the reference video and the video segments in the non-reference videos that are aligned with the scene of the reference video.

[0027] In this embodiment, based on the same shooting environment, different devices shoot multiple videos from different angles, and further, image frames with the same or similar content in the multiple videos are matched.

[0028] In this step, a reference video is identified, and based on the reference video, video segments that align with the scene in the reference video are found in non-reference videos other than the reference video.

[0029] For example, the reference video includes the sunrise process, and based on this step, the corresponding segment of the sunrise process is found in all the videos.

[0030] S120: Extract all image frames from the reference video and the scene-aligned video clip in chronological order according to the first time unit to obtain multiple image groups, where image frames with the same time order are in the same image group.

[0031] In this step, a scene-aligned reference video and a video clip are obtained. After alignment, the reference video and the video clip have the same duration. Therefore, based on the first time unit as the standard, all image frames in the reference video and the scene-aligned video clip are extracted in chronological order to obtain multiple image groups.

[0032] In this embodiment, after scene alignment, starting from the same position, at least one image frame is extracted from the reference video and multiple video segments every second to form an image group. For example, every second, five image frames are extracted from each video segment in the reference video and multiple video segments to form an image group; alternatively, every second, one image frame is extracted from each video segment in the reference video and multiple video segments to form an image group. Specifically, the duration of the first time unit and the number of frames extracted each time can be selected according to actual needs, and will not be elaborated here.

[0033] S130: Display the image that matches the display strategy from multiple image groups.

[0034] In this embodiment, after determining multiple image groups, not every image in each image group needs to be displayed; instead, images matching the display strategy are displayed. In this embodiment, the display strategy involves matching key information from multiple image groups with each of the multiple videos. This key information includes, but is not limited to, people, scenery, and buildings.

[0035] For example, see Figure 2 After scene alignment of videos A, B, and C and extraction of multiple image groups, images A-1, A-2, and A-3 that match the key information of video A are displayed; images B-1, B-2, and B-3 that match the key information of video B are displayed; and images C-1, C-2, and C-3 that match the key information of video C are displayed.

[0036] In the embodiments of this application, multiple videos captured in the same scene are used as the processing objects. First, a reference video is determined, and video segments aligned with the scene of the reference video are found in other videos (i.e., non-reference videos). After alignment, all image frames in the reference video and the scene-aligned video segments are extracted in chronological order according to a first time unit. Image frames with the same chronological order are in the same image group, thus obtaining multiple image groups. Finally, images from the multiple image groups that match the display strategy are displayed. Based on the embodiments of this application, images matching the information of multiple videos including the same scene can be extracted and displayed, thereby improving the processing and analysis capabilities of video data.

[0037] like Figure 3 As shown, in the flow of the image display method of another embodiment of this application, S110 includes:

[0038] S1101: Select the shortest video among multiple videos as the reference video.

[0039] In this step, based on the duration of multiple videos, the shortest video is selected as the reference video.

[0040] S1102: Using the first video as the reference video for scene alignment, scene alignment is performed on the other videos in the multiple videos except for the reference video in turn to obtain multiple video segments with scene alignment. The first video is the video obtained by deleting the first N seconds of image frames and the last N seconds of image frames of the reference video, where N is a positive integer.

[0041] In this embodiment, scene alignment of multiple videos requires selecting a reference video for aligning the others. To ensure the robustness of the alignment algorithm, an adaptive scale reference video selection strategy is designed, as exemplified below:

[0042] First, select the shortest video as the reference video. Then, based on the reference video's duration, select consecutive frames spanning multiple seconds as the baseline video. For reference, subtract 6 seconds from the reference video's duration (3 seconds before and after), and use the resulting value as the baseline video's duration. Here, N is set to 3. For example, if the reference video is 32 seconds long, select 32-6=26 consecutive seconds of video as the baseline video. When the reference video is very short, to ensure sufficient coverage for the baseline video, the value of N can be adjusted to ensure the baseline video's duration is at least 5 seconds.

[0043] In this embodiment, the length of the reference video varies according to the different durations of the reference videos; the longer the reference video, the longer the reference video. Furthermore, the reference video can cover multiple scenes within the reference video. Therefore, the reference video selection method provided in this embodiment can ensure that scene-aligned video segments are found among other videos.

[0044] like Figure 4 As shown, in the flow of the image display method of another embodiment of this application, S1102 includes:

[0045] S11021: Using the first video as the reference video for sliding window alignment, slide the first video for each of the other videos.

[0046] Optionally, for the first video, extract, for example, 5 image frames per second based on the timestamp information; for other videos, extract the same number of frames per second, for example, 5 image frames per second based on the timestamp information.

[0047] The more image frames extracted per second, the more precise the matching. Additionally, extracting video image frames at fixed time intervals, such as 5 image frames per second as mentioned above, can increase the proportion of static frames while maintaining the speed of subsequent algorithms.

[0048] S11022: Every M seconds when the first video slides, calculate the sum of similarities between all image frames in the first video and the corresponding image frames in the video to be aligned, where N / M is an integer.

[0049] In this step, the first video serves as a sliding window ruler, which can be used to slide and align other videos.

[0050] For ease of understanding, all extracted image frames from the videos are arranged sequentially in chronological order, with one video corresponding to one ruler, and each mark on the ruler representing one frame. Furthermore, the sliding ruler is aligned with the ruler of the video to be aligned; after one alignment, the sliding ruler is moved to the next mark.

[0051] Each slide represents a single tick (one frame). Based on the number of frames Q extracted per second, the slide corresponds to 1 / Q seconds, and M = 1 / Q.

[0052] For example, if each of the multiple videos extracts 5 image frames per second, meaning 1 second includes 5 image frames, then each frame slide corresponds to a 0.2-second slide.

[0053] In this step, every M seconds that the first video slides, the sum of the similarities between all the image frames in the first video and the corresponding image frames in the video to be aligned is calculated.

[0054] For example, if the first video is 26 seconds long and has 5 image frames per second, then there are 26 × 5 image frames. The 130 image frames in the first video are paired with the 130 image frames at corresponding positions in other videos (videos to be aligned) to calculate the similarity of 130 sets of image frames. Based on the similarity of the 130 sets of image frames, the total similarity is obtained.

[0055] Where N / M is an integer, to ensure that the first video can be aligned with any image frame of other videos at least once by sliding.

[0056] S11023: After the first video slides for 2N seconds, the position with the highest sum of similarity in the video to be aligned is determined as the video segment for scene alignment.

[0057] The first video slides for 2N seconds, meaning the sliding range is from -N seconds to N seconds. For example, it slides forward for N seconds and then backward for N seconds.

[0058] For example, if the reference video is 32 seconds long, then the first video selects frames that are 32-6=26 seconds long, which can be from the 4th second to the 29th second. The sliding range of the first video is -3 seconds to 3 seconds.

[0059] In this step, after the sliding is completed, the position with the highest sum of similarity is found.

[0060] The sum of similarities is used to represent the similarity between any video segment in the video to be aligned and the first video.

[0061] For example, the first video is slid for 2N seconds, and the number of slids is 2N / M. In 2N / M slids, the video segment with the largest sum of similarity in the video to be aligned is found.

[0062] Optionally, when the video segment with the highest sum of similarity is found in the video to be aligned, the alignment position is recorded, and the corresponding video segment in the video to be aligned is determined based on the alignment position.

[0063] For example, starting from the alignment position, identify a video segment with the same duration as the reference video.

[0064] In this embodiment, a sliding window alignment method is provided to find video segments that are aligned with a reference video scene in the video, thereby ensuring the accuracy and robustness of the alignment.

[0065] like Figure 5 As shown, in the flow of the image display method of another embodiment of this application, S11022 includes:

[0066] S110221: Every M seconds when the first video slides, the difference operator algorithm is used to calculate the first similarity between all image frames in the first video and the corresponding image frames in the video to be aligned.

[0067] In this embodiment, every M seconds, all image frames in the first video are aligned one by one with the corresponding image frames in the video to be aligned. For any two aligned image frames, the pixel grayscale difference is calculated.

[0068] The differential operator algorithm is used to iterate through each pixel in two image frames, calculate their grayscale difference (i.e., the difference between the grayscale values ​​of the two pixels), and then accumulate these grayscale differences to obtain the difference value. The accumulation process can be described by the following formula (1):

[0069] S = ∑(|I(x,y) - J(x,y)|) (1)

[0070] In formula (1), S represents the difference between two image frames, I and J represent the pixel values ​​in the two image frames, and x and y represent the pixel coordinates. The smaller the difference, the more similar the two image frames are; conversely, the larger the difference, the greater the difference between the two image frames.

[0071] Furthermore, the difference values ​​are normalized to between 1 and 0 to obtain the first similarity, as shown in formula (2) below:

[0072] First similarity = 1 - S / (N1 * 255) (2)

[0073] In formula (2), S represents the difference between two image frames, N1 is the total number of pixels in the two image frames, and 255 is the range of pixel values ​​(assuming an 8-bit grayscale image). This formula calculates the similarity score between two image frames under the influence of the total number of pixels and the range of pixel values. The smaller the difference, the higher the first similarity; when they are completely identical, the first similarity is 1.

[0074] S110222: Sum the first similarity scores to obtain the total similarity score.

[0075] In this step, the number of aligned groups is: the duration of the base video × the number of frames extracted per second, so that the number of first similarities is the same as the number of aligned groups. These first similarities are accumulated to obtain the total similarity.

[0076] In this embodiment, a method for calculating the sum of similarities is provided, wherein a first similarity between two image frames is calculated using a difference operator, and the accuracy of the matching result can be ensured by ensuring the accuracy of the first similarity.

[0077] like Figure 6 As shown, in the flow of the image display method of another embodiment of this application, S130 includes:

[0078] S1301: Calculate the second similarity of each image group in multiple image groups.

[0079] Optionally, for any image group, a second similarity is calculated between the images in the image group.

[0080] Optionally, a similarity calculation method based on deep learning operators can be designed. This method leverages the ability of deep learning models to extract multi-dimensional features, fully exploring the semantic feature information in the two image frames to be compared. The similarity is calculated based on the differences between the two sets of semantic feature information, improving the robustness of the image similarity algorithm for various types of captured content. (For reference:)

[0081] A similarity calculation method based on deep learning operators compares the similarity between two image frames, using a pre-trained PyTorch Deep Residual Network (ResNet50) neural network model as the image feature extractor. This model can quickly extract feature information from two image frames. ResNet50 extracts a 1*1000 feature vector, where 1000 elements represent different feature information.

[0082] Furthermore, the Euclidean distance between the feature vectors of the two image frames is calculated to measure the similarity between the two image frames. The formula for calculating the Euclidean distance is shown in formula (3):

[0083] Euclidean distance = sqrt(sum(pow(AB,2))) (3)

[0084] In formula (3), A and B represent feature vectors, pow() is a power function, and 2 is an exponent. This Euclidean distance is normalized to a range between 1 and 0, where 1 represents two identical image frames and 0 represents two completely different image frames. See formula (4):

[0085] Similarity = 1 / (1 + Euclidean distance) (4)

[0086] The similarity between two image frames can be calculated using formula (4). The closer the similarity is to 1, the higher the similarity between the two image frames; conversely, the lower the similarity is, the greater the difference. This method does not require building a training dataset; it only needs to use the existing ResNet50 model for feature extraction, thus enabling fast and accurate calculation of the similarity between two image frames.

[0087] Optionally, within an image group, the similarity between two image frames is calculated, and then the average of all the obtained similarities is taken to obtain a second similarity.

[0088] Optionally, within an image group, the similarity between two image frames is calculated, and then the minimum similarity among all similarities is identified as the second similarity.

[0089] S1302: Determine at least one first image group among a plurality of image groups whose second similarity is greater than a first threshold.

[0090] In this step, image groups with a second similarity less than or equal to the first threshold are filtered out, while image groups with a second similarity greater than the first threshold are retained, i.e., at least one first image group in the embodiments of this application.

[0091] For example, if the reference video is 12 seconds long, and one second is taken as the first time unit, then 12 image groups can be obtained. The second similarity of each of the 12 image groups is calculated, and finally the image groups with a second similarity greater than the first threshold are retained.

[0092] S1303: Display at least one image from the first image group that matches the display strategy.

[0093] In this embodiment of the application, the display strategy specifically involves matching key information from at least one first image group with each of the multiple videos. After determining the key information in the multiple videos, images that match the key information in the multiple videos are determined from the at least one first image group and displayed.

[0094] In summary, this application designs a sliding window video alignment method that significantly reduces the negative impact of rapidly changing scenes on multi-video alignment, adapting to the alignment of videos of different durations. Firstly, it designs a more accurate similarity calculation method based on deep learning feature extraction, improving the accuracy of keyframe matching across multiple videos and enabling matching of a wider variety of video content, such as forests, shopping malls, flowers, and people. Secondly, it designs an adaptive deduplication strategy for video content, effectively removing duplicate matching results and ensuring optimal matching outcomes. Thirdly, it designs the entire processing flow from video alignment to keyframe matching and result deduplication, significantly improving video processing efficiency and shortening processing time. Therefore, this application greatly reduces the impact of video content variation on matching results, thereby optimizing matching performance under various complex shooting scenarios.

[0095] The image display method provided in this application can be executed by an image display device. This application uses an image display device executing the image display method as an example to illustrate the image display device provided in this application.

[0096] Figure 7 A block diagram of an image display apparatus according to an embodiment of this application is shown. The apparatus includes:

[0097] The determination module 10 is used to determine, among multiple videos including the same scene, a reference video and a video segment in non-reference videos that is aligned with the scene of the reference video;

[0098] Extraction module 20 is used to extract all image frames in the reference video and scene-aligned video clip in chronological order according to the first time unit, and obtain multiple image groups, wherein image frames with the same time order are in the same image group;

[0099] Display module 30 is used to display images from multiple image groups that match the display strategy.

[0100] In the embodiments of this application, multiple videos captured in the same scene are used as the processing objects. First, a reference video is determined, and video segments aligned with the scene of the reference video are found in other videos (i.e., non-reference videos). After alignment, all image frames in the reference video and the scene-aligned video segments are extracted in chronological order according to a first time unit. Image frames with the same chronological order are in the same image group, thus obtaining multiple image groups. Finally, images from the multiple image groups that match the display strategy are displayed. Based on the embodiments of this application, images matching the information of multiple videos including the same scene can be extracted and displayed, thereby improving the processing and analysis capabilities of video data.

[0101] Optionally, module 10 is defined, including:

[0102] The first determining unit is used to determine the shortest video among multiple videos as the reference video;

[0103] The alignment unit is used to take the first video as the reference video for scene alignment, and sequentially perform scene alignment on the other videos in the multiple videos except the reference video to obtain multiple scene-aligned video segments. The first video is the video obtained by deleting the first N seconds of image frames and the last N seconds of image frames of the reference video, where N is a positive integer.

[0104] Optionally, the alignment unit includes:

[0105] A sliding subunit is used to slide the first video as the reference video for sliding window alignment, and for each of the other videos, slide the first video.

[0106] The calculation subunit is used to calculate the sum of similarity between all image frames in the first video and the corresponding image frames in the video to be aligned every M seconds of sliding in the first video, where N / M is an integer;

[0107] The sub-unit is determined to identify the video segment with the highest sum of similarity in the video to be aligned after the first video slides for 2N seconds.

[0108] Optionally, the sub-unit is calculated, specifically for:

[0109] Every M seconds of scrolling in the first video, the difference operator algorithm is used to calculate the first similarity between all image frames in the first video and the corresponding image frames in the video to be aligned.

[0110] The first similarity scores are summed to obtain the total similarity score.

[0111] Optionally, the display module 30 includes:

[0112] A computing unit is used to calculate the second similarity of each image group in multiple image groups;

[0113] The second determining unit is used to determine at least one first image group among a plurality of image groups whose second similarity is greater than a first threshold.

[0114] A display unit is used to display at least one image from a first image group that matches the display strategy.

[0115] The image display device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0116] The image display device in this application embodiment can be a device with a motion system. The motion system can be an Android motion system, an iOS motion system, or other possible motion systems, and this application embodiment does not specifically limit it.

[0117] The image display device provided in this application embodiment can implement the various processes implemented in the above method embodiments, and will not be described again here to avoid repetition.

[0118] Optionally, such as Figure 8 As shown, this application embodiment also provides an electronic device 100, including a processor 101, a memory 102, and a program or instructions stored in the memory 102 and executable on the processor 101. When the program or instructions are executed by the processor 101, they implement the various steps of any of the above-described image display method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0119] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0120] Figure 9 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0121] The electronic device 1000 includes, but is not limited to, the following components: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, processor 1010, camera 1011, etc.

[0122] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 9 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0123] The processor 1010 is configured to identify a reference video and a video segment in a non-reference video that is aligned with the scene of the reference video among multiple videos including the same scene; extract all image frames in the reference video and the scene-aligned video segment in chronological order according to a first time unit to obtain multiple image groups, wherein image frames in the same chronological order are in the same image group; and the display unit 1006 is configured to display images in the multiple image groups that match the display strategy.

[0124] In the embodiments of this application, multiple videos captured in the same scene are used as the processing objects. First, a reference video is determined, and video segments aligned with the scene of the reference video are found in other videos (i.e., non-reference videos). After alignment, all image frames in the reference video and the scene-aligned video segments are extracted in chronological order according to a first time unit. Image frames with the same chronological order are in the same image group, thus obtaining multiple image groups. Finally, images from the multiple image groups that match the display strategy are displayed. Based on the embodiments of this application, images matching the information of multiple videos including the same scene can be extracted and displayed, thereby improving the processing and analysis capabilities of video data.

[0125] Optionally, the processor 1010 is further configured to determine the shortest video among the plurality of videos as the reference video; using the first video as the reference video for scene alignment, and sequentially performing scene alignment on the other videos among the plurality of videos except the reference video to obtain a plurality of scene-aligned video segments, wherein the first video is a video obtained by deleting the first N seconds of image frames and the last N seconds of image frames of the reference video, where N is a positive integer.

[0126] Optionally, the processor 1010 is further configured to use the first video as a reference video for sliding window alignment, and slide the first video for each of the other videos; calculate the sum of similarities between all image frames in the first video and the corresponding image frames in the video to be aligned every M seconds, where N / M is an integer; and determine the position with the largest sum of similarities in the video to be aligned as the video segment for scene alignment after the first video has slid for 2N seconds.

[0127] Optionally, the processor 1010 is further configured to, every M seconds of sliding in the first video, use a difference operator algorithm to calculate the first similarity between all image frames in the first video and the corresponding image frames in the video to be aligned; and accumulate the first similarity to obtain the sum of the similarities.

[0128] Optionally, the processor 1010 is further configured to calculate a second similarity for each of the plurality of image groups; determine at least one first image group among the plurality of image groups whose second similarity is greater than a first threshold; and the display unit 1006 is further configured to display an image in the at least one first image group that matches the display strategy.

[0129] In summary, this application designs a sliding window video alignment method that significantly reduces the negative impact of rapidly changing scenes on multi-video alignment, adapting to the alignment of videos of different durations. Firstly, it designs a more accurate similarity calculation method based on deep learning feature extraction, improving the accuracy of keyframe matching across multiple videos and enabling matching of a wider variety of video content, such as forests, shopping malls, flowers, and people. Secondly, it designs an adaptive deduplication strategy for video content, effectively removing duplicate matching results and ensuring optimal matching outcomes. Thirdly, it designs the entire processing flow from video alignment to keyframe matching and result deduplication, significantly improving video processing efficiency and shortening processing time by more than 30% compared to existing algorithms. Therefore, this application greatly reduces the impact of video content variation on matching results, thereby optimizing matching performance under various complex shooting scenarios.

[0130] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or video images obtained by an image capture device (such as a camera) in video image capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here. The memory 1009 can be used to store software programs and various data, including but not limited to applications and motion systems. Processor 1010 may integrate an application processor and a modem processor. The application processor mainly handles the action system, user page, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may also not be integrated into processor 1010.

[0131] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0132] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0133] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image display method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0134] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0135] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described image display method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0136] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0137] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image display method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of this application.

[0140] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image display method, characterized in that, The method includes: Among multiple videos containing the same scene, a reference video is identified, as well as video segments in non-reference videos that are aligned with the scene of the reference video; Based on the first time unit, all image frames in the reference video and the scene-aligned video segment are extracted in chronological order to obtain multiple image groups, wherein image frames with the same chronological order are in the same image group; Displaying images from the plurality of image groups that match a display strategy, wherein the display strategy is to display images from the plurality of image groups that match key information of each of the plurality of videos.

2. The method according to claim 1, characterized in that, The determination of the reference video, and the video segments in the non-reference videos that are aligned with the scene of the reference video, include: The video with the shortest duration among the plurality of videos is determined as the reference video; Using the first video as the reference video for scene alignment, scene alignment is performed sequentially on the other videos in the plurality of videos except for the reference video to obtain a plurality of scene-aligned video segments. The first video is a video obtained by deleting the first N seconds of image frames and the last N seconds of image frames of the reference video, where N is a positive integer.

3. The method according to claim 2, characterized in that, The process involves using the first video as the reference video for alignment, and then sequentially performing scene alignment on the other videos (excluding the reference video) among the plurality of videos to obtain multiple scene-aligned video segments, including: Using the first video as the reference video for sliding window alignment, slide the first video for each of the other videos; Every M seconds of scrolling in the first video, the sum of similarity between all image frames in the first video and the corresponding image frames in the video to be aligned is calculated, where N / M is an integer; After the first video has been scrolled for 2N seconds, the position with the highest sum of similarity in the video to be aligned is determined as the video segment for scene alignment.

4. The method according to claim 3, characterized in that, The step of calculating the sum of similarities between all image frames in the first video and corresponding image frames in the video to be aligned every M seconds of scrolling in the first video includes: Every M seconds of scrolling in the first video, the difference operator algorithm is used to calculate the first similarity between all image frames in the first video and the corresponding image frames in the video to be aligned. The first similarity scores are summed to obtain the total similarity score.

5. The method according to claim 1 or 2, characterized in that, Displaying the image that matches the display strategy from the plurality of image groups includes: Calculate a second similarity for each of the plurality of image groups, wherein the second similarity is the similarity between all image frames within each image group; Determine at least one first image group among the plurality of image groups whose second similarity is greater than a first threshold; Display the image in the at least one first image group that matches the display strategy.

6. An image display device, characterized in that, The device includes: The determination module is used to determine, among multiple videos including the same scene, a reference video and a video segment in a non-reference video that is aligned with the scene of the reference video; The extraction module is used to extract all image frames from the reference video and the scene-aligned video segment in chronological order according to a first time unit, to obtain multiple image groups, wherein image frames with the same chronological order are in the same image group; The display module is used to display images from the plurality of image groups that match the display strategy, wherein the display strategy is to display images from the plurality of image groups that match the key information of each of the plurality of videos.

7. The apparatus according to claim 6, characterized in that, The determining module includes: The first determining unit is used to determine the video with the shortest duration among the plurality of videos as the reference video; An alignment unit is used to take the first video as the reference video for scene alignment, and sequentially perform scene alignment on the other videos in the plurality of videos except the reference video to obtain a plurality of scene-aligned video segments. The first video is a video obtained by deleting the first N seconds of image frames and the last N seconds of image frames of the reference video, where N is a positive integer.

8. The apparatus according to claim 7, characterized in that, The alignment unit includes: A sliding subunit is used to slide the first video as a reference video for sliding window alignment for each of the other videos; The calculation subunit is used to calculate the sum of similarities between all image frames in the first video and the corresponding image frames in the video to be aligned every M seconds of sliding in the first video, where N / M is an integer; The determination subunit is used to determine the position with the largest sum of similarity in the video to be aligned as the video segment for scene alignment after the first video slides for 2N seconds.

9. The apparatus according to claim 8, characterized in that, The computational subunit is specifically used for: Every M seconds of scrolling in the first video, the difference operator algorithm is used to calculate the first similarity between all image frames in the first video and the corresponding image frames in the video to be aligned. The first similarity scores are summed to obtain the total similarity score.

10. The apparatus according to claim 6 or 7, characterized in that, The display module includes: A calculation unit is used to calculate a second similarity for each of the plurality of image groups, wherein the second similarity is the similarity between all image frames within each image group; The second determining unit is used to determine at least one first image group among the plurality of image groups in which the second similarity is greater than the first threshold; A display unit is configured to display an image from the at least one first image group that matches the display strategy.

Images