Method, apparatus and electronic device for determining image frame

Abstract

This disclosure provides a method, apparatus, and electronic device for determining image frames, relating to the field of image processing, specifically the field of image frame processing, and particularly to a method for determining image frames. The specific implementation scheme is as follows: In response to receiving a page jump instruction, a page jump video of the jump from the current page to the target page is acquired; the page jump video is divided into frames to obtain multiple image frames; the page jump stage to which the multiple image frames belong is determined based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage have the same type; and the target image frame belonging to a different page jump stage is located among the multiple image frames.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, specifically to the field of image frame processing, and particularly to a method for determining image frames. Background Technology

[0002] Currently, when identifying the first and last frames of a video during page transitions, traditional image processing algorithms can be used. However, the current image processing algorithm relies on manual parameter adjustments during its development. A single scenario may encounter multiple different processes, and each process variation requires process control and adaptation of the image processing algorithm, making the process quite complex. Summary of the Invention

[0003] This disclosure provides a method, apparatus, and electronic device for determining image frames, to at least solve the technical problem of low efficiency in locating target image frames in related technologies.

[0004] According to one aspect of this disclosure, a method for determining image frames is provided, comprising: in response to receiving a page jump instruction, acquiring a page jump video of a jump from the current page to a target page; dividing the page jump video into frames to obtain multiple image frames; determining the page jump stage to which the multiple image frames belong based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage are of the same type; and locating the target image frame belonging to a different page jump stage among the multiple image frames.

[0005] According to another aspect of this disclosure, an image frame determination apparatus is provided, comprising: a first acquisition module, configured to acquire a page jump video from the current page to a target page in response to receiving a page jump instruction; a frame segmentation module, configured to segment the page jump video into multiple image frames; a first determination module, configured to determine the page jump stage to which the multiple image frames belong based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage are of the same type; and a positioning module, configured to locate target image frames belonging to different page jump stages among the multiple image frames.

[0006] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the above embodiments.

[0007] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to perform any of the methods described above.

[0008] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0009] In this disclosure, in response to receiving a page jump instruction, a page jump video of the jump from the current page to the target page is first obtained; the page jump video is segmented into frames to obtain multiple image frames; the page jump stage to which the multiple image frames belong is determined based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage are of the same type; and the target image frames belonging to different page jump stages among the multiple image frames are located, thereby improving the efficiency of determining the target image frames. Since multiple image frames can be assigned to different page jump stages based on the similarity between them, and image frames belonging to the same page jump stage are of the same type, the required image frames can be efficiently located through the page jump stage, thereby at least solving the technical problem of low efficiency in locating target image frames in related technologies.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1a This is a flowchart of a method for determining an image frame according to an embodiment of the present disclosure;

[0013] Figure 1b This is a schematic diagram of an interactive interface according to an embodiment of the present disclosure;

[0014] Figure 2 This is a schematic diagram of multiple image frames in a page-jumping video according to an embodiment of the present disclosure;

[0015] Figure 3 This is a schematic diagram of image frame switching according to an embodiment of the present disclosure;

[0016] Figure 4 This is a schematic diagram of the feature representation of an image frame according to an embodiment of the present disclosure;

[0017] Figure 5 This is a schematic diagram of a category label mapping method according to an embodiment of the present disclosure;

[0018] Figure 6 This is a schematic diagram of another category label mapping method according to an embodiment of the present disclosure;

[0019] Figure 7 This is a schematic diagram of a linear mapping layer according to an embodiment of this application;

[0020] Figure 8 This is a flowchart of another method for determining an image frame according to an embodiment of the present disclosure;

[0021] Figure 9 This is a flowchart of another method for determining an image frame according to an embodiment of the present disclosure;

[0022] Figure 10 This is a schematic diagram of an image frame determination device according to an embodiment of the present disclosure;

[0023] Figure 11 This is a schematic block diagram of an electronic device 1100 according to an embodiment of the present disclosure. Detailed Implementation

[0024] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0026] According to embodiments of this disclosure, a method for determining image frames is provided. It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowcharts, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] The method embodiments provided in this disclosure can be executed in a mobile terminal, computer terminal, or similar electronic device.

[0028] For example, the image frame determination method provided in this disclosure can be applied to performance testing scenarios of page navigation in handheld devices. When the interactive interface of the handheld device receives a page navigation command, the video / image currently displayed on the interactive interface is triggered to navigate to the target page. The target page then displays multiple image frames, and finally, the target image frame belonging to different page navigation stages is located. This provides a technical means to improve the positioning efficiency of page navigation functions in handheld devices. The key to this technical solution is that multiple image frames can be assigned to different page navigation stages based on their similarity. Moreover, since image frames belonging to the same page navigation stage are of the same type, the technical solution provided by this invention enables efficient positioning of the required image frames at different page navigation stages, thereby improving the performance of page navigation testing.

[0029] Based on the above feasible scenario, the various optional schemes of the image frame determination method provided in this disclosure are described in detail below:

[0030] This disclosure provides, for example Figure 1a The method for determining an image frame is shown. Figure 1a This is a flowchart of a method for determining an image frame according to an embodiment of this disclosure, such as... Figure 1a As shown, one possible solution may include the following steps:

[0031] Step S102: In response to receiving a page jump instruction, obtain the page jump video from the current page to the target page.

[0032] The page navigation instructions described above can be generated based on the user's touch actions on the current page. These touch actions can be touching a control on the current page that allows navigation to the target page.

[0033] The page redirection instructions mentioned above can be generated based on a test task, which primarily tests the page redirection speed from the current page to the target page. There are no specific limitations on how the page redirection instructions are generated; they can be generated according to the actual situation.

[0034] The "current page" mentioned above can be any browsing page, including web pages, application pages, and game pages; there are no restrictions on the type of the current page. The "target page" mentioned above can be the page to which the user is to be redirected; the target page can also be a web page, application page, or game page; there are no restrictions on the type of the target page.

[0035] It should be noted that the current page and the target page can be of different types. For example, it is possible to jump from the current page of the application to the target page of the webpage. This is only an example and is not a specific limitation.

[0036] The page transition video described above is used to represent a transition video between different displayed pages. For example, it could be a page transition video from the current page to a target page. The page transition video may include a portion of image frames before the transition and a portion of image frames after the transition.

[0037] The aforementioned multiple image frames can be arranged in chronological order of the jump. Each image frame contains a first frame and a last frame. The first frame can be the last image frame of the current page at the moment before the jump in the page jump video; the last frame can be the first image frame of the target page at the moment after the jump ends in the page jump video.

[0038] It should be noted that the display page here refers to the page displayed on the electronic device. The display page can be a web page, an application list page, a video page, an image page, etc. There is no limitation on the type of page to be displayed here.

[0039] Step S104: Divide the page jump video into frames to obtain multiple image frames.

[0040] In one optional embodiment, the page jump video can be divided into frames at a preset frequency to obtain multiple image frames. For example, the page jump video can be divided into frames at 60fps (frame rate) to obtain multiple image frames. The frame rate is not specifically limited here.

[0041] In another alternative embodiment, the frame rate of the frames can be determined based on the needs of the test.

[0042] The aforementioned image frames can be sorted according to the time sequence of page jumps.

[0043] Step S106: Determine the page jump stage to which multiple image frames belong based on the similarity between multiple image frames.

[0044] Among them, the image frames belonging to the same page jump stage are of the same type.

[0045] The page transition process described above can be roughly divided into three stages: the pre-transition stage, the transition stage, and the post-transition stage. Furthermore, based on the type of image frames used during the transition, the transition stage can be further divided into several intermediate transition phases.

[0046] In one alternative embodiment, image frames with high similarity can be assigned to the same page transition stage, while image frames with low similarity can be assigned to different page transition stages.

[0047] In one optional embodiment, the first image frame among multiple image frames can be used as the first reference image frame. By judging the similarity between the other image frames and the first reference image frame, the page transition stage to which the multiple image frames belong can be determined. Optionally, the similarity between the multiple image frames and the first reference image frame can be judged sequentially. If the similarity between an image frame and the first reference image frame is greater than a preset threshold, it can be determined that the image frame and the first reference image frame belong to the same transition stage, that is, the stage before the page transition. If the similarity between an image frame and the first reference image frame is less than or equal to the preset threshold, it can be determined that the image frame and the first reference image frame do not belong to the same transition stage. In this case, the image frame can be determined as the second reference image frame for the stage in the page transition.

[0048] Furthermore, the similarity between the second reference image and subsequent image frames can be determined. If the similarity between an image frame and the second reference image is less than or equal to a preset threshold, it can be determined that the image frame and the second reference image frame do not belong to the same jump stage. In this case, the image frame can be determined as the third reference image for the next jump stage.

[0049] Furthermore, the similarity between the third reference image and subsequent image frames can be determined. If the similarity between an image frame and the third reference image is greater than a preset threshold, it indicates that the next jump stage is the stage after the page jump. Thus, it is possible to determine the page jump stage to which multiple image frames belong based on the similarity between multiple image frames.

[0050] It should be noted that the above embodiments are only illustrative examples. The specific number of page jump stages and the method of determining the page jump stage to which multiple image frames belong based on the similarity between multiple image frames can be determined according to the actual situation.

[0051] Step S106: Locate the target image frame that belongs to a different page transition stage among multiple image frames.

[0052] The target image frame mentioned above can be the first and last frame of multiple image frames. The first frame can be the image frame corresponding to the start of the page jump video, and the last frame can be the image frame corresponding to the end of the page jump video. The target image frame mentioned above can also be an image frame obtained according to user needs.

[0053] Figure 1b This is a schematic diagram of an interactive interface according to an embodiment of the present disclosure, such as... Figure 1bAs shown, the current page can first be displayed on the interactive interface. Then, users can click the start jump control to jump from the current page to the target page. After the jump is completed, a page jump video can be generated based on the jump process between the current page and the target page and displayed on the interactive interface.

[0054] Figure 2 This is a schematic diagram of multiple image frames in a page-jumping video according to an embodiment of the present disclosure, such as... Figure 2 As shown, the page transition video can contain four page transition stages. Image frames 1 and 2 can be the first transition stage before the page transition, image frames 3 and 4 can be the second transition stage during the page transition, image frames 5 and 6 can be the third transition stage during the page transition, and image frames 7 and 8 can be the fourth transition stage after the page transition. Among them, image frame 2 is the first frame and image frame 7 is the last frame.

[0055] In one optional embodiment, since the page jump stage can divide different types of image frames, the first and last frames among multiple image frames can be determined more efficiently based on the page jump stage. Optionally, the last image frame in the page jump stage before the page jump can be taken as the first frame, and the first image frame in the page jump stage after the page jump can be taken as the last frame.

[0056] In another alternative embodiment, after determining the first and last frames, the performance of page jump can be tested using the first and last frames. Since determining the first and last frames is relatively fast, the testing speed can be increased, thereby improving the efficiency of the test.

[0057] Speed ​​performance is one of the most noticeable metrics for users, significantly impacting click-through rates, user session duration, and long-term user frequency. Speed ​​evaluation dimensions are diverse, including application-specific dimensions, business-specific dimensions, competitor-specific dimensions, scenario-specific dimensions, phone type, and network conditions. Therefore, evaluation tasks are characterized by periodicity and repetition. Currently, many evaluation tasks are still entirely manual, and the long cycle of manual evaluation further impacts product iteration cycles.

[0058] A crucial aspect of the evaluation process is the automatic identification of the first and last frames using an algorithm. Current methods employ traditional image processing techniques, analyzing the changing process based on different scenarios and manually constructing feature recognition algorithms. This disclosure enables rapid first and last frame identification, allowing even users unfamiliar with image processing to achieve accurate automatic first and last frame recognition without adjusting parameters or performing process control.

[0059] Through the above steps, in response to a received page jump instruction, the page jump video from the current page to the target page is first obtained; the page jump video is segmented into frames to obtain multiple image frames; the page jump stage to which the multiple image frames belong is determined based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage are of the same type; the target image frames belonging to different page jump stages among the multiple image frames are located, thereby improving the efficiency of determining the target image frames. Since multiple image frames can be assigned to different page jump stages based on the similarity between them, and image frames belonging to the same page jump stage are of the same type, the required image frames can be efficiently located through the page jump stage, thus at least solving the technical problem of low efficiency in locating target image frames in related technologies.

[0060] Optionally, determining the page transition stage to which multiple image frames belong based on the similarity between multiple image frames includes: determining any two image frames among the multiple image frames; and determining the page transition stage to which multiple image frames belong based on the similarity between any two image frames.

[0061] The two image frames mentioned above can be any two image frames selected from multiple image frames, and the two image frames mentioned above can also be any two adjacent image frames from multiple image frames.

[0062] In one optional embodiment, if the similarity between any two image frames is small, it indicates that the two image frames belong to different page transition stages; if the similarity between any two image frames is large, it indicates that the two images belong to the same page transition stage.

[0063] In another optional embodiment, when any two image frames are randomly selected from multiple image frames, a binary search method can be used to accelerate the confirmation of similarity between multiple image frames. For example, the similarity between any two image frames can be determined. If the similarity between any two image frames is large, it means that the two image frames belong to the same page jump stage, and the similarity of other image frames between any two image frames does not need to be calculated again, as they all belong to the same page jump stage. If the similarity between any two image frames is small, it means that the two image frames do not belong to the same page jump stage. In this case, an image frame can be determined from between any two image frames, and the page jump type of multiple image frames can be determined based on this image frame. The binary search method can reduce the number of similarity calculations, thereby improving the efficiency of determining the page jump stage to which multiple image frames belong.

[0064] For example, when any two image frames are the first and last image frames, we can first determine the similarity between them. If the similarity is high, it means they belong to the same page transition stage. If the similarity is low, it means they belong to different page transition stages. In this case, we can determine the intermediate image frames based on the first and last image frames and determine their similarity. If the similarity is high, it means they belong to the same page transition stage. Therefore, all image frames between the first and intermediate image frames belong to the same page transition stage. We don't need to determine the other image frames between the first and intermediate image frames, which reduces the number of similarity calculations and improves the efficiency of determining the page transition stage of multiple image frames.

[0065] In another optional embodiment, when any two image frames are any two adjacent image frames among multiple image frames, similarity can be calculated sequentially based on any two adjacent image frames. If the similarity between any two adjacent image frames is large, it is determined that any two image frames belong to the same page jump stage. If the similarity between any two image frames is small, it is determined that any two image frames belong to different page jump stages. The image frame with the later time sequence among any two image frames is determined as the first image frame of the next jump stage, thereby determining the page jump stage to which multiple image frames belong.

[0066] By following the steps above, the efficiency of determining the page transition stage to which multiple image frames belong can be improved based on the similarity between any two image frames.

[0067] Optionally, determining the page transition stage to which multiple image frames belong based on the similarity between any two image frames includes: generating a first allocation result for any two image frames to be assigned to the same page transition stage in response to the similarity between any two image frames being greater than a preset threshold; generating a second allocation result for any two image frames to be assigned to different page transition stages in response to the similarity between any two image frames being less than or equal to the preset threshold; and determining the page transition stage to which multiple image frames belong based on the first allocation result and the second allocation result.

[0068] The aforementioned preset thresholds can be set by the user. They are mainly used to divide image frames into different types of thresholds.

[0069] In one optional embodiment, if the similarity between any two image frames is greater than a preset threshold, it can be determined that the two image frames are of the same type, and the two image frames can be assigned to the same page jump stage, generating a first assignment result; if the similarity between any two image frames is less than or equal to the preset threshold, it can be determined that the two image frames are of different types, and the two image frames can be assigned to different page jump stages, generating a second assignment result. The page jump stage to which the multiple image frames belong can be determined based on the final first and second assignment results of the multiple image frames.

[0070] Through the above steps, multiple image frames can be quickly allocated to the corresponding page jump stages based on the first allocation result and the second allocation result, thereby improving the allocation efficiency.

[0071] Optionally, determining the target image frame based on the page jump stage includes: determining the initial image frame of the multiple image frames based on the page jump stage; displaying the multiple image frames and the initial image frame on the interactive interface; and determining the initial image frame as the target image frame in response to receiving a confirmation instruction for the initial image frame.

[0072] The initial image frames mentioned above can be the first and last frames determined directly based on the page jump stage. However, these initial image frames may be inaccurate. Therefore, it is necessary to adjust the initial image frames to obtain an accurate last frame.

[0073] In one optional embodiment, after obtaining the page transition stage, multiple image frames can be displayed based on the page transition stage, and the last image frame of each page transition stage can be used as a transition image frame. This transition image frame is mainly used to indicate that the image frame jumps from the previous page transition stage to the next page transition stage. The initial image frames of the multiple image frames can be determined based on the transition image frames. Optionally, the transition image frame of the first page transition stage can be used as the first frame, and the first image frame of the last page transition stage, that is, the image frame adjacent to the transition image frame of the previous page transition stage, can be used as the last frame.

[0074] In another alternative embodiment, multiple image frames and an initial image frame can be displayed on the interactive interface so that the user can determine whether the initial image frame is accurate. If the initial image frame is highly accurate, a confirmation command for the initial image frame can be generated through a confirmation control, and the initial image frame can be determined as the target image frame based on the confirmation command.

[0075] Optionally, the method further includes: in response to receiving a switching instruction for the initial image frame, determining that the image frame corresponding to the switching instruction is the target image frame.

[0076] In one optional embodiment, the initial image frame can be an image frame marked by an image box. If the initial image frame is not accurately represented, the image box can be switched to another image frame by performing a switching operation, so that the final switched image frame can be the target image frame.

[0077] Figure 3 This is a schematic diagram of image frame switching according to an embodiment of the present disclosure, such as... Figure 3 As shown, the image frames can be switched by moving the buttons corresponding to the image frames, thereby determining the final target image frame. The left button is mainly used to confirm the first frame, and the right button is mainly used to confirm the last frame.

[0078] The switching operation described above can be generated by sliding, moving, scrolling, or other operations on multiple image frames.

[0079] Optionally, after obtaining multiple image frames corresponding to the page jump video, the method further includes: using a stage classification network to identify the multiple image frames to obtain the page jump stage.

[0080] The aforementioned stage classification network can be a neural network, where it is primarily used to determine the page transition stages of multiple image frames. Using a stage classification network can improve the efficiency of recognizing multiple image frames.

[0081] In one optional embodiment, multiple image frames are feature-encoded by an encoding module in a staged classification network to obtain image features of multiple image frames.

[0082] Figure 4 This is a schematic diagram illustrating the feature representation of an image frame according to an embodiment of the present disclosure. After obtaining the page jump video, the page jump video can be divided into frames using lines to obtain multiple image frames. Temporal enhancement can be performed on these multiple image frames. For example, two image frame sequences of length [T, x*T] can be extracted, with the two image frame sequences overlapping by 0.25*length, to obtain two comparison data sets, X1 and X2. Then, these two data sets are randomly sampled to obtain two temporal data sets, Xi and Xj. Then, spatial enhancement of the image is performed, such as image flipping, blurring, color conversion, random cropping, etc., in random combinations to obtain enhanced data, i.e., the enhanced two image frame sequences.

[0083] Furthermore, the enhanced sequences of two image frames can be fed into an encoding module to obtain encoded features. This encoding module can consist of a ResNet50 (residual network) and a transformer (feature extraction network). During training, the ResNet50 can have its first four blocks fixed, adjusting only the last block, or all block parameters can be fixed. The aforementioned feature extraction network can be trained using contrastive learning. The input to the entire encoding module is a sequence of images segmented into T frames, and the output is a T*2048 vector. Then, a projection (prediction module) is used to obtain the feature representation of T*1024 image frames. The prediction module can be a linear mapping function, and it can input the feature representation into a stage classification network to determine the page transition stage corresponding to multiple sample image frames.

[0084] Optionally, the method further includes: acquiring multiple sample image frames; determining the labeled page jump stage to which the multiple sample image frames belong based on the similarity between the multiple sample image frames; identifying the multiple sample image frames using an initial stage classification network to obtain the sample page jump stage; and training the initial stage classification network based on the labeled page jump stage and the sample page jump stage to obtain a stage classification network.

[0085] In one optional embodiment, multiple sample image frames can be acquired, and the page jump stages of the multiple sample image frames can be labeled based on the similarity between the multiple sample image frames to obtain labeled page jump stages. An initial stage classification network can be used to identify the multiple sample image frames to obtain sample page jump stages. It can be determined whether the difference between the sample page jump stages and the labeled page jump stages is large. If the difference between the two stages is large, a loss function needs to be directly constructed based on the labeled page jump stages and the sample page jump stages, and the initial stage classification network is trained based on the loss function to obtain a stage classification network.

[0086] Figure 5 This is a schematic diagram of a category label mapping method according to an embodiment of the present disclosure, such as... Figure 5 As shown, related technologies generally perform direct category mapping, but this can lead to discontinuous changes in categories. Figure 6 This is a schematic diagram of another category label mapping method according to an embodiment of the present disclosure, such as... Figure 6 As shown, in this disclosure, multiple image frames are identified according to the page jump stage, which can make the categories of multiple image frames change continuously.

[0087] By using the aforementioned stage classification network and introducing an additional loss function, the classification of each stage can be corrected, thereby improving the accuracy of page transition stage classification. Furthermore, there is a one-to-one correspondence between multiple image frames within the same page transition stage and the corresponding page transition stage. Figure 7 This is a schematic diagram of a linear mapping layer according to an embodiment of this application. For different stages, the types of image frames and the page jump stages are in a one-to-one correspondence.

[0088] Optionally, determining the annotation page jump stage to which the multiple sample image frames belong based on the similarity between the multiple sample image frames includes: determining any two sample image frames among the multiple sample image frames; and determining the annotation page jump stage to which the multiple sample image frames belong based on the similarity between the two sample image frames.

[0089] The two sample image frames mentioned above can be any two sample image frames selected from multiple sample image frames, and the two sample image frames mentioned above can also be any two adjacent sample image frames from multiple sample image frames.

[0090] In one optional embodiment, if the similarity between any two sample image frames is small, it indicates that the two sample image frames belong to different annotation page jump stages; if the similarity between any two sample image frames is large, it indicates that the two images belong to the same annotation page jump stage.

[0091] In another optional embodiment, when any two sample image frames are randomly selected from multiple sample image frames, a binary search method can be used to accelerate the confirmation of similarity between multiple sample image frames. For example, the similarity between any two sample image frames can be determined. If the similarity between any two sample image frames is large, it means that the two sample image frames belong to the same labeled page jump stage, and the similarity of other sample image frames between any two sample image frames does not need to be calculated again, as they all belong to the same labeled page jump stage. If the similarity between any two sample image frames is small, it means that the two sample image frames do not belong to the same labeled page jump stage. In this case, a sample image frame can be determined from between any two sample image frames, and the page jump type of multiple sample image frames can be further determined based on this sample image frame. The binary search method can reduce the number of similarity calculations, thereby improving the efficiency of determining the labeled page jump stage to which multiple sample image frames belong.

[0092] For example, when any two sample image frames are the first and last sample image frames, we can first determine the similarity between them. If the similarity is high, it means they belong to the same annotation page transition stage. If the similarity is low, it means they belong to different annotation page transition stages. In this case, we can determine the intermediate sample image frames based on the first and last sample image frames and determine their similarity. If the similarity is high, it means they belong to the same annotation page transition stage. Then, all sample image frames between the first and intermediate sample image frames belong to the same annotation page transition stage. We don't need to determine other sample image frames between the first and intermediate sample image frames, which reduces the number of similarity calculations and improves the efficiency of determining the annotation page transition stage of multiple sample image frames.

[0093] In another optional embodiment, when any two sample image frames are any two adjacent sample image frames among multiple sample image frames, similarity can be calculated sequentially based on any two adjacent sample image frames. If the similarity between any two adjacent sample image frames is large, it is determined that any two sample image frames belong to the same annotation page jump stage. If the similarity between any two sample image frames is small, it is determined that any two sample image frames belong to different annotation page jump stages. The sample image frame with the later time sequence among any two sample image frames is determined as the first sample image frame of the next jump stage, thereby determining the annotation page jump stage to which multiple sample image frames belong.

[0094] By following the steps above, the efficiency of determining the labeled page jump stage to which multiple sample image frames belong can be improved based on the similarity between any two sample image frames.

[0095] Optionally, determining the annotation page transition stage to which multiple sample image frames belong based on the similarity between any two sample image frames includes: generating a third allocation result where the similarity between any two sample image frames is greater than a preset threshold, assigning the two sample image frames to the same annotation page transition stage; generating a fourth allocation result where the similarity between any two sample image frames is less than or equal to a preset threshold, assigning the two sample image frames to different annotation page transition stages; and determining the annotation page transition stage to which multiple sample image frames belong based on the third and fourth allocation results. The preset threshold can be set by the user and mainly represents the threshold for classifying image frames into different types.

[0096] In one optional embodiment, if the similarity between any two sample image frames is greater than a preset threshold, it can be determined that the two sample image frames are of the same type, and the two sample image frames can be assigned to the same annotation page jump stage, and a first allocation result is generated; if the similarity between any two sample image frames is less than or equal to the preset threshold, it can be determined that the two sample image frames are of different types, and the two sample image frames can be assigned to different annotation page jump stages, and a second allocation result is generated. The annotation page jump stage to which the multiple sample image frames belong can be determined based on the final first allocation result and the second allocation result of the multiple sample image frames.

[0097] Through the above steps, multiple sample image frames can be quickly assigned to the corresponding annotation page jump stage based on the third and fourth allocation results, thereby improving the allocation efficiency.

[0098] Optionally, the method further includes: determining the target jump stage in which the target image frame is located during the page jump stage; deleting the image frame corresponding to the target jump stage in the page jump video to obtain the target jump video; and storing the target jump video.

[0099] The aforementioned target jump stage can be a stage in the page jump process. This stage generally does not contain useful information. Therefore, the image frames corresponding to the target jump stage in the page jump video can be deleted to reduce the content resources occupied by the target jump video.

[0100] Currently, the first and last frames are identified primarily using the following methods:

[0101] The first approach is a page state recognition process built upon traditional image processing algorithms such as page blank state analysis, template matching, and image similarity. It uses a sequential execution method to determine the first and last frames. Assuming a first frame is provided, it starts by searching forward, using template matching to find the frame where the page changes as the starting point. Then, it uses page blank state detection to find the point where page loading is stable. Finally, it searches backward to find the frame where the page does not change, which is then taken as the last frame. This method is primarily used for scenes where page elements are static.

[0102] The second approach is to classify page state stages based on traditional machine learning methods. Once the page transition states are known, the time consumed between any state changes can be calculated. This solution is currently open source.

[0103] However, both of the above methods have certain drawbacks, as follows:

[0104] The first approach involves manually designing features, which requires analyzing the scenario, selecting the appropriate algorithm, and building the algorithm analysis workflow. This requires users to have a certain level of image processing and analysis knowledge. Furthermore, the construction process relies on manually adjusting parameters. A single scenario may encounter multiple different processes; for example, some page transitions may involve a period of blank loading, while others may jump directly without this blank loading period. These different process variations necessitate new process control and adaptation for traditional algorithms.

[0105] The second approach relies on traditional feature extraction operators, which are sensitive to lighting, rotation, and resolution. This can easily lead to false positives and low resolution. Furthermore, this method analyzes features from individual images, losing important information from adjacent images and thus reducing overall accuracy.

[0106] This disclosure can divide the screen recording process into various stages, making it easier to quickly locate the execution stage and saving time spent manually searching for a large number of video frames.

[0107] This disclosure provides, for example Figure 8 Another method for determining image frames is shown. Figure 8 This is a flowchart of another method for determining an image frame according to an embodiment of the present disclosure, such as... Figure 8 As shown, the method may include the following steps:

[0108] Step S802: In response to receiving a page jump instruction, obtain the page jump video from the current page to the target page;

[0109] Step S804: Divide the page jump video into frames to obtain multiple image frames;

[0110] Step S806: Determine any two image frames from the multiple image frames;

[0111] Step S808: Determine the page jump stage to which multiple image frames belong based on the similarity between any two image frames;

[0112] Step S810: Locate the target image frame that belongs to a different page transition stage among multiple image frames.

[0113] This disclosure provides, for example Figure 9 Another method for determining image frames is shown. Figure 9 This is a flowchart of another method for determining an image frame according to an embodiment of the present disclosure, such as... Figure 9 As shown, the method may include the following steps:

[0114] Step S902: In response to receiving a page jump instruction, obtain the page jump video from the current page to the target page;

[0115] Step S904: Divide the page jump video into frames to obtain multiple image frames;

[0116] Step S906: Determine the page jump stage to which the multiple image frames belong based on the similarity between the multiple image frames.

[0117] Among them, the image frames belonging to the same page jump stage are of the same type;

[0118] Step S908: Determine the initial image frames of multiple image frames based on the page jump stage;

[0119] Step S910: Display multiple image frames and the initial image frame on the interactive interface;

[0120] Step S912: In response to receiving the confirmation instruction for the initial image frame, the initial image frame is determined to be the target image frame.

[0121] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0122] Figure 10 This is a schematic diagram of an image frame determination device according to an embodiment of the present disclosure, such as... Figure 10 As shown, the image frame determination device 1000 includes: an acquisition module 1002, a framing module 1004, a first determination module 1006, and a second determination module 1008.

[0123] The system includes a first acquisition module, which acquires a page jump video from the current page to the target page in response to a page jump instruction; a frame segmentation module, which segments the page jump video into multiple image frames; a first determination module, which determines the page jump stage to which the multiple image frames belong based on the similarity between the multiple image frames, wherein image frames belonging to the same page jump stage are of the same type; and a positioning module, which locates the target image frame belonging to a different page jump stage among the multiple image frames.

[0124] Optionally, the first determining module is further configured to determine any two image frames among the multiple image frames, and determine the page jump stage to which the multiple image frames belong based on the similarity between the two image frames.

[0125] Optionally, the first determining module is further configured to generate a first allocation result in which any two image frames are assigned to the same page jump stage in response to the similarity between any two image frames being greater than a preset threshold, and to generate a second allocation result in which any two image frames are assigned to different page jump stages in response to the similarity between any two image frames being less than or equal to the preset threshold, and to determine the page jump stages to which multiple image frames belong based on the first allocation result and the second allocation result.

[0126] Optionally, the first determining module includes: a first determining unit, configured to determine any two image frames among a plurality of image frames; and a second determining unit, configured to determine the page jump stage to which the plurality of image frames belong based on the similarity between the two image frames.

[0127] Optionally, the second determining unit includes: a first generating subunit, configured to generate a first allocation result for assigning any two image frames to the same page transition stage in response to a similarity between any two image frames being greater than a preset threshold; a second generating subunit, configured to generate a second allocation result for assigning any two image frames to different page transition stages in response to a similarity between any two image frames being less than or equal to a preset threshold; and a determining subunit, configured to determine the page transition stages to which multiple image frames belong based on the first allocation result and the second allocation result.

[0128] Optionally, the second determining module includes: a third determining unit, used to determine an initial image frame of multiple image frames based on the page jump stage; a display unit, used to display multiple image frames and the initial image frame on the interactive interface; and a fourth determining unit, used to determine the initial image frame as the target image frame in response to receiving a confirmation instruction for the initial image frame.

[0129] Optionally, the device further includes a third determining module, configured to determine, in response to receiving a switching instruction for an initial image frame, that the image frame corresponding to the switching instruction is the target image frame.

[0130] Optionally, the device further includes: a first recognition module, used to recognize multiple image frames using a stage classification network to obtain page jump stages.

[0131] Optionally, the device further includes: a second acquisition module for acquiring multiple sample image frames; a fourth determination module for determining the labeled page jump stage to which the multiple sample image frames belong based on the similarity between the multiple sample image frames; a second recognition module for recognizing the multiple sample image frames using an initial stage classification network to obtain the sample page jump stage; and a training module for training the initial stage classification network based on the labeled page jump stage and the sample page jump stage to obtain a stage classification network.

[0132] Optionally, the device is further configured to determine the target jump stage in which the target image frame is located during the page jump stage, delete the image frame corresponding to the target jump stage in the page jump video, obtain the target jump video, and store the target jump video.

[0133] Optionally, the second acquisition module includes: a fifth determining unit, configured to determine any two sample image frames among the plurality of sample image frames; and a sixth determining unit, configured to determine the labeled page jump stage to which the plurality of sample image frames belong based on the similarity between the arbitrary two sample image frames.

[0134] Optionally, the sixth determining unit includes: a third generating subunit, configured to generate a third allocation result in which the two sample image frames are assigned to the same annotation page jump stage in response to the similarity between the two sample image frames being greater than a preset threshold; a fourth generating subunit, configured to generate a fourth allocation result in which the two sample image frames are assigned to different annotation page jump stages in response to the similarity between the two sample image frames being less than or equal to the preset threshold; and a jump stage determining subunit, configured to determine the annotation page jump stage to which the plurality of sample image frames belong based on the third allocation result and the fourth allocation result.

[0135] According to another embodiment of this disclosure, an electronic device is also provided, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of any of the above embodiments.

[0136] According to another embodiment of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is also provided, wherein the computer instructions are used to cause a computer to perform any of the methods described in the above embodiments.

[0137] According to another embodiment of this disclosure, a computer program product is also provided, including a computer program that, when executed by a processor, implements the method of any of the above embodiments.

[0138] Figure 11 This is a schematic block diagram of an electronic device 1100 according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0139] like Figure 11 As shown, device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in read-only memory (ROM) 1102 or a computer program loaded from storage unit 1108 into random access memory (RAM) 1103. The RAM 1103 may also store various programs and data required for the operation of device 1100. The computing unit 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Input / output (I / O) interface 1105 is also connected to bus 1104.

[0140] Multiple components in device 1100 are connected to I / O interface 1105, including: input unit 1106, such as keyboard, mouse, etc.; output unit 1107, such as various types of monitors, speakers, etc.; storage unit 1108, such as disk, optical disk, etc.; and communication unit 1109, such as network card, modem, wireless transceiver, etc. Communication unit 1109 allows device 1100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] The computing unit 1101 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the method for determining image frames. For example, in some embodiments, the method for determining image frames may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on device 1100 via ROM 1102 and / or communication unit 1109. When the computer program is loaded into RAM 1103 and executed by the computing unit 1101, one or more steps of the method for determining image frames described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform a method for determining image frames by any other suitable means (e.g., by means of firmware).

[0142] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0143] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0144] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0146] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0147] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0148] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0149] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for determining an image frame, comprising: In response to receiving a page redirection command, retrieve the page redirection video from the current page to the target page; The video of the page jump is divided into frames to obtain multiple image frames; The page transition stage to which the multiple image frames belong is determined based on the similarity between the multiple image frames, wherein image frames belonging to the same page transition stage are of the same type; Locate the target image frame belonging to different page transition stages among the multiple image frames; The method of determining the page transition stage to which the plurality of image frames belong based on the similarity between the plurality of image frames includes: generating a first allocation result in which the two image frames are assigned to the same page transition stage in response to the similarity between any two image frames being greater than a preset threshold; generating a second allocation result in which the two image frames are assigned to different page transition stages in response to the similarity between any two image frames being less than or equal to the preset threshold; and determining the page transition stage to which the plurality of image frames belong based on the first allocation result and the second allocation result, wherein the page transition stage includes: a stage before page transition, a stage during page transition, and a stage after page transition. Locating a target image frame belonging to different page transition stages among the plurality of image frames includes: determining an initial image frame of the plurality of image frames based on the page transition stage, wherein the initial image frame is the first and last frame determined according to the page transition stage; displaying the plurality of image frames and the initial image frame on an interactive interface; and determining the initial image frame as the target image frame in response to receiving a confirmation instruction for the initial image frame. The method further includes: determining the target jump stage in which the target image frame is located in the page jump stage, wherein the target jump stage is a stage in the page jump stage; deleting the image frame corresponding to the target jump stage in the page jump video to obtain the target jump video; and storing the target jump video.

2. The method according to claim 1, further comprising: Determine any two image frames from the plurality of image frames.

3. The method according to claim 1, further comprising: In response to receiving a switching instruction for the initial image frame, the image frame corresponding to the switching instruction is determined to be the target image frame.

4. The method according to claim 1, after obtaining multiple image frames corresponding to the page jump video, the method further includes: The multiple image frames are identified using a stage classification network to obtain the page jump stage.

5. The method according to claim 4, further comprising: Acquire multiple sample image frames; The annotation page jump stage to which the multiple sample image frames belong is determined based on the similarity between the multiple sample image frames; The initial stage classification network is used to identify the multiple sample image frames to obtain the sample page jump stage; The initial stage classification network is trained based on the labeled page jump stage and the sample page jump stage to obtain the stage classification network.

6. The method according to claim 5, wherein determining the labeled page jump stage to which the plurality of sample image frames belong based on the similarity between the plurality of sample image frames includes: Determine any two sample image frames from the plurality of sample image frames; The labeled page jump stage to which the plurality of sample image frames belong is determined based on the similarity between any two sample image frames.

7. The method according to claim 6, wherein determining the labeled page jump stage to which the plurality of sample image frames belong based on the similarity between any two sample image frames includes: In response to the similarity between any two sample image frames being greater than a preset threshold, a third allocation result is generated to assign the two sample image frames to the same annotation page jump stage; In response to the similarity between any two sample image frames being less than or equal to the preset threshold, a fourth allocation result is generated, assigning the two sample image frames to different annotation page jump stages; The annotation page jump stage to which the plurality of sample image frames belong is determined based on the third allocation result and the fourth allocation result.

8. An image frame determining device, comprising: The first acquisition module is used to acquire the page jump video from the current page to the target page in response to receiving a page jump instruction; The frame segmentation module is used to segment the page jump video into multiple image frames. The first determining module is used to determine the page jump stage to which the plurality of image frames belong based on the similarity between the plurality of image frames, wherein image frames belonging to the same page jump stage are of the same type; The positioning module is used to locate the target image frames belonging to different page transition stages among the multiple image frames; The first determining module includes: The first generation subunit is configured to generate a first allocation result for assigning the two image frames to the same page jump stage in response to the similarity between any two image frames in the plurality of image frames being greater than a preset threshold. The second generation subunit is used to generate a second allocation result for assigning the two image frames to different page jump stages in response to the similarity between any two image frames in the plurality of image frames being less than or equal to the preset threshold. A determining subunit is configured to determine the page transition stage to which the plurality of image frames belong based on the first allocation result and the second allocation result, wherein the page transition stage includes: a stage before page transition, a stage during page transition, and a stage after page transition; The positioning module includes: The third determining unit is used to determine the initial image frame of the plurality of image frames based on the page jump stage, wherein the initial image frame is the first frame and the last frame determined according to the page jump stage; The display unit is used to display the plurality of image frames and the initial image frame on the interactive interface; The fourth determining unit is configured to determine the initial image frame as the target image frame in response to receiving a confirmation instruction for the initial image frame; The fourth determining unit is further configured to determine the target jump stage in which the target image frame is located in the page jump stage, wherein the target jump stage is a stage in the page jump stage; delete the image frame corresponding to the target jump stage in the page jump video to obtain the target jump video; and store the target jump video.

9. The apparatus according to claim 8, wherein the first determining module comprises: The first determining unit is used to determine any two image frames among the plurality of image frames.

10. The apparatus of claim 8, further comprising: The third determining module is used to determine, in response to receiving a switching instruction for the initial image frame, the image frame corresponding to the switching instruction as the target image frame.

11. The apparatus of claim 8, further comprising: The first recognition module is used to identify the multiple image frames using a stage classification network to obtain the page jump stage.

12. The apparatus of claim 11, further comprising: The second acquisition module is used to acquire multiple sample image frames; The fourth determining module is used to determine the labeled page jump stage to which the plurality of sample image frames belong based on the similarity between the plurality of sample image frames; The second recognition module is used to recognize the multiple sample image frames using the initial stage classification network to obtain the sample page jump stage; The training module is used to train the initial stage classification network based on the labeled page jump stage and the sample page jump stage to obtain the stage classification network.

13. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images