Time determination method and device

Abstract

The embodiment of the invention provides a moment determination method and device, and the method comprises the steps: obtaining first video data which is used for recording the operation steps executed by a user in the interaction process with an interface and the change condition of the interface, and inputting the first video data into a large-scale language model, namely a target analysis model; according to the method, the switching time of each interface in the video data can be automatically and efficiently identified and extracted, and a large language model is adopted for intelligent analysis, so that the time and energy of manual analysis are saved, and the labor cost for determining the switching time of each interface in the interface interaction process is reduced.

Description

Technical Field

[0001] The present invention relates to the field of Internet technology, and in particular to a time determination method and device. Background Art

[0002] In some application scenarios, for some software with more complex operations, when users try to implement a specific function, they may encounter difficulties due to their unfamiliarity with the software interface interaction methods. In this case, it is particularly important to provide users with a software usage guide. The software usage guide can provide a detailed introduction to the various interface interaction methods of the software and their corresponding functions. Among them, the interface interaction method refers to the various ways of information transmission and operation between users and the interface of each software, including but not limited to click jump, flip, slide, zoom, long press, drag, etc.

[0003] To provide users with software usage guides, it's necessary to understand the interface interaction methods of each software. Currently, obtaining software interface interaction methods is mainly achieved through manual analysis of collected video data of the interface interaction process. This method can manually identify the specific moments when each interface in the software switches, and then use these moments to sort out the complete interface interaction method of the software. However, this method has many drawbacks: on the one hand, it requires a lot of time and effort; on the other hand, with the continuous increase in the number of software and the continuous innovation of interface interaction design, the complexity of manual analysis is increasing, resulting in high labor costs. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method and device for determining the time to switch between interfaces, thereby reducing the manual cost of determining the time to switch between interfaces during the interface interaction process. The specific technical solution is as follows:

[0005] In a first aspect of the present invention, a method for determining a time is provided, the method comprising:

[0006] Acquire first video data obtained during the acquisition interface interaction process; wherein the first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface;

[0007] Input the first video data into a target analysis model, and obtain the first and second moments corresponding to each interface appearing in the output first video data; wherein the target analysis model is a large language model, the first moment is used to indicate the moment when another interface switches to the corresponding interface in the first video data, and the second moment is used to indicate the moment when the corresponding interface switches to another interface in the first video data.

[0008] In a possible implementation, obtaining the first video data obtained during the acquisition interface interaction process includes:

[0009] Acquire multiple initial video data obtained during the acquisition interface interaction process;

[0010] Video data that meets a preset video quality condition is determined from the multiple initial video data as the first video data; wherein the preset video quality condition is used to determine video data that meets the video quality requirement of the target analysis model for the input video data.

[0011] In one possible implementation, the method further includes:

[0012] Obtaining a question text input for the first video data; wherein the question text is used to guide the target analysis model to analyze and output a first moment and a second moment corresponding to each interface appearing within a target time range in the first video data;

[0013] Inputting the first video data into a target analysis model and obtaining a first moment and a second moment corresponding to each interface appearing in the output first video data includes:

[0014] The first video data and the question text are input into the target analysis model to obtain the first moment and second moment corresponding to each interface appearing in the output first video data; wherein, the first moment and the second moment include the first moment and second moment corresponding to each interface appearing in the first video data within the target time range.

[0015] In one possible implementation,

[0016] The method further comprises:

[0017] For each of the interfaces, video data between a first moment corresponding to the interface and a second moment corresponding to the interface are intercepted from the first video data as second video data corresponding to the interface;

[0018] The second video data are output.

[0019] In one possible implementation, the method further includes:

[0020] Obtaining an interface name corresponding to each interface appearing in the first video data output by the target analysis model;

[0021] For each of the interfaces, the interface name of the interface and the second video data corresponding to the interface are output accordingly.

[0022] In one possible implementation, the method further includes:

[0023] Determine an interface whose interface name is a preset first interface name, and modify the interface name of the determined interface to a second interface name corresponding to the preset first interface name; wherein, the second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

[0024] In a second aspect of the present invention, a time determination device is provided, comprising:

[0025] A data acquisition module is used to acquire first video data obtained during the interaction process of the acquisition interface; wherein the first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface;

[0026] A data analysis module is used to input the first video data into a target analysis model to obtain the first moment and second moment corresponding to each interface appearing in the output first video data; wherein, the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

[0027] In a possible implementation, the data acquisition module includes:

[0028] The first data acquisition submodule is used to acquire a plurality of initial video data obtained during the interface interaction process of the acquisition interaction terminal;

[0029] The second data acquisition submodule is used to determine video data that meets the preset video quality conditions from the multiple initial video data as the first video data; wherein the preset video quality conditions are used to determine video data that meets the video quality requirements of the target analysis model for the input video data.

[0030] In a possible implementation, the device further includes:

[0031] a text acquisition module, configured to acquire a question text input for the first video data; wherein the question text is used to guide the target analysis model to analyze and output a first moment and a second moment corresponding to each interface appearing within a target time range in the first video data;

[0032] The data analysis module includes:

[0033] The first sub-module of data analysis is used to input the first video data and the question text into the target analysis model to obtain the first moment and the second moment corresponding to each interface appearing in the output first video data; wherein, the first moment and the second moment include the first moment and the second moment corresponding to each interface appearing in the first video data within the target time range.

[0034] In one possible implementation,

[0035] a data interception module, configured to intercept, for each of the interfaces, video data between a first moment corresponding to the interface and a second moment corresponding to the interface in the first video data as second video data corresponding to the interface;

[0036] A data output module is used to output each of the second video data.

[0037] In a possible implementation, the device further includes:

[0038] a name acquisition module, configured to acquire an interface name corresponding to each interface appearing in the first video data output by the target analysis model;

[0039] The interface parsing module is used to output the interface name of each interface and the second video data corresponding to the interface.

[0040] In a possible implementation, the device further includes:

[0041] A name modification module is used to determine an interface whose interface name is a preset first interface name, and to modify the interface name of the determined interface to a second interface name corresponding to the preset first interface name; wherein, the second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

[0042] In another aspect of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, any of the above-mentioned methods for determining the moment is implemented.

[0043] In another aspect of the present invention, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned methods for determining the time.

[0044] A method and device for determining a moment provided by an embodiment of the present invention can automatically and efficiently identify and extract the switching moments of each interface in the video data by acquiring first video data for recording the operation steps performed by a user during interaction with an interface and changes in the interface, and inputting the data into a large language model, namely a target analysis model. Since a large language model is used for intelligent analysis, the time and effort of manual analysis are saved, and the manual cost of determining the switching moments of each interface during the interface interaction process is reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for describing the embodiments or the prior art.

[0046] Figure 1 Schematic diagram of the flow of the time determination method in an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram of a first method of determining time in an embodiment of the present invention;

[0048] Figure 3 This is a second schematic diagram of time determination in an embodiment of the present invention;

[0049] Figure 4 This is a third schematic diagram of time determination in an embodiment of the present invention;

[0050] Figure 5 This is a fourth schematic diagram of time determination in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of a first type of determination result output in an embodiment of the present invention;

[0052] Figure 7 This is a first schematic diagram of a target video frame according to an embodiment of the present invention;

[0053] Figure 8 This is a second schematic diagram of a target video frame according to an embodiment of the present invention;

[0054] Figure 9 This is a second schematic diagram of determining the output of the result in an embodiment of the present invention;

[0055] Figure 10 This is a schematic diagram of a first method for determining time in an embodiment of the present invention;

[0056] Figure 11 This is a schematic diagram of a second method for determining time in an embodiment of the present invention;

[0057] Figure 12 This is a third schematic diagram of determining result output in an embodiment of the present invention;

[0058] Figure 13 Schematic diagram of the structure of a time determination device in an embodiment of the present invention;

[0059] Figure 14 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present invention will be described below with reference to the accompanying drawings in the embodiments of the present invention.

[0061] In order to reduce the labor cost of determining the switching time of each interface during the interface interaction process, the present invention provides a time determination method, device, electronic device and storage medium. The time determination method provided by the present invention is first exemplified below.

[0062] The time determination method provided by the present invention can be applied to a time determination device, which can be any electronic device, including but not limited to a DC (Data Center) terminal, a computer, a mobile terminal, etc.

[0063] See also Figure 1 , Figure 1 : is a flow chart of a method for determining time in an embodiment of the present invention, the method comprising:

[0064] S101: Acquire first video data obtained during the interaction process of the acquisition interface.

[0065] The first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes in the interface.

[0066] S102: Input the first video data into a target analysis model, and obtain a first moment and a second moment corresponding to each interface appearing in the output first video data.

[0067] Among them, the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

[0068] By applying the above embodiment, by obtaining the first video data for recording the operation steps performed by the user in the process of interacting with the interface and the changes in the interface, and inputting it into the large language model, i.e., the target analysis model, the switching moments of each interface in the video data can be automatically and efficiently identified and extracted. Since a large language model is used for intelligent analysis, it not only saves time and energy for manual analysis, but also avoids the low accuracy caused by manual negligence, thereby reducing the labor cost of determining the switching moments of each interface during the interface interaction process and improving the accuracy of the moment determination.

[0069] The above steps S101-S102 are exemplarily described below:

[0070] In step S101, the first video data may be obtained by capturing and recording the interface interaction process of the interactive terminal by a capturing device.

[0071] An interactive terminal is any electronic device that can support user interaction, including but not limited to mobile phones, computers, etc.

[0072] A data acquisition device is any device with data acquisition capabilities, including but not limited to the MTT (Mobile Testing Toolkit) integrated with the Appium framework, an open-source mobile automated testing framework, as well as high-definition cameras and professional screen recording software.

[0073] Taking the MTT as an example, in which the acquisition device is integrated with the appium framework, MTT supports multiple programming languages such as Java and Python to write test scripts for capturing and recording the interface interaction process of the interactive terminal. Among them, Java and Python are two different programming languages. After receiving the written test script, the MTT end executes the test script and obtains the first video data, which ensures the accuracy and completeness of the video data acquisition and greatly improves the flexibility of data acquisition. After the first video data is acquired, the first video data will be stored. The first video data can be stored in any storage device, or the first video data can be uploaded to the OSS (Object Storage Service, object storage service) server and stored by the OSS server. When acquiring the first video data, the analysis device can obtain the first video data from the OSS server.

[0074] In step S102, the target analysis model analyzes the first video data based on the input first video data using image recognition, deep learning, natural language processing and other means, and outputs the moment when each interface appearing in the first video data, that is, the first moment, and the moment when each interface disappears, that is, the second moment.

[0075] Specifically, the target analysis model identifies and analyzes elements in the video image of the first video data. When the first appearance of a landmark element of a specific interface is detected, the current moment is determined to be the first moment of that interface. The second moment is when the interface disappears, that is, when all elements on the interface are no longer displayed or the key features of the interface disappear, the model determines the current moment to be the second moment of that interface.

[0076] In a possible embodiment, the target analysis model may be pre-configured with a question text, which enables the target analysis model to output the first moment and the second moment.

[0077] In another possible embodiment, the target analysis model can also receive user-defined question text (i.e., question text hereinafter) input for the first video data, and guide the target analysis model to analyze and output the first moment and second moment corresponding to each interface appearing in the first video data within the target time range. The question text will be explained below and will not be repeated here.

[0078] The target analysis model is any model in the Large Language Model (LLM). For example, it may be GPT4.0-VISION, which is a language model.

[0079] For example, see Figure 2 , Figure 2 This is a schematic diagram of the first method of determining the time in an embodiment of the present invention. Taking the first video data as the video software interface interaction process of the acquisition interaction terminal, the target analysis model is GPT4.0-VISION, and the pre-configured question text is "Analyze the time nodes of all interface changes after opening the video software in this video" as an example, after the first video data is input into GPT4.0-VISION, the first and second moments output by the target analysis model are obtained as follows:

[0080] “1, 00:00:00; 2. 00:00:00:05.034; 3. 00:00:00:05.034; 4. 00:00:06; 5. 00:00:06; 6. 00:00:22.017; 7, 00:00:22.017; 8, 00:00:29.034; 9. 00:00:30; 10, 00:00:33.051; 11, 00:00:33.051;

[0091] 12, 00:00:40.068.

[0092] That is, in the first video data, there are a total of 6 interfaces, which are respectively recorded as interface A, interface B, interface C, interface D, interface E and interface F for the convenience of description.

[0093] The first moment corresponding to interface A is 00:00:00, and the second moment is 00:00:00:05.034; the first moment corresponding to interface B is 00:00:00:05.034, and the second moment is 00:00:06; the first moment corresponding to interface C is 00:00:06, and the second moment is 00:00:22.017; the first moment corresponding to interface D is 00:00:22.017, and the second moment is 00:00:29.034; the first moment corresponding to interface E is 00:00:30, and the second moment is 00:00:33.051; the first moment corresponding to interface F is 00:00:33.051, and the second moment is 00:00:40.068.

[0094] That is, from the 0th second to the 5.034th millisecond, the interactive terminal displays Interface A. At 5.034th millisecond, the interface switches from Interface A to Interface B. From 5.034th millisecond to the 6th second, the displayed interface is Interface B. At the 6th second, the interface switches from Interface B to Interface C. From the 6th second to 22.017th second, the interactive terminal displays Interface C. At 22.017th second, the interface switches from Interface C to Interface D. From 22.017th second to 29.034th second, the interactive terminal displays Interface D. At the 30th second, the interface switches to Interface E. From the 30th second to 33.051th second, the interactive terminal displays Interface E. At the 33.051th second, the interface switches from Interface E to Interface F. From 33.051th second to 40.068th second, the interactive terminal displays Interface F.

[0095] In a possible embodiment, in order to further improve the accuracy of moment determination, the first video data may be video data whose video quality obtained by screening satisfies a preset video quality condition. Specifically, the above step S101 includes:

[0096] S1011, acquiring a plurality of initial video data obtained during the interface interaction process of the acquisition interaction terminal.

[0097] S1012 , determining video data that meets a preset video quality condition from a plurality of initial video data as first video data.

[0098] Among them, the preset video quality conditions are used to determine video data that meets the video quality requirements of the target analysis model for input video data. They can be set by professional staff based on work experience or according to industry standards, and are not specifically limited here.

[0099] For example, the preset video quality condition may be that the video frame rate reaches a certain value to ensure that the target analysis model can accurately output the corresponding interface switching moments based on the first video data. The preset video quality condition may also be that the video integrity meets a certain standard to ensure that the target analysis model can output the corresponding moments of all interface switching during the interface interaction process based on the first video data. The examples are not exhaustive here.

[0100] By applying the above embodiment, multiple initial video data sets of the interface interaction process are acquired and filtered to select video data that meets the preset video quality requirements from the multiple initial video data sets. This is used as the first video data, eliminating low-quality video data that does not meet the standards. This ensures that the video data used for input into the target analysis model for analysis is of high quality, effectively avoiding analysis errors caused by video quality issues. In this way, the method can more accurately capture the details of the interface interaction, thereby improving the accuracy of moment determination.

[0101] In the aforementioned step S102, the target analysis model may also receive a question text input for the first video data to output a first time and a second time corresponding to each interface appearing within a target time range in the first video data. The method will be described below. Specifically, after obtaining the question text input for the first video data, the aforementioned step S102 includes:

[0102] S1021: Input the first video data and the question text into the target analysis model, and obtain the first moment and the second moment corresponding to each interface appearing in the output first video data.

[0103] The first moment and the second moment include the first moment and the second moment corresponding to each interface appearing within the target time range in the first video data.

[0104] For example, see Figure 3 , Figure 3 This is a second schematic diagram of determining the time in the embodiment of the present invention, with the aforementioned Figure 2 For example, the query text for the first video data input is "Analyze all the time nodes where the interface changes occur from the 22nd to the 30th second in this video." That is, the target time range is from the 22nd to the 30th second. After the first video data is input into GPT4.0-VISION, the first and second moments of the target analysis model output are obtained as follows:

[0105] “1, 00:00:06; 2. 00:00:22.017; 3. 00:00:22.017; 4. 00:00:29.034; 5. 00:00:30;

[0110] 6. 00:00:30. ”

[0111] The first moment corresponding to interface C is 00:00:06, and the second moment is 00:00:22.017. The first moment corresponding to interface D is 00:00:22.017, and the second moment is 00:00:29.034. The first moment corresponding to interface E is 00:00:30, and the second moment is also 00:00:30.

[0112] By applying the above embodiment, by obtaining the question text input by the user for the first video data, and inputting the question text together with the video data into the target analysis model, the model can understand the user's intention more accurately, thereby outputting the first moment and second moment corresponding to each interface in the video, which are consistent with the user's question, thereby improving the pertinence and accuracy of moment determination, saving not only the time and energy of manual analysis, but also avoiding the low accuracy caused by manual negligence, thereby reducing the labor cost of moment determination and improving the accuracy of moment determination.

[0113] For the above first video data, if the user wants to know more information in the first video data, the user can also add "analyze the time nodes of all interface changes after opening the video software in this video, and what content is contained in the interface" in the question text. Based on the question text, see Figure 4 , Figure 4 This is a third schematic diagram of time determination in an embodiment of the present invention. After the first video data is input into GPT4.0-VISION, the first time, second time and other contents output by the target analysis model are obtained as follows:

[0114] "1. From 00:00:00 to 00:00:00:05.034, a mobile phone desktop is displayed with a cloud background and some software icons;

[0115] 2. From 00:00:00:05.034 to 00:00:06, a loading animation appears, which may be a click to enter a video software;

[0116] From 00:00:06 to 00:00:22.017, the video shows a series of screen changes with advertising text and images, seemingly displaying information about a product or offer.

[0117] 4. From 00:00:22.017 to 00:00:29.034, a video list was displayed on the interface, and the user was browsing different video content;

[0118] 5. From 00:00:30 to 00:00:33.051, the interface displays the user's personal account information and membership service information;

[0119] 6. From 00:00:33.051 to 00:00:40.068, the interface displayed the prices and privileges of different membership services. Users may be viewing or selecting different membership service options;

[0120] In summary, in this video, the user appears to perform the following interactive steps: viewing the phone's home screen, opening the video app, viewing in-app advertising, browsing the video list and potential content, viewing personal account information, and browsing and managing membership services. Each step demonstrates the different functions of the video app and the user's possible operation path.

[0121] That is, in the first video data, the first moment corresponding to the mobile phone main interface is 00:00:00, and the second moment is 00:00:00:05.034. The mobile phone main interface contains a cloud background and some software icons. The first moment corresponding to the video software interface is 00:00:00:05.034, and the second moment is 00:00:06. The video software interface contains a loading animation. The software advertising interface corresponds to 00:00:06 to 00:00:22.017. The software advertising interface contains text and image content used to indicate relevant information about products or discounts. The first moment corresponding to the video list interface is 00:00:22.017, and the second moment is 00:00:29.034. The video list interface contains different video content. The first moment corresponding to the personal account interface is 00:00:30, and the second moment is 00:00:33.051. The personal account interface contains the user's personal account information and membership service information. The first time corresponding to the member interface is 00:00:33.051, and the second time is 00:00:40.068. The member interface contains the prices and privilege descriptions of different member services.

[0122] In a possible embodiment, when providing the software usage guide to the user according to the determined time, in order to allow the user to selectively browse the specific operation process or content included in one or more interfaces according to their own needs, the method further includes:

[0123] S103 , for each interface, intercepting video data between a first moment corresponding to the interface and a second moment corresponding to the interface from the first video data as second video data corresponding to the interface.

[0124] S104: Output each second video data.

[0125] In step S103, each second video data obtained contains only the same interface. In the present invention, the same interface can refer to the same main interface, or it can refer to the same main interface and the same sub-interface. For example, assuming that the main interface of one interface is a user-centered interface, and the interface also includes an account management sub-interface displayed in the form of a pop-up window, etc., and the main interface of another interface is a user-centered interface, but the interface does not include an account management sub-interface, then in one possible embodiment, the two interfaces are considered to be the same interface, and in another possible embodiment, the two interfaces are considered not to be the same interface.

[0126] For example, the above Figure 2 Taking the first and second moments of the GPT4.0-VISION output as an example, the captured second video data can be recorded as second video data 1, second video data 2, second video data 3, second video data 4, second video data 5, and second video data 6. The second video data 1 only contains interface A, the second video data 2 only contains interface B, the second video data 3 only contains interface C, the second video data 4 only contains interface D, the second video data 5 only contains interface E, and the second video data 6 only contains interface F.

[0127] Using the above embodiment, different users may be interested in different interfaces within a video. This interception method allows users to obtain videos of corresponding interfaces based on their needs, satisfying their personalized information needs. Instead of sifting through massive amounts of raw video data, users can directly obtain video content containing specific interfaces. For example, when viewing a product promotional video, users can quickly locate the product display interface and the product display clip of interest, saving time and effort and improving the user experience.

[0128] In another possible embodiment, in order to enable the user to more intuitively obtain video data including an interface that meets the user's needs, the method further includes:

[0129] S105: Obtain an interface name corresponding to each interface appearing in the first video data output by the target analysis model.

[0130] S106 , for each interface, output the interface name and the second video data corresponding to the interface.

[0131] For example, the above Figure 2 For example, see Figure 5 , Figure 5 This is a fourth schematic diagram of time determination in an embodiment of the present invention. After the first video data is input into GPT4.0-VISION, the first and second moments of obtaining the output of the target analysis model are:

[0132] “1. Initial interface: 00:00:00;

[0133] 2. Initial interface: 00:00:00:05.034;

[0134] 3. Video playback prompt: 00:00:00:05.034;

[0135] 4. Video playback prompt: 00:00:06;

[0136] 5. Advertising introduction: 00:00:06;

[0137] 6. Advertising introduction: 00:00:22.017;

[0138] 7. Content browsing: 00:00:22.017;

[0139] 8. Content browsing: 00:00:29.034;

[0140] 9. Personal account information: 00:00:30;

[0141] 10. Personal account information: 00:00:33.051;

[0142] 11. Member service interface: 00:00:33.051;

[0143] 12. Member service interface: 00:00:40.068.

[0144] That is, the interface name of interface A is the initial interface, the interface name of interface B is the video playback prompt, the interface name of interface C is the advertising introduction, the interface name of interface D is the content browsing, the interface name of interface E is the personal account information, and the interface name of interface F is the member service page.

[0145] The target analysis model names each interface based on knowledge learned during training. It can also extract keywords from videos or combine multiple information sources to create comprehensive naming. For example, the domain-specific knowledge and semantic information accumulated by the target analysis model during learning enables it to more accurately identify the core features and functions of interfaces across different domains and types of videos, thereby generating more meaningful names.

[0146] In addition to drawing naming criteria from training knowledge, the target analysis model can also extract key information from the audio content of the video, such as important words and prompt sounds in the dialogue, to assist in refining the interface name. Furthermore, for interfaces with specific visual elements, the model can use image recognition technology to extract key logos, text, and other elements from the screen as part of the naming target analysis model.

[0147] By applying the above embodiment, the target analysis model outputs the interface name of each interface, and for each interface, the interface name and second video data are output accordingly. This allows users to intuitively understand the specific interface to which each video clip belongs, avoiding confusion. For example, when analyzing videos of complex applications, clear interface names allow users to understand the corresponding content category at a glance. This helps users better understand and distinguish between different interfaces, allowing them to grasp the entire interaction process more quickly and accurately, reducing the difficulty for users to get started with the software and improving the user experience.

[0148] Since the interface names of the interfaces with the same function in different software are different, for example, the interface for membership recharge is named "Member Service Interface" in software A and "My" in software B. In this case, the user cannot know that "Member Service Interface" and "My" are used to indicate the same interface.

[0149] Based on this, in order to facilitate subsequent management and search by users, the interface name is determined to be the interface with the preset first interface name, and the interface name of the determined interface is modified to the second interface name corresponding to the preset first interface name.

[0150] The second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

[0151] For example, for the above example, the preset name of the first interface is "Member Service Interface" and "My", and the preset name of the second interface corresponding to the first interface name is "Member Center". Based on this, "Member Service Interface" and "My" are uniformly modified to "Member Center".

[0152] By applying the above embodiment, an additional step of determining and modifying the interface name is added. After the interface name is determined, it is modified to the corresponding second interface name. This not only helps to unify the identification of video data on the same interface, but also facilitates subsequent management and search. Users can therefore grasp the entire interaction process more quickly and accurately, reducing the difficulty for users to get started with the software and improving the user experience.

[0153] In the above content, each second video data and the corresponding relationship between the second video data and the interface and interface name are the determination results obtained by moment determination based on the first video data, and the software usage guide can be provided to the user by outputting or directly displaying the determination results.

[0154] There are many ways to output the results. You can choose the corresponding output method based on user needs or usage scenarios. The following are several possible implementation methods:

[0155] Method 1: The device displays the result determined by the time.

[0156] In this manner, the moment determination device displays a result interface, the determination interface includes a sub-interface corresponding to each interface, and the second video data corresponding to the interface is displayed in the corresponding sub-interface.

[0157] Specifically, each interface can be assigned a unique identifier, such as a number or name, and then these identifiers can be associated and stored with the corresponding second video data. When displayed, the corresponding second video data is displayed using the identifier as an index. When the second video data is displayed, it can be played in the form of a small window, or any frame of the data can be captured for display, without specific limitations here.

[0158] For example, the above Figure 2 Taking the first and second moments of the GPT4.0-VISION output as examples, it is determined that the result interface includes sub-interface 301 corresponding to interface A, sub-interface 302 corresponding to interface B, sub-interface 303 corresponding to interface C, sub-interface 304 corresponding to interface D, sub-interface 305 corresponding to interface E, and sub-interface 306 corresponding to interface F.

[0159] The second video data 1 corresponds to interface A, the second video data 2 corresponds to interface B, the second video data 3 corresponds to interface C, the second video data 4 corresponds to interface D, the second video data 5 corresponds to interface E, and the second video data 6 corresponds to interface F.

[0160] Assign a unique ID to each interface. Interface A is identified as 001, interface B is identified as 002, interface C is identified as 003, interface D is identified as 004, interface E is identified as 005, and interface F is identified as 006. Figure 6 , Figure 6This is a first schematic diagram of determining the result output in an embodiment of the present invention. In sub-interface 601, the second video data 1 is displayed with identifier 001 as the index, in sub-interface 602, the second video data 2 is displayed with identifier 002 as the index, in sub-interface 603, the second video data 3 is displayed with identifier 003 as the index, in sub-interface 604, the second video data 4 is displayed with identifier 004 as the index, in sub-interface 605, the second video data 5 is displayed with identifier 005 as the index, and in sub-interface 606, the second video data 6 is displayed with identifier 006 as the index.

[0161] Method 2: Output to a display device, and display the determination result on the display device.

[0162] In this way, the moment determination device sends the determination result to a display device for display. The display device can be any device capable of data display, including but not limited to specific display software, such as a data analysis and visualization platform (similar to professional software such as "Sky Eye"), various hardware devices, such as high-resolution large-screen all-in-one computers, and projectors.

[0163] The specific steps of displaying the determination result of the device display are the same as those of determining the determination result of the device display at the aforementioned moment, and will not be repeated here.

[0164] Method 3: Output to a display device for storage.

[0165] In this way, the moment determination device sends the determination result to the display device, which then stores the determination result locally. When the user needs to view the determination result, the determination result is retrieved from the device and displayed locally.

[0166] After obtaining the determination result, in a possible embodiment, in order to display the determination result, a determination result interface can be displayed, and the determination result interface includes a display area corresponding to each interface. For each interface, the second video data corresponding to the interface is displayed in the corresponding display area.

[0167] The specific display steps have been described in detail in the above-mentioned method 1 and will not be repeated here.

[0168] By applying the above embodiment, by dividing the result interface into display areas corresponding to each interface, the corresponding second video data is displayed in the corresponding display area for each interface, so that the user can intuitively understand the detailed process of the interface interaction, thereby making it easier for the user to know how to operate the software, reducing the difficulty for the user to get started with the software, and improving the user experience.

[0169] In step S104, video data containing the complete transition process of each interface is usually output. However, in some specific scenarios, users may focus more on the specific details of a certain interface. If the complete video data is still output at this time, the user will have to watch the entire video in order to filter out the information that meets their needs.

[0170] For example, Figure 2 Taking the embodiment shown as an example, in the scenario of membership recharge, users are often more concerned about the membership prices of different video software, while in the second video data 6, there may be only one video frame that clearly displays the membership price.

[0171] For example, a fitness instruction platform provides users with various fitness courses and exercise guides. To ensure the accuracy and scientific nature of the exercises it provides, it needs to pay attention to the exercise demonstration standards of other similar platforms. In fitness instructional videos on other platforms, only one frame clearly demonstrates the standard posture and key points of a specific exercise. The rest may be a coherent demonstration of the exercise or the instructor's explanation. This platform can capture this frame and use it as a reference to optimize its own exercise demonstration content, providing users with more professional and reliable fitness guidance.

[0172] For example, a photography tips-sharing platform provides users with shooting tips. To provide users with more practical and accurate shooting parameter setting recommendations, it is necessary to understand the shooting parameter recommendations of other similar platforms. In photography tutorial videos on other platforms, only one frame clearly demonstrates the camera shooting parameter settings for a specific scene, while the rest may be a demonstration of the shooting effect or an explanation of the shooting strategy. This platform captures this single frame to obtain valuable shooting parameter information, which it uses to improve its own tutorial content and help users better master photography techniques.

[0173] Based on this, in a possible embodiment, in order to improve the readability of the determination results that meet the user's own needs, video frames can be captured from the second video data corresponding to each interface as the target video frames corresponding to each interface.

[0174] To improve accuracy and efficiency in capturing target video frames, various methods can be employed. For example, keyword matching can be used to identify video frames containing specific information and determine the time point at which they occur, allowing the corresponding video frame to be captured. Furthermore, advanced technologies such as image recognition and machine learning can be combined to further enhance the intelligence and automation of video frame capture.

[0175] See also Figure 7 , Figure 7This is a first schematic diagram of the target video frame in an embodiment of the present invention. In video software A, the user is an old user. The left half of the picture is the intercepted target video frame containing different levels of membership and corresponding prices. The right half of the picture is the first video data belonging to the interface. The position of the progress bar below is the position of the target video frame in the first video data. The execution time is 03:00:47 on November 11, 2024, and the execution result is success.

[0176] See also Figure 8 , Figure 8 This is a second schematic diagram of the target video frame in an embodiment of the present invention. In video software B, the user is a new user. The left half of the picture is the intercepted target video frame containing prices for different levels of membership. The right half of the picture is the first video data belonging to the interface. The position of the progress bar below is the position of the target video frame in the first video data. The execution time is 3:14:10 on November 11, 2024, and the execution result is success.

[0177] In certain scenarios, users are more focused on the specific details of an interface. Outputting the complete video data for each interface requires users to spend a lot of time sifting through information, which is inefficient and provides a poor user experience. By intercepting specific video frames, users can more efficiently access specific information, improve the user experience, avoid interference from irrelevant information, and meet personalized needs.

[0178] After the target video frame is captured, the target video frame may be output. Specifically, the corresponding relationship between each second video data, each target video frame, each second video data and each interface may be output.

[0179] For example, the aforementioned Figure 6 For example, see Figure 9 , Figure 9 This is a second schematic diagram of determining the result output in an embodiment of the present invention. In the sub-interface 601, the second video data 1 and the target video frame 1 are displayed with the identifier 001 as the index; in the sub-interface 602, the second video data 2 and the target video frame 2 are displayed with the identifier 002 as the index; in the sub-interface 603, the second video data 3 and the target video frame 3 are displayed with the identifier 003 as the index; in the sub-interface 604, the second video data 4 and the target video frame 4 are displayed with the identifier 004 as the index; in the sub-interface 605, the second video data 5 and the target video frame 5 are displayed with the identifier 005 as the index; in the sub-interface 606, the second video data 6 and the target video frame 6 are displayed with the identifier 006 as the index.

[0180] In order to more clearly illustrate the time determination method provided by the present invention, the following will take the acquisition device as the MTT end, the storage device as the OSS server end, the time determination device as the DC end, and the display device as the Sky Eye as an example, and illustrate with corresponding diagrams:

[0181] See also Figure 10 , Figure 10 This is a schematic diagram of a first method for determining time in an embodiment of the present invention, including:

[0182] S1001, write test scripts on the MTT side;

[0183] S1002, the MTT terminal executes the test script;

[0184] S1003: After obtaining the first video data (ie, the resource in the figure), the resource is uploaded to the OSS server, and the OSS server stores the resource.

[0185] S1004, after uploading the resources to the OSS server, it indicates that the current test script has been successfully executed, and the MTT side will synchronize the execution results to the DC side.

[0186] S1005: The OSS server stores resources.

[0187] S1006: After receiving the execution result synchronized by the MTT end, the DC end stores the execution result.

[0188] S1007: The DC downloads resources from the OSS server.

[0189] S1008: The DC performs resource screening, which is equivalent to the aforementioned steps S1011 and S1012.

[0190] S1009: The DC performs resource analysis, i.e., determines the time and outputs the video data, which is equivalent to the aforementioned steps S103 and S104;

[0191] S1010, the DC end synchronizes the determination result to the Sky Eye, which is equivalent to method 2 in the above steps.

[0192] S1011, Tianyan stores the determination result, which is equivalent to the method 2 in the above steps;

[0193] S1012, Tianyan displays the determination results and supports screening of the determination results, which is equivalent to method 2 in the above content.

[0194] Filtering the determination results is to support users in choosing to display part of the determination results or all of the determination results according to their own needs.

[0195] See also Figure 11 , Figure 11 This is a second schematic diagram of the time determination method in an embodiment of the present invention. Taking the video membership recharge scenario as an example, the user wants to understand the membership prices, membership packages and membership recharge page structure of different software to select software that meets their needs for membership recharge.

[0196] The user enters the video software 1 through the interactive terminal to recharge the membership, connects the MTT to the interactive terminal, collects the first video data of the interface interaction process of the video software 1 in the interactive terminal, inputs the first video data to the DC end respectively to determine the first moment and the second moment in the interface interaction process, as well as the second video data, the interception of the target video frame, and finally outputs the corresponding relationship between each second video data, each target video frame, each second video data and each interface. By viewing the target video frame and the second video data corresponding to each interface, the user can understand the membership prices of different software to achieve price monitoring, understand the membership packages of different software to achieve product monitoring, and understand the membership recharge page structure of different software to achieve design monitoring. If the user is a product designer, he can monitor competitors based on the analysis results and improve his own products.

[0197] For example, the video software includes three interfaces, namely the software homepage, member center and main cashier. Figure 12 , Figure 12 This is a third schematic diagram of determining the output of the result in an embodiment of the present invention. The user can choose to perform page monitoring or price monitoring in the display device. Page monitoring includes competitor monitoring (i.e., design monitoring) and page monitoring (i.e., product monitoring). At the same time, the software name, software version, time, life cycle, and the interface to be displayed (you can choose to display all interfaces or one or more interfaces) as well as the target video frame corresponding to the interface and the number of second video data will be displayed. The life cycle is the interface interaction method corresponding to the specific function viewed by the user.

[0198] In this example, the software name is Video Software 1, the software version is September 2024 - V9.1, the user selects to display analysis results as of September 1, 2024, the lifecycle is Member, and the scenario is All. The target number of video frames for the software homepage is 3, the number of secondary video data is 1, and the total number is 4. The target number of video frames for the member center is 7, the number of secondary video data is 1, and the total number is 8. The target number of video frames for the main checkout counter is 3, the number of secondary video data is 1, and the total number is 4.

[0199] Corresponding to the aforementioned time determination method, an embodiment of the present invention further provides a time determination device, see Figure 13 , Figure 13 Schematic diagram of the structure of the time determination device in an embodiment of the present invention, including:

[0200] The data acquisition module 1301 is used to acquire first video data obtained during the interaction process of the acquisition interface.

[0201] The first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface.

[0202] The data analysis module 1302 is configured to input the first video data into a target analysis model, and obtain a first moment and a second moment corresponding to each interface appearing in the output first video data.

[0203] Among them, the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

[0204] By applying the above embodiment, by obtaining the first video data for recording the operation steps performed by the user during the interaction with the interface and the changes in the interface, and inputting it into the large language model, i.e., the target analysis model, the switching moments of each interface in the video data can be automatically and efficiently identified and extracted. Since a large language model is used for intelligent analysis, it not only saves time and energy for manual analysis, but also avoids the low accuracy caused by manual negligence, thereby reducing the labor cost of determining the switching moments of each interface during the interface interaction process and improving the accuracy of the moment determination.

[0205] In a possible implementation, the data acquisition module includes:

[0206] The first data acquisition submodule is used to acquire a plurality of initial video data obtained during the interface interaction process of the acquisition interaction terminal;

[0207] The second data acquisition submodule is used to determine video data that meets the preset video quality conditions from the multiple initial video data as the first video data; wherein the preset video quality conditions are used to determine video data that meets the video quality requirements of the target analysis model for the input video data.

[0208] In a possible implementation, the device further includes:

[0209] a text acquisition module, configured to acquire a question text input for the first video data; wherein the question text is used to guide the target analysis model to analyze and output a first moment and a second moment corresponding to each interface appearing within a target time range in the first video data;

[0210] The data analysis module includes:

[0211] The first sub-module of data analysis is used to input the first video data and the question text into the target analysis model to obtain the first moment and the second moment corresponding to each interface appearing in the output first video data; wherein, the first moment and the second moment include the first moment and the second moment corresponding to each interface appearing in the first video data within the target time range.

[0212] In a possible implementation, the device further includes:

[0213] a data interception module, configured to intercept, for each of the interfaces, video data between a first moment corresponding to the interface and a second moment corresponding to the interface in the first video data as second video data corresponding to the interface;

[0214] A data output module is used to output each of the second video data.

[0215] In a possible implementation, the device further includes:

[0216] a name acquisition module, configured to acquire an interface name corresponding to each interface appearing in the first video data output by the target analysis model;

[0217] The interface parsing module is used to output the interface name of each interface and the second video data corresponding to the interface.

[0218] In a possible implementation, the device further includes:

[0219] A name modification module is used to determine an interface whose interface name is a preset first interface name, and to modify the interface name of the determined interface to a second interface name corresponding to the preset first interface name; wherein, the second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

[0220] The embodiment of the present invention further provides an electronic device, such as Figure 14 As shown, it includes a processor 1401, a communication interface 1402, a memory 1403 and a communication bus 1404, wherein the processor 1401, the communication interface 1402, and the memory 1403 communicate with each other through the communication bus 1404.

[0221] Memory 1403, used for storing computer programs;

[0222] The processor 1401 is configured to execute the program stored in the memory 1403 by performing the following steps:

[0223] Acquire first video data obtained during the acquisition interface interaction process; wherein the first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface;

[0224] The first video data is input into a target analysis model to obtain the first moment and the second moment corresponding to each interface appearing in the output first video data; wherein the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

[0225] The communication bus mentioned in the terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or only one type of bus.

[0226] The communication interface is used for communication between the above terminal and other devices.

[0227] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0228] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0229] In another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the time determination method described in any of the above embodiments is implemented.

[0230] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is executed on a computer, the computer executes the time determination method described in any one of the above embodiments.

[0231] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0232] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0233] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0234] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for determining time, characterized in that: The method comprises: Acquire first video data obtained during the acquisition interface interaction process; wherein the first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface; The first video data is input into a target analysis model to obtain the first moment and the second moment corresponding to each interface appearing in the output first video data; wherein the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

2. The method according to claim 1, characterized in that The acquiring of the first video data obtained during the acquisition interface interaction process includes: Acquire multiple initial video data obtained during the acquisition interface interaction process; Video data that meets a preset video quality condition is determined from the multiple initial video data as the first video data; wherein the preset video quality condition is used to determine video data that meets the video quality requirement of the target analysis model for the input video data.

3. The method according to claim 1, characterized in that The method further comprises: Obtaining a question text input for the first video data; wherein the question text is used to guide the target analysis model to analyze and output a first moment and a second moment corresponding to each interface appearing within a target time range in the first video data; Inputting the first video data into a target analysis model and obtaining a first moment and a second moment corresponding to each interface appearing in the output first video data includes: The first video data and the question text are input into the target analysis model to obtain the first moment and second moment corresponding to each interface appearing in the output first video data; wherein, the first moment and the second moment include the first moment and second moment corresponding to each interface appearing in the first video data within the target time range.

4. The method according to claim 1, wherein The method further comprises: For each of the interfaces, video data between a first moment corresponding to the interface and a second moment corresponding to the interface are intercepted from the first video data as second video data corresponding to the interface; The second video data are output.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining an interface name corresponding to each interface appearing in the first video data output by the target analysis model; For each of the interfaces, the interface name of the interface and the second video data corresponding to the interface are output accordingly.

6. The method according to claim 5, characterized in that The method further comprises: Determine an interface whose interface name is a preset first interface name, and modify the interface name of the determined interface to a second interface name corresponding to the preset first interface name; wherein, the second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

7. A time determination device, characterized in that: The device comprises: A data acquisition module is used to acquire first video data obtained during the interaction process of the acquisition interface; wherein the first video data is used to record the operation steps performed by the user during the interaction with the interface and the changes of the interface; A data analysis module is used to input the first video data into a target analysis model to obtain the first moment and second moment corresponding to each interface appearing in the output first video data; wherein, the target analysis model is a large language model, the first moment is used to represent the moment of switching from other interfaces to the corresponding interface in the first video data, and the second moment is used to represent the moment of switching from the corresponding interface to other interfaces in the first video data.

8. The device according to claim 7, characterized in that The data acquisition module includes: The first data acquisition submodule is used to acquire a plurality of initial video data obtained during the interaction process of the acquisition interface; a second data acquisition submodule, configured to determine, from the plurality of initial video data, video data that satisfies a preset video quality condition as first video data; wherein the preset video quality condition is used to determine video data that meets the video quality requirement of the target analysis model for the input video data; The device further comprises: a text acquisition module, configured to acquire a question text input for the first video data; wherein the question text is used to guide the target analysis model to analyze and output a first moment and a second moment corresponding to each interface appearing within a target time range in the first video data; The data analysis module includes: A first data analysis submodule is configured to input the first video data and the question text into a target analysis model, and obtain a first moment and a second moment corresponding to each interface appearing in the output first video data; wherein the first moment and the second moment include the first moment and the second moment corresponding to each interface appearing in the first video data within the target time range; The device further comprises: a data interception module, configured to intercept, for each of the interfaces, video data between a first moment corresponding to the interface and a second moment corresponding to the interface in the first video data as second video data corresponding to the interface; a data output module, configured to output each of the second video data; The device further comprises: a name acquisition module, configured to acquire an interface name corresponding to each interface appearing in the first video data output by the target analysis model; An interface parsing module, configured to output, for each of the interfaces, the interface name and the second video data corresponding to the interface; The device further comprises: A name modification module is used to determine an interface whose interface name is a preset first interface name, and to modify the interface name of the determined interface to a second interface name corresponding to the preset first interface name; wherein, the second interface name and the preset first interface name are used to identify the same interface, and the interface name used to identify the same interface is modified to the same second interface name.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory for storing computer programs; A processor, configured to implement the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps of any one of claims 1 to 6 are implemented.

Images