Video-based information query method, device, equipment and storage medium

Abstract

The present application discloses a video-based information query method, device, equipment and storage medium, and relates to the field of intelligent search technology. The specific implementation scheme is: by acquiring the target video screen, determining the text recognition result of the target video screen, and the text recognition result includes: the text information in the target video screen, and finally based on the text recognition result, obtaining the information query result corresponding to the above text information, and displaying it. In this technical scheme, the information query result corresponding to the text information is queried based on the text recognition result of the target video screen, without the user switching the terminal interface, nor the user manually entering the text to be queried, which improves the query efficiency and user experience of the video text.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular, to a video-based information query method, device, equipment, and storage medium, which can be used in the field of intelligent search technology. Background Art

[0002] In daily life, when users watch videos on mobile phones, tablet computers and other terminal devices, they may be interested in certain texts in the video being played (for example, road signs, billboards, book titles, subtitles, etc.). At this time, in order to understand the relevant knowledge of the text of interest, it is usually necessary to use a search engine to query the relevant information of the text.

[0003] In the prior art, the main method for querying text in a video is: the user remembers the text to be searched that the user is interested in, then controls the terminal device to switch from the video player page to the search page, enters the above text to be searched in the search page, and then obtains relevant information about the text to be searched, and finally switches back to the video player page.

[0004] However, the above solution requires the user to control the terminal device to perform page switching and jumping, the operation process is cumbersome, and the text to be searched needs to be manually input by the user, resulting in low user search efficiency and poor experience. Summary of the invention

[0005] The embodiments of the present application provide a video-based information query method, apparatus, device and storage medium, which are used to solve the problem of low user search efficiency in existing video text search queries.

[0006] In a first aspect, an embodiment of the present application provides a method for querying video information, including:

[0007] Get the target video screen;

[0008] Determine a text recognition result of the target video screen, wherein the text recognition result includes: text information in the target video screen;

[0009] Based on the text recognition result, obtaining the information query result corresponding to the text information;

[0010] The information query result is displayed.

[0011] In a second aspect, an embodiment of the present application provides a video information query method, comprising:

[0012] receiving a text query request from a terminal device, the text query request comprising: text information, the text information being text obtained by performing text recognition on a target video screen;

[0013] Obtaining information query results corresponding to the text information;

[0014] Send the information query result to the terminal device.

[0015] In a third aspect, an embodiment of the present application provides a video-based information query device, including: an acquisition module, a processing module, and a display module;

[0016] The acquisition module is used to acquire the target video picture;

[0017] The processing module is used to determine the text recognition result of the target video screen, the text recognition result includes: the text information in the target video screen, and based on the text recognition result, obtain the information query result corresponding to the text information;

[0018] The display module is used to display the information query result.

[0019] In a fourth aspect, an embodiment of the present application provides a video-based information query device, including: a receiving module, a processing module and a sending module;

[0020] The receiving module is used to receive a text query request from a terminal device, wherein the text query request includes: text information, wherein the text information is text obtained by performing text recognition on a target video screen;

[0021] The processing module is used to obtain the information query result corresponding to the text information;

[0022] The sending module is used to send the information query result to the terminal device.

[0023] In a fifth aspect, an embodiment of the present application provides a terminal device, including:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0027] In a sixth aspect, an embodiment of the present application provides a server, including:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the second aspect.

[0031] In a seventh aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0032] In an eighth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0033] In a ninth aspect, an embodiment of the present application provides a video-based information query method, comprising:

[0034] Acquire a text recognition result of a target video screen, wherein the text recognition result includes: text information in the target video screen;

[0035] Based on the text recognition result, an information query result corresponding to the text information is determined.

[0036] The video-based information query method, device, equipment and storage medium provided in the embodiments of the present application obtain a target video screen, determine the text recognition result of the target video screen, and the text recognition result includes: the text information in the target video screen, and finally obtain the information query result corresponding to the text information based on the text recognition result, and display it. In this technical solution, the information query result corresponding to the text to be queried is queried based on the text recognition result of the target video screen, without the user switching the terminal interface or manually inputting the text to be queried, thereby improving the query efficiency and user experience of the video text.

[0037] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0039] Figure 1 It is a schematic diagram of an application scenario of the video-based information query method provided by the present application;

[0040] Figure 2 is a flowchart of a video-based information query method provided in the first embodiment of the present application;

[0041] Figure 3is an interactive schematic diagram of a video-based information query method provided in the second embodiment of the present application;

[0042] Figure 4 It is a schematic diagram of a terminal device presenting a target video screen through a current playback interface in an embodiment of the present application;

[0043] Figure 5 is an interactive schematic diagram of a video-based information query method provided in the third embodiment of the present application;

[0044] Figure 6 is a schematic diagram of presenting a processed target video screen through a display interface of a terminal device in an embodiment of the present application;

[0045] Figure 7 is a schematic diagram of a terminal device selecting a text to be queried based on a text selection indication by a user;

[0046] Figure 8 is a flowchart of a video-based information query method provided in the fourth embodiment of the present application;

[0047] Fig. 9 is a schematic diagram of an interface for displaying information query results in a pop-up window in an embodiment of the present application;

[0048] Fig.10 It is an information interaction block diagram of a video-based information query method provided in an embodiment of the present application;

[0049] Fig.11 is a structural diagram of a video-based information query device provided in the first embodiment of the present application;

[0050] Fig.12 is a schematic diagram of the structure of a video-based information query device provided in the second embodiment of the present application;

[0051] Fig.13 is a block diagram of a terminal device used to implement the video-based information query method provided in an embodiment of the present application;

[0052] Fig.14 It is a block diagram of a server used to implement the video-based information query method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0054] Before introducing the technical solution of the present application, the terms involved in the embodiments of the present application are first explained:

[0055] OCR text recognition

[0056] Optical character recognition (OCR) refers to the process of analyzing and identifying image files of text materials to obtain text and layout information. In other words, the text in the image is recognized and returned in the form of text. A typical OCR technical solution can be divided into two parts: text detection and text recognition. Text detection is to detect the location, range and layout of the text in the image, and usually also includes layout analysis and text line detection. Text detection mainly determines which positions in the image have text and how large the range of the text is. Text recognition is to recognize the text content based on text detection and convert the text information in the image into text information. Text recognition mainly determines what each character detected by text is.

[0057] Search Engines

[0058] The so-called search engine is a search technology that uses specific strategies to retrieve specified information from the Internet and feed it back to users based on user needs and certain algorithms. Search engines rely on a variety of technologies, such as web crawler technology, search and ranking technology, web page processing technology, big data processing technology, natural language processing technology, etc., to provide fast and highly relevant information services for information retrieval users. The core modules of search engine technology generally include crawlers, indexing, retrieval and sorting, etc., and a series of other auxiliary modules can be added to create a better network usage environment for users.

[0059] This application mainly utilizes the above-mentioned OCR text recognition and search engine technology to achieve the task of searching for text in the target video, without limiting its specific implementation principle.

[0060] Before introducing the technical solution of the present application, the application scenarios of the embodiments of the present application are briefly introduced:

[0061] With the development of terminal devices, the application fields of videos are becoming more and more extensive. When users watch videos on terminal devices, they are sometimes interested in the texts (road signs, billboards, book names, some texts in subtitles, etc.) that appear in the videos. If they want to learn more about the relevant information, they need to search for content related to these texts on search engines.

[0062] With respect to the video text query method in the prior art, the user remembers the text to be searched, then controls the terminal device to switch from the video player page to the search app or search engine website page, and then enters the text to be searched in the search app or search engine website page to obtain relevant information. However, this method requires the user to operate the terminal device to jump from the video player page to the search engine app or search engine website page, and needs to manually enter the relevant text for search. The operation process is cumbersome, and when the phrase to be searched is relatively uncommon, it is inconvenient for the user to input, which will affect the efficiency and experience of the user's search. Therefore, in order to facilitate the user, the embodiment of the present application provides a solution that can recognize the text appearing in the video, and then jump to the search app or web page for search display, so as to facilitate the user's acquisition of information.

[0063] The video-based information query method provided by the present application has a typical application scenario of searching for related content based on the text in the video playback screen. The overall idea is: when it is necessary to identify the text in the video, the terminal device sends the video screen carrying the text to the server for OCR text recognition, and queries the information based on the recognized text included in the OCR text recognition result. Among them, the terminal device can also present the text at the corresponding position of the video screen on the video interface, obtain the selected phrase to be searched, and after receiving the user's search instruction, send the selected phrase to be searched to the search engine, obtain the search results, and superimpose a pop-up window on the video interface for display; finally, based on the user's pop-up window closing instruction, close the pop-up window, return to the video playback page to continue playing the video.

[0064] For example, Figure 1 Schematic diagram of the application scenario of the video-based information query method provided by this application. Figure 1 As shown, the application scenario may include: at least one terminal device ( Figure 1 Three terminal devices are shown, namely terminal device 111, terminal device 112, and terminal device 113, a network 12, and a server 13. Each terminal device and the server 13 can communicate through the network 12.

[0065] For example, in Figure 1 In the application scenario shown, the server 13 can receive a text recognition request sent by a user through a terminal device through the network 12, process the target video image included in the text recognition request, and return the obtained text recognition result to the terminal device through the network 12.

[0066] The server 13 can also receive a text query request sent by a user through a terminal device through the network 12, search the text to be queried carried in the text query request, obtain an information query result corresponding to the text to be queried, and return it to the terminal device through the network 12.

[0067] It should be noted that the attached Figure 1 This is only a schematic diagram of an application scenario provided by the embodiment of the present application. Figure 1 does not limit the equipment included in Figure 1 The positional relationship between the devices is limited and can be set according to actual needs.

[0068] In practical applications, a terminal device is a device that inputs programs and data into a computer or receives computer output processing results via communication facilities. It is usually set up in a convenient place where it can use communication facilities to connect to a remote computer for work. It is mainly composed of a communication interface control device and a dedicated or selected input and output device. Exemplarily, the terminal device of the embodiment of the present application is a terminal with a display screen and capable of performing video playback through the display, such as a mobile phone, a tablet computer, a smart TV and other devices.

[0069] The server is a cloud server that provides computing or application services for other clients (such as PCs, smart phones, ATMs and other terminals, and even large devices such as train systems) in the network. In the embodiment of the present application, the server has an image and text recognition function, and can query related content through a search engine.

[0070] It is understandable that the embodiments of the present application do not limit the specific implementation and functions of the terminal device and the server, which can be determined according to actual needs and will not be repeated here.

[0071] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0072] Figure 2 1 is a flowchart of a video-based information query method provided in the first embodiment of the present application. For example, the present application embodiment is explained with the execution subject being a terminal device. Figure 2 As shown, the method may include the following steps:

[0073] S201, obtaining a target video image.

[0074] Video text is composed of images, and each video screen is a frame of image. In actual applications, when a user watches a target video through a terminal device, if there is text that the user is interested in on the current video interface, the user can operate the terminal device to enable the terminal device to obtain the current video interface, that is, the target video screen.

[0075] In the embodiments of the present application, there are multiple ways for the terminal device to obtain the target video screen. For example, one way is: after the video is paused, the background obtains the current video interface on the pause interface and uses it as the target video screen; another way is: based on the user's screenshot instruction, the target video screen corresponding to the current video interface is obtained. The embodiments of the present application do not limit the method for obtaining the target video screen, which can be determined according to actual needs and will not be repeated here.

[0076] As an example, S201 may be implemented through the following steps:

[0077] According to the user's video pause request, the target video is paused; and the target video screen corresponding to the target video displayed on the current playback interface is obtained.

[0078] As another example, S201 may be implemented through the following steps:

[0079] According to the video screenshot instruction of the user, the target video screen is obtained, and the target video screen is the screen of the video currently being played by the terminal device.

[0080] S202: Determine a text recognition result of a target video picture, where the text recognition result includes: text information in the target video picture.

[0081] In an embodiment of the present application, after the terminal device acquires the target video picture, it is necessary to perform text recognition on the target video picture to determine the text recognition result of the target video picture.

[0082] In a possible design of the present application, if the terminal device has a text recognition capability, the terminal device can recognize the text in the target video screen, thereby obtaining the text information in the target video screen.

[0083] In another possible design of the present application, if the terminal device does not have the ability to recognize text, then the terminal device can send the target video screen to the server, and the server will recognize the text in the target video screen and transmit the text information in the target video screen to the terminal device after obtaining it.

[0084] Optionally, in the above possible designs of the present application, the terminal device or server usually adopts OCR text recognition technology to recognize text on the target video screen. It can also adopt other methods. The embodiment of the present application does not limit the method of text recognition, which can be selected according to actual needs and will not be repeated here.

[0085] S203: Based on the text recognition result, obtain the information query result corresponding to the text information.

[0086] In an embodiment of the present application, after the terminal device obtains the above text recognition result, in one possible design, the terminal device can directly perform information retrieval based on the text information in the text recognition result. In another possible design, the terminal device can first display it on the display interface of the terminal device for the user to operate it. For example, the terminal device can receive a text selection instruction issued by the user based on the above text recognition result; secondly, based on the text selection instruction, select the text to be queried in the above text information, and finally perform information retrieval on the text to be queried.

[0087] Optionally, information retrieval can be understood as a process of querying text information or text to be queried through a search engine, and then obtaining information query results corresponding to the text information or text to be queried.

[0088] Normally, in order to obtain comprehensive query information, the terminal device usually sends text information or text to be queried to the server, and the server searches for relevant information of the text information or text to be queried in the vocabulary of the entire network, and after obtaining the corresponding information query result, sends it to the terminal device.

[0089] S204: Display the above information query result.

[0090] In an embodiment of the present application, after the terminal device obtains the above-mentioned information query results, it displays them on the display interface of the terminal device. For example, without switching the video playback interface, the terminal device can display the information query results in the form of a pop-up page, so that the user can understand the relevant query information corresponding to the text in the target video screen.

[0091] The video-based information query method provided by the embodiment of the present application obtains the target video screen, determines the text recognition result of the target video screen, and the text recognition result includes: the text information in the target video screen, and finally obtains the information query result corresponding to the text information based on the text recognition result, and displays it. In this technical solution, the information query result corresponding to the text information is queried based on the text recognition result of the target video screen, without the user switching the terminal interface or manually inputting the text to be queried, thereby improving the query efficiency and user experience of the video text.

[0092] Based on the above embodiments, Figure 3 2 is an interactive diagram of a video-based information query method provided by the second embodiment of the present application. The present application embodiment is described by the information interaction between the terminal device and the server. Figure 3 As shown, in this embodiment, the above S202 can be implemented by the following steps:

[0093] S301. The terminal device sends a text recognition request to a server, where the text recognition request carries a target video picture.

[0094] In an embodiment of the present application, after the terminal device obtains the target video screen, it can transmit it to the server for text recognition. Exemplarily, a text recognition mark is set on the current playback interface of the terminal device. When the terminal device receives a user's click operation on the mark, it can transmit a text recognition request carrying the target video screen to the server for text recognition.

[0095] For example, Figure 4 Schematic diagram of the terminal device presenting the target video screen through the current playback interface in the embodiment of the present application. If the target video currently played by the terminal device mainly tells about "beautiful mountains and rivers", the video background is "A Lake". Therefore, referring to Figure 4 As shown, the target video screen has text information "Beautiful Mountains and Rivers" and "Lake A". At the same time, the playback display interface of the terminal device has a "text recognition" logo (or button) and a search logo (or button).

[0096] For example, the terminal device pauses the playback of the video on the video APP of the terminal device according to the pause instruction of the user. At this time, the playback interface of the terminal device stops at the screen where the user wants to search for text in the video, that is, the target video screen, such as Figure 4 At this time, if the user clicks the "Recognize Text" button on the current playback interface, the terminal device will send a screenshot of the target video screen of the current interface to the server through the video APP, specifically, to the OCR text recognition cloud service.

[0097] S302: The server performs text recognition on the received target video picture to obtain a text recognition result, which includes: text information in the target video picture.

[0098] In this embodiment, after receiving a text recognition request carrying a target video picture from a terminal device, the server performs text recognition on the target video picture through an OCR text recognition method, determines the text information in the target video picture, and obtains a text recognition result.

[0099] Optionally, the text recognition result may include not only the text information in the target video screen, but also the location information of the text information.

[0100] For example, when receiving the target video screen, the OCR text recognition cloud service on the server can recognize the text in the target video screen and determine the location information of the text, that is, the coordinate information of the recognized text in the target video screen.

[0101] S303: The server sends the text recognition result of the target video image to the terminal device.

[0102] Optionally, after obtaining the text recognition result of the target video screen, the server may transmit the text recognition result to the terminal device so that the terminal device displays it.

[0103] In an embodiment of the present application, a server is used to perform text recognition on a target video screen, and the text recognition result is fed back to a terminal device. This not only realizes the recognition of image text, but also improves the efficiency of text recognition, laying the foundation for subsequent automatic query of the text to be queried in the target video screen.

[0104] The specific implementation of the server recognizing the text in the target video screen can be determined according to actual needs and will not be described in detail here.

[0105] In the video-based information query method provided in the embodiment of the present application, the terminal device sends a text recognition request carrying a target video screen to the server, and the server performs text recognition on the received target video screen, obtains a text recognition result, and feeds it back to the terminal device. That is, the server realizes the recognition of image text with high recognition efficiency, which lays a foundation for automatic query of the text to be queried in the target video screen.

[0106] Exemplarily, based on the above embodiments, Figure 5 : is an interactive diagram of the video-based information query method provided by the third embodiment of the present application. The present application embodiment is described by the information interaction between the terminal device and the server. In the embodiment of the present application, the text recognition result also includes: the location information of the text information. Therefore, after the terminal device determines the text recognition result of the target video screen, it can also display it. For example, refer to Figure 5 As shown, the method may also include the following steps:

[0107] S501. The terminal device overlays the text information on a corresponding position of a target video screen according to the position information of the text information to obtain a processed target video screen.

[0108] In an embodiment of the present application, after the terminal device receives the text recognition result of the target video screen from the server, in order to improve the user's visual experience, the terminal device can superimpose the above text information on the corresponding position of the target video screen according to the position information of the text information in the text recognition result, that is, superimpose the text information at each position in the target video screen on the corresponding position of the target video screen to obtain the processed target video screen.

[0109] S502: Present the processed target video image on the display interface of the terminal device.

[0110] Exemplarily, after the terminal device processes the text recognition result to obtain the processed target video image, it can present it on the display interface of the terminal device so that the user can process it.

[0111] By superimposing the text information on the corresponding position of the target video screen and displaying it on the display interface, users can accurately and clearly distinguish the different text information included in the text recognition results, making it possible for subsequent users to select text information at different positions.

[0112] For example, Figure 6 Schematic diagram of presenting a processed target video screen through a display interface of a terminal device in an embodiment of the present application. Figure 6 is in the above Figure 4 Based on the implementation of Figure 6 As shown, in this embodiment, the text information "Beautiful Mountains and Rivers" and "Lake A" on the target video screen are respectively superimposed on corresponding positions of the target video screen, and the "Beautiful Mountains and Rivers" and "Lake A" are both texts that can be selected.

[0113] Exemplarily, the video APP of the terminal device obtains the text information of each position in the target video screen, that is, after determining the text information in the target video screen and the coordinate information corresponding to the text information, it is displayed on the interface of the video APP, such as Figure 6 shown.

[0114] It is understood that in the embodiment of the present application, the coordinate information of the text information is determined in units of pixels of the target video screen, with the upper left corner as the origin of the coordinate system. When displayed, the text recognition result can correspond to the approximate position of the text in the target video screen.

[0115] The terminal device superimposes the text information in the text recognition result on the corresponding position of the target video screen to obtain a processed target video screen, and presents the processed target video screen on the display interface of the terminal device, so that the user can accurately and clearly distinguish the different text information included in the text recognition result, and provides the possibility for subsequent users to select text information in different positions.

[0116] Further, in the embodiments of the present application, referring to Figure 5 As shown, the method may also include the following steps:

[0117] S503: The terminal device obtains a text selection instruction issued by the user according to the text recognition result.

[0118] The text selection indication is used to indicate the text to be queried in the selected text information.

[0119] In an embodiment of the present application, after the terminal device presents the acquired text recognition results on the display interface, the user can use the text recognition results to determine the text of interest (referred to as the text to be queried in this embodiment) and issue a text selection indication to instruct the terminal device to select it.

[0120] It is understandable that the user can select the text to be queried in a variety of ways, for example, by touching or clicking certain text displayed on the display interface to issue a text selection indication, or by voice. The embodiment of the present application does not limit the specific way in which the user issues a text selection indication to select the text to be queried, which can be determined according to the actual scenario and will not be repeated here.

[0121] Further, in the embodiments of the present application, referring to Figure 5 As shown, the above S203 can be implemented by the following steps:

[0122] S504: The terminal device selects the text to be queried in the text recognition results according to the text selection instruction.

[0123] In an embodiment of the present application, after the terminal device obtains the text selection indication from the user, it can perform a selection operation on the text to be queried, that is, select the text to be queried indicated by the user based on the text selection indication.

[0124] For example, Figure 7 It is a schematic diagram of the terminal device selecting the text to be queried based on the user's text selection indication. Figure 7 As shown, assuming that the user is interested in "Lake A" in the target video screen, Lake A is selected and marked with a dark shadow.

[0125] S505: The terminal device obtains a text search instruction from the user, and generates a text query request according to the text search instruction, where the text query request carries the selected text to be queried.

[0126] In an embodiment of the present application, the terminal device can detect the user's operation in real time and perform corresponding operations based on the user's operation. For example, after the terminal device selects the text to be queried, when the user issues a text search instruction by clicking a search mark on the display interface or by voice, the terminal device can obtain the user's text search instruction and then generate a text query request based on the selected text to be queried.

[0127] S506: The terminal device sends a text query request to the server.

[0128] In an embodiment of the present application, since the content collected on the Internet is relatively complete, the server can provide query services through a search engine on the Internet. Its processing capability is relatively strong and the scope of query is relatively wide. Therefore, the terminal device sends the text query request to the server, so that the server performs the query operation.

[0129] For example, refer to the above Figure 7 As shown, if the user wants to search for a certain text, he needs to select the relevant text, that is, select the text to be queried. After the user selects the text to be queried, he can click the search icon on the current display interface to send the text query request to the server, that is, the video APP will send the text to be queried to the search engine interface.

[0130] S507: The server obtains information query results corresponding to the text information according to the received text query request.

[0131] In the embodiment of the present application, after receiving the text query request, the server can query the network according to the text information in the text query request to obtain the information query result corresponding to the text information. It can be understood that the text information is the text obtained by performing text recognition on the target video screen, and can also be the text selected by the terminal device according to the text selection result of the user.

[0132] Exemplarily, the server queries and retrieves information on the query text through a search engine to obtain information query results. Specifically, the search engine retrieves relevant content based on the query text in the text query request to obtain information query results.

[0133] S508: The server sends the information query result to the terminal device.

[0134] In this embodiment, after the server obtains the information query result corresponding to the text to be queried, it can send it to the terminal device so that the terminal device can display the obtained information query result.

[0135] The video-based information query method provided in the embodiment of the present application, the terminal device obtains the text selection indication issued by the user based on the text recognition result, and selects the text to be queried in the text recognition result according to the text selection indication, and obtains the user's text search indication, and generates a text query request according to the text search indication, and sends it to the server. The server obtains the information query result corresponding to the text to be queried according to the received text query request, and feeds it back to the terminal device. The purpose of information query is achieved without the terminal device switching the display page, thereby improving the user search efficiency and user experience.

[0136] Exemplarily, based on the above embodiments, Figure 8 4 is a flowchart of a video-based information query method provided in the fourth embodiment of the present application. The present application embodiment is described with a terminal device as the execution subject. Figure 8 As shown, in this embodiment, the above S204 can be implemented by the following steps:

[0137] S801. Display the information query result in a pop-up window on the display interface of the terminal device.

[0138] In an embodiment of the present application, one implementation method for a terminal device to display information query results is to display them in a pop-up window, which is similar to the result presentation method obtained by a user performing a target query on a search page, making it convenient for the user to operate on it.

[0139] Further, in the embodiments of the present application, referring to Figure 8 As shown, the method may also include the following steps:

[0140] S802: Process the above information query result according to the pop-up window operation instruction of the user.

[0141] The pop-up window operation instruction includes at least one of the following operations: a pop-up window closing instruction, a pop-up window page turning instruction, and a page sliding down instruction.

[0142] For example, when the terminal device displays the information query results on the display interface of the terminal device in a pop-up window manner, the user can perform some processing operations on the information query results in the pop-up window page, for example, the terminal device performs operations such as sliding down and turning pages on the pop-up window page according to the user operation to browse the information and click on the search results. After the user obtains the information, the user can also issue a pop-up window closing instruction to remove the pop-up window page and return to the video playback page to continue playing the video.

[0143] For example, when the pop-up operation instruction is a pop-up closing instruction, the terminal device can close the pop-up window displaying the information query results according to the user's pop-up closing instruction, thereby controlling the terminal device to continue playing the paused target video on the display interface.

[0144] For example, Fig. 9 Schematic diagram of an interface for displaying information query results in a pop-up window in an embodiment of the present application. Fig. 9 As shown, after receiving the information query results, the video APP of the terminal device displays the results in a pop-up window. The user can scroll down or turn the page, and can continue to interact with the search engine to obtain the content of the next page.

[0145] The specific operations that users perform on the above information query results can be determined based on actual needs and will not be elaborated here.

[0146] In the video-based information query method provided by the embodiment of the present application, after obtaining the information query result of the text to be queried, the terminal device can display the information query result in a pop-up window on the display interface of the terminal device, and then process the above information query result according to the pop-up window operation instruction of the user. This technical solution can also realize the query operation of the text in the target video without the user switching the display page of the terminal device, simplifying the query operation process and improving the efficiency of video text query.

[0147] It can be seen from the above embodiments that Fig.10 is an information interaction block diagram of the video-based information query method provided by the embodiment of the present application. Fig.10 As shown, the technical solution of the present application is mainly: for the video played in the mobile application of the terminal device, when the text (including subtitles) that the user is interested in appears in the video screen, the user can pause the video playback (or screenshot), click the quick search button, and then the video app will send the acquired target video screen to the cloud OCR text recognition service of the server, so that it can recognize the text and its position in the video, obtain the text recognition result (text information at each position), and then feed it back to the terminal device, so that the terminal device displays the text at the corresponding position of the target video screen. In the process of displaying the text recognition result on the display interface of the terminal device, the user long presses the text, and a cursor for selecting the text will appear. The start and start positions can be moved left and right, and then the search button on the display interface is clicked. The terminal device will send the text to be queried (selected text) to the search engine of the server for information query through the video APP. After obtaining the information query result, it will be fed back to the terminal device and displayed through a pop-up window.

[0148] In an embodiment of the present application, the operation of initiating a search by a user is faster and more convenient. There is no need to remember the keywords to be searched, nor is there any need to manually enter text. During the search process, the search results can be obtained by staying on the current interface of the current video app. There is no need to switch to other apps or open a browser to search. Furthermore, after obtaining the search results, the pop-up page can be closed and the video can be continued to be watched, providing a smoother experience.

[0149] The above describes the specific implementation of the data processing method mentioned in the embodiment of the present application. The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0150] Fig.11 1 is a schematic diagram of the structure of a video-based information query device provided in the first embodiment of the present application. The device can be integrated into a terminal device or implemented through a terminal device. Fig.11 As shown, in this embodiment, the video-based information query device 110 may include: an acquisition module 1101 , a processing module 1102 and a display module 1103 .

[0151] The acquisition module 1101 is used to acquire the target video picture;

[0152] The processing module 1102 is used to determine the text recognition result of the target video screen, the text recognition result includes: the text information in the target video screen, and based on the text recognition result, obtain the information query result corresponding to the text information;

[0153] The display module 1103 is used to display the information query result.

[0154] Optional, such as Fig.11 As shown, in the embodiment of the present application, the device further includes: a sending module 1104 and a receiving module 1105.

[0155] Optionally, in a possible design of the present application, the processing module 1102 is used to determine the text recognition result of the target video screen, specifically:

[0156] The processing module 1102 is specifically configured to send a text recognition request to a server through the sending module 1104 , wherein the text recognition request carries the target video picture, and receive a text recognition result of the target video picture from the server through the receiving module 1105 .

[0157] In another possible design of the present application, the text recognition result further includes: position information of the text information; the processing module 1102 is further configured to superimpose the text information on a corresponding position of the target video screen according to the position information of the text information, so as to obtain a processed target video screen;

[0158] The display module 1103 is also used to present the processed target video image on the display interface of the terminal device.

[0159] Optionally, in another possible design of the present application, the acquisition module 1101 is further used to obtain a text selection indication issued by the user according to the text recognition result, wherein the text selection indication is used to indicate the selection of the text to be queried in the text information;

[0160] Correspondingly, the processing module 1102 is used to obtain the information query result corresponding to the text information based on the text recognition result, specifically:

[0161] The processing module 1102 is specifically configured to:

[0162] According to the text selection indication, selecting the text to be queried in the text recognition result;

[0163] Acquire a text search instruction from a user, and generate a text query request according to the text search instruction, wherein the text query request carries the selected text to be queried;

[0164] The text query request is sent to the server through the sending module 1104 , and the information query result corresponding to the text to be queried is received from the server through the receiving module 1105 .

[0165] Optionally, in another possible design of the present application, the display module 1103 is specifically used to display the information query result in a pop-up window on the display interface of the terminal device.

[0166] Exemplarily, in an embodiment of the present application, the processing module 1102 is also used to process the information query result according to the user's pop-up operation instruction, and the pop-up operation instruction includes at least one of the following operations: a pop-up closing instruction, a pop-up page turning instruction, and a page sliding down instruction.

[0167] Optionally, in another possible design of the present application, the acquisition module 1101 is specifically used to pause the playback of the target video according to the user's video pause request, and obtain the target video screen corresponding to the target video displayed on the current playback interface.

[0168] Optionally, in another possible design of the present application, the acquisition module 1101 is specifically used to acquire the target video screen according to the user's video screenshot instruction, and the target video screen is the screen of the video currently playing on the terminal device.

[0169] The device provided in the embodiment of the present application can be used to execute the solution of the terminal device in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0170] Fig.12 2 is a schematic diagram of the structure of a video-based information query device provided in the second embodiment of the present application. The device can be integrated into a server or implemented by a server. Fig.12 As shown, in this embodiment, the video-based information query device 120 may include: a receiving module 1201 , a processing module 1202 and a sending module 1203 .

[0171] The receiving module 1201 is used to receive a text query request from a terminal device, wherein the text query request includes: text information, which is text obtained by performing text recognition on a target video screen;

[0172] The processing module 1202 is used to obtain the information query result corresponding to the text to be queried;

[0173] The sending module 1203 is used to send the information query result to the terminal device.

[0174] In one embodiment of the present application, the receiving module 1201 is further configured to receive a text recognition request from the terminal device before receiving the text query request from the terminal device, wherein the text recognition request carries the target video picture;

[0175] The processing module 1202 is further used to perform text recognition on the target video screen to obtain a text recognition result, wherein the text recognition result includes: text information in the target video screen;

[0176] The sending module 1203 is further configured to send the text recognition result to the terminal device.

[0177] The device provided in the embodiment of the present application can be used to execute the server solution in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be repeated here.

[0178] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0179] Furthermore, according to an embodiment of the present application, the present application also provides a terminal device, a server and a computer-readable storage medium.

[0180] Fig.13 It is a block diagram of a terminal device for implementing the video-based information query method provided in the embodiment of the present application. In the embodiment of the present application, the terminal device can represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices and other similar computing devices, which are terminals with video playback capabilities.

[0181] For example, Fig.13 As shown, the terminal device may include: at least one processor 1301, and a memory 1302 that is communicatively connected to the at least one processor; wherein the memory 1302 stores instructions that can be executed by the at least one processor 1301, and the instructions are executed by the at least one processor 1301 so that the at least one processor 1301 can execute the above Figures 2 to 10 The scheme of the terminal device in the embodiment shown.

[0182] Optionally, in an embodiment of the present application, the terminal device may further include: an input device 1303 and an output device 1304. The processor 1301, the memory 1302, the input device 1303 and the output device 1304 may be connected via a bus or other means. Fig.13 The example of connecting through bus is taken in the following.

[0183] Furthermore, in an embodiment of the present application, the terminal device also includes a display 1305, and the display 1305 is used to display the played video and display the information query results corresponding to the text information.

[0184] Fig.14 It is a block diagram of a server for implementing the video-based information query method provided by the embodiment of the present application. In the embodiment of the present application, the server is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0185] For example, Fig.14 As shown, the server may include: at least one processor 1401, and a memory 1402 that is communicatively connected to the at least one processor; wherein the memory 1402 stores instructions that can be executed by the at least one processor 1401, and the instructions are executed by the at least one processor 1401 so that the at least one processor 1401 can execute the above Figures 2 to 10 The server scheme in the illustrated embodiment.

[0186] Optionally, in an embodiment of the present application, the server may further include: an input device 1403 and an output device 1404. The processor 1401, the memory 1402, the input device 1403 and the output device 1404 may be connected via a bus or other means. Fig.14 The example of connecting through bus is taken in the following.

[0187] It is understandable that the above Fig.13 and Fig.14 The components shown, their connections and relationships, and their functions, are merely examples, and are not meant to limit implementations of the application described and / or claimed herein.

[0188] In the above Fig.13 and Fig.14 As shown in the schematic diagram, the terminal device and the server can also include interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on a memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0189] In the above Fig.13 and Fig.14 The schematic diagram shown in the figure shows that the memory is the non-transient computer-readable storage medium provided by the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the method provided by the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the method provided by the present application.

[0190] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the video-based information query method in the embodiment of the present application (for example, the memory 1302 corresponds to the attached Fig.11 The acquisition module 1101, the processing module 1102, the display module 1103, the sending module 1104 and the receiving module 1105 shown in the figure; the memory 1402 corresponds to the attached Fig.12 The processor executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory, that is, implements the method in the above method embodiment.

[0191] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the terminal device and / or the server, etc. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the terminal device and / or the server via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0192] The input device can receive input digital or character information, and generate key signal input related to user settings and function control of the terminal device and / or server, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator bar, one or more mouse buttons, a trackball, a joystick and other input devices. The output device may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0193] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0194] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0195] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0196] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0197] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0198] Furthermore, the embodiment of the present application also provides a video-based information query method, comprising:

[0199] Acquire a text recognition result of a target video screen, wherein the text recognition result includes: text information in the target video screen;

[0200] Based on the text recognition result, an information query result corresponding to the text information is determined.

[0201] According to the technical solution of the embodiment of the present application, by determining the text recognition result of the target video screen, the text recognition result includes: the text information in the target video screen, and based on the text recognition result, the information query result corresponding to the text information is obtained. In this technical solution, the information query result corresponding to the text to be queried is queried based on the text recognition result of the target video screen, without the user switching the terminal interface or manually inputting the text to be queried, thereby improving the query efficiency and user experience of the video text.

[0202] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0203] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. A video-based information query method, characterized in that: include: Get the target video screen; In response to a user triggering an operation on a recognition control on a video playback interface, sending a text recognition request to a server; The text recognition request carries the target video picture; So that the server can recognize the text in the target video screen and obtain the text recognition result of the target video screen; Receiving a text recognition result of the target video screen sent by the server, wherein the text recognition result includes: text information in the target video screen and location information of the text information; According to the position information of the text information, the text information is superimposed on the corresponding position of the target video picture to obtain a processed target video picture; Presenting the processed target video image on a display interface of a terminal device; In response to a user's long press operation on the text information, a cursor for selecting text is displayed; based on the user's movement of the cursor, a starting position and an ending position of the text to be queried are determined in the text information to obtain the text to be queried; In response to a user triggering an operation on a search control on a video playback interface, obtaining a text search instruction from the user, and generating a text query request according to the text search instruction, wherein the text query request carries the selected text to be queried; Sending the text query request to a server; receiving information query results from the server; the information query results including a plurality of contents related to the text to be queried; The information query results are displayed in pages in the form of pop-up windows, so that the user can interact with the search engine in the pop-up window page to browse the information query results and obtain the next page of information query results.

2. The method according to claim 1, characterized in that The method further comprises: The information query result is processed according to the pop-up window operation instruction of the user, wherein the pop-up window operation instruction includes at least one of the following operations: a pop-up window closing instruction, a pop-up window page turning instruction, and a page sliding down instruction.

3. The method according to claim 1, characterized in that: The step of obtaining the target video picture includes: Pause the target video according to the user's video pause request; Get the target video screen corresponding to the target video displayed on the current playback interface.

4. The method according to claim 1, characterized in that: The step of obtaining the target video picture includes: According to the video screenshot instruction of the user, the target video screen is obtained, and the target video screen is the screen of the video currently being played by the terminal device.

5. A video-based information query method, characterized in that: include: A text query request is received from a terminal device, the text query request comprising: a text to be queried, the text query request carrying the text to be queried; the text query request is sent by the terminal device in response to a user triggering an operation of a search control on a video playback interface, the text to be queried is obtained by determining a starting position and an ending position of the text to be queried from the text information based on a user moving a cursor displayed on the terminal after a user long-presses the text information; the text information is text obtained by performing text recognition on a target video screen; the text information is superimposed on a corresponding position of the target video screen; Obtaining information query results corresponding to the text to be queried; the information query results include multiple contents related to the text to be queried; Sending the information query result to the terminal device; displaying the information query result in pages on the terminal interface in the form of a pop-up window, so that the user can interact with the search engine in the pop-up window page to browse the information query result and obtain the next page of information query results; Before receiving the text query request from the terminal device, the method further includes: Receiving a text recognition request from a terminal device, wherein the text recognition request is sent by the terminal device in response to a user triggering an operation on a recognition control on a video playback interface, and the text recognition request carries the target video picture; Performing text recognition on the target video screen to obtain a text recognition result, wherein the text recognition result includes: text information in the target video screen and position information of the text information; The text recognition result is sent to the terminal device.

6. A video-based information query device, characterized in that: include: Acquisition module, processing module and display module; The acquisition module is used to acquire the target video picture; The processing module is used to send a text recognition request to the server in response to a user triggering operation on a recognition control on a video playback interface, wherein the text recognition request carries the target video picture; So that the server can recognize the text in the target video screen and obtain the text recognition result of the target video screen; receiving a text recognition result of the target video screen sent by the server, the text recognition result including: text information in the target video screen and position information of the text information; and superimposing the text information on a corresponding position of the target video screen according to the position information of the text information to obtain a processed target video screen; The display module is used to present the processed target video picture on the display interface of the terminal device; The device also includes: a sending module and a receiving module; The acquisition module is further used to respond to a user's long press operation on the text information; The processing module is further used to: in response to a user's long press operation on the text information, display a cursor for selecting text; based on the user's movement of the cursor, determine the starting position and the ending position of the text to be queried in the text information to obtain the text to be queried; in response to the user's triggering operation on the search control on the video playback interface, obtain the user's text search instruction, and generate a text query request according to the text search instruction, wherein the text query request carries the selected text to be queried; send the text query request to the server through the sending module, and receive the information query result corresponding to the text to be queried from the server through the receiving module; the information query result includes multiple contents related to the text to be queried; The display module is also used to display the information query results in pages in the form of pop-up windows, so that the user can interact with the search engine in the pop-up window page to browse the information query results and obtain the next page of information query results.

7. The device according to claim 6, characterized in that The processing module is further used to process the information query result according to the user's pop-up window operation instruction, and the pop-up window operation instruction includes at least one of the following operations: a pop-up window closing instruction, a pop-up window page turning instruction, and a page sliding down instruction.

8. The device according to claim 6, characterized in that The acquisition module is specifically used to pause the playing of the target video according to the video pause request of the user, and obtain the target video screen corresponding to the target video displayed on the current playback interface.

9. The device according to claim 6, characterized in that The acquisition module is specifically used to acquire the target video screen according to the video screenshot instruction of the user, and the target video screen is the screen of the video currently being played by the terminal device.

10. A video-based information query device, characterized in that: include: Receiving module, processing module and sending module; The receiving module is used to receive a text query request from a terminal device, the text query request includes: a text to be queried, the text query request carries the text to be queried; the text query request is sent by the terminal device in response to a user triggering an operation of a search control on a video playback interface, the text to be queried is obtained by determining the starting position and the ending position of the text to be queried from the text information based on the user's movement of a cursor displayed on the terminal after a long press operation on the text information; the text information is text obtained by performing text recognition on a target video screen; the text information is superimposed on a corresponding position of the target video screen; The processing module is used to obtain the information query result corresponding to the text to be queried; the information query result includes a plurality of contents related to the text to be queried; The sending module is used to send the information query result to the terminal device; the information query result is displayed in pages on the terminal interface in the form of a pop-up window, so that the user can interact with the search engine in the pop-up window page to browse the information query result and obtain the next page of information query results; The receiving module is further configured to receive a text recognition request from the terminal device before receiving the text query request from the terminal device, wherein the text recognition request is sent by the terminal device in response to a user triggering an identification control on the video playback interface, and the text recognition request carries the target video screen; The processing module is further used to perform text recognition on the target video screen to obtain a text recognition result, wherein the text recognition result includes: text information in the target video screen and position information of the text information; The sending module is further used to send the text recognition result to the terminal device.

11. A terminal device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 4.

12. A server, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of claim 5 .

13. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 4.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method of claim 5.

15. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 4.

Images