Focus recognition method and device, computer equipment and computer readable storage medium

Abstract

The invention discloses a focus recognition method and device, computer equipment and a computer readable storage medium. The method comprises the steps that to-be-processed data information is acquired; when the to-be-processed data information contains view tree data, analyzing the view tree data to obtain text analysis result information; when the text analysis result information is first result information, determining target focus information based on the page image information and a target processing model; the first result information represents that the view tree data does not contain the target text information. By adopting the method provided by the invention, character-free focus recognition and focus recognition of dynamic content change can be realized.

Description

Technical Field

[0001] This application relates to the field of display technology, specifically to a focus recognition method, apparatus, computer device, and computer-readable storage medium. Background Technology

[0002] Focus location recognition is fundamental to automated operations such as automated testing and automated path traversal. Existing focus recognition methods typically identify the focus region, perform OCR on the identified focus region to obtain the text within the focus region, and then use the identified text as the focus recognition result. However, existing focus recognition methods cannot recognize focus points without text. Summary of the Invention

[0003] This application provides a focus recognition method, apparatus, computer device, and computer-readable storage medium, which can realize focus recognition without text and focus recognition with dynamically changing content.

[0004] The technical solution adopted by this invention to solve the problem is as follows:

[0005] Firstly, this application provides a focus identification method, including:

[0006] Obtain the data to be processed;

[0007] When the data to be processed contains view tree data, the view tree data is parsed to obtain the text parsing result information;

[0008] When the text parsing result is the first result information, the target focus information is determined based on the page image information and the target processing model; the first result information indicates that the view tree data does not contain target text information.

[0009] In some embodiments of this application, after parsing the view tree data to obtain text parsing results, the process includes:

[0010] When the text parsing result is the second result information, the target text information and the candidate focus information are matched; the second result information indicates that the target text information is contained in the view tree data;

[0011] When the candidate focus information contains focus information that matches the target text information, the target focus information is determined based on a first quantity; the first quantity is the number of focus information in the candidate focus information that matches the target text information.

[0012] When the candidate focus information does not contain focus information that matches the target text information, the target text information, page image information, and first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

[0013] In some embodiments of this application, target focus information is determined based on a first quantity, including:

[0014] When the first quantity is less than or equal to the first threshold, the focus information that matches the target text information in the candidate focus information is determined as the target focus information;

[0015] When the first quantity exceeds the first threshold, the target text information, page image information, and first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

[0016] In some embodiments of this application, the target processing model includes a second processing model and a third processing model, which determine target focus information based on page image information and the target processing model, including:

[0017] The page image information and the second prompt information are input into the second processing model, and the first focus information is output through the second processing model.

[0018] The page image information and the third prompt information are input into the third processing model, and the second focus information is output through the third processing model.

[0019] Based on the first focus information and the second focus information, the target focus information is determined.

[0020] In some embodiments of this application, the target processing model further includes a fourth processing model, wherein the number of model parameters of the fourth processing model is greater than the number of model parameters of the second processing model and the number of model parameters of the fourth processing model is greater than the number of model parameters of the third processing model. Based on the first focus information and the second focus information, target focus information is determined, including:

[0021] When the first focus information and the second focus information match, the first focus information or the second focus information is determined as the target focus information;

[0022] When the first focus information and the second focus information do not match, the page image information and the fourth prompt information are input into the fourth processing model, and the target focus information is output through the fourth processing model.

[0023] In some embodiments of this application, after obtaining the data to be processed, the process includes:

[0024] When the data to be processed does not contain view tree data, the page image information and the fifth prompt information are input into the fifth processing model, and the third focus information is output through the fifth processing model.

[0025] Input the page image information and the sixth prompt information into the sixth processing model, and output the fourth focus information through the sixth processing model;

[0026] Based on the third and fourth focus information, the target focus information is determined.

[0027] In some embodiments of this application, the target focus information is determined based on the third focus information and the fourth focus information, including:

[0028] When the third and fourth focus information match, the third or fourth focus information is determined as the target focus information;

[0029] When the third and fourth focus information do not match, the page image information and the seventh prompt information are input into the seventh processing model, and the target focus information is output through the seventh processing model; wherein, the number of model parameters of the seventh processing model is greater than the number of model parameters of the fifth processing model and the number of model parameters of the seventh processing model is greater than the number of model parameters of the sixth processing model.

[0030] Secondly, this application provides a focus recognition device, comprising:

[0031] The data acquisition module is used to acquire data information to be processed.

[0032] The data parsing module is used to parse the view tree data to obtain text parsing results when the data to be processed contains view tree data.

[0033] The focus recognition module is used to determine the target focus information based on the page image information and the target processing model when the text parsing result information is the first result information; the first result information indicates that the view tree data does not contain target text information.

[0034] Thirdly, this application also provides a computer device, which includes:

[0035] One or more processors;

[0036] Memory; and

[0037] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor to implement the focus recognition method of any of the first aspects.

[0038] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, the computer program being loaded by a processor to perform the steps in the focus identification method of any one of the first aspects.

[0039] The beneficial effects of this invention are as follows: By acquiring data information to be processed, and when the data information to be processed contains view tree data, the view tree data is parsed to obtain text parsing result information. When the text parsing result information is the first result information, the target focus information is determined based on the page image information and the target processing model, which can realize focus recognition without text. In addition, by determining the target focus information through the target processing model, the obtained focus recognition result can be matched with the user-defined focus information and the focus of dynamically changing content can be identified. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a scene diagram of the focus recognition system provided in an embodiment of the present invention;

[0042] Figure 2 This is a schematic flowchart of an embodiment of the focus recognition method provided by the present invention;

[0043] Figure 3 This is a schematic diagram of an embodiment of the display page provided in this invention;

[0044] Figure 4 This is a schematic flowchart of another embodiment of the focus recognition method provided in this invention;

[0045] Figure 5 This is a schematic diagram of another embodiment of the display page provided in this invention;

[0046] Figure 6 A schematic flowchart of another embodiment of the focus recognition method provided in this invention;

[0047] Figure 7 This is a schematic diagram of a specific embodiment of the focus recognition device provided in this invention.

[0048] Figure 8 This is a schematic diagram of an embodiment of the computer device provided in this invention. Detailed Implementation

[0049] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0050] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more features.

[0051] In this application, the term "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes are not described in detail to avoid obscuring the description of this application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0052] It should be noted that since the method in this application embodiment is executed in a computer device, the processing objects of each computer device exist in the form of data or information, such as time, which is essentially time information. It is understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data that exist so that the computer device can process them. Specific details will not be elaborated here.

[0053] This application provides a focus identification method, apparatus, computer device, and computer-readable storage medium, which will be described in detail below.

[0054] The focus recognition method provided in this application is applied to a focus recognition system. Please refer to [link / reference]. Figure 1 , Figure 1 This is a schematic diagram of a focus recognition system provided in an embodiment of this application. The focus recognition system may include a computer device 100, such as... Figure 1 Computer equipment in the country.

[0055] In some embodiments, the computer device 100 is mainly used to acquire data information to be processed; when the data information to be processed contains view tree data, the view tree data is parsed to obtain text parsing result information; when the text parsing result information is first result information, the target focus information is determined based on the page image information and the target processing model; the first result information indicates that the view tree data does not contain target text information, and focus recognition without text can be achieved; in addition, by determining the target focus information through the target processing model, the obtained focus recognition result can be matched with the user-defined focus information and the focus of dynamically changing content can be identified.

[0056] In this embodiment, the computer device 100 can be a standalone server, a server network, or a server cluster. For example, the computer device 100 described in this embodiment includes, but is not limited to, a computer, a network host, a single network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing.

[0057] It is understood that the computer device 100 used in the embodiments of this application can be a device that includes both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such a device may include: cellular or other communication devices having a single-line display, a multi-line display, or a cellular or other communication device without a multi-line display. Specifically, the computer device 100 may be a desktop terminal or a mobile terminal, and may also be one of the following: an augmented reality (AR) device, a mobile phone, a tablet computer, a laptop computer, etc.

[0058] Those skilled in the art will understand that Figure 1 The application environment shown is merely one application scenario of the solution in this application and does not constitute a limitation on the application scenario of the solution in this application. Other application environments may include those that are more specific to this application. Figure 1 The number of computer devices shown is more or less, for example Figure 1 Only one computer device is shown in the image. It is understood that the focus recognition system may also include one or more other services, which are not limited here.

[0059] In addition, the focus recognition system may also include a memory for storing data, such as focus information, such as target focus information, candidate focus information, etc., and prompt information, such as first prompt information, second prompt information, etc.

[0060] It should be noted that, Figure 1The schematic diagram of the focus recognition system shown is merely an example. The focus recognition system and scenario described in this application are for the purpose of more clearly illustrating the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of focus recognition systems and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0061] like Figure 2 The diagram shown is a flowchart of an embodiment of the focus recognition method in this application. The focus recognition method may include the following steps S201 to S203, as detailed below:

[0062] Step S201: Obtain the data information to be processed.

[0063] In this embodiment, the data to be processed is a set of parameters for focus identification. The data to be processed includes one or more of the following: the screenshot URL of the current page, candidate focus information, view tree data, and the desired focus result. The screenshot URL of the current page refers to the network address or local path that can locate the screenshot file of the current page; the page image information can be obtained based on the screenshot URL of the previous page. The candidate focus information is a set of focus information predefined by the user, including multiple focus information items stored in an array. For example, candidate focus information can be represented as: ["User{icon}","Set up Google TV"]. The view tree data refers to the hierarchical structure data of all visible (or accessible) UI controls on the current screen, captured by the uiautomator tool. Essentially, it is a tree-like data structure of buttons, text boxes, images, and other controls on the interface, organized according to their nesting relationships. The desired focus result refers to the focus result expected by the user, which is one of the focus information items among the candidate focus information. For example, referring to Table 1, an example of the data information to be processed provided in this embodiment is shown. In this example of data information to be processed, the data information to be processed includes options and screenshot. The data information to be processed may include window_dump and expect, or it may not include window_dump and expect. Here, options represents candidate focus information, screenshot represents the screenshot URL of the current page, window_dump represents view tree data, and expect represents the expected focus result.

[0064] Table 1 Examples of data information to be processed

[0065]

[0066] Optionally, the data to be processed can be obtained through modules configured within the computer device itself, or it can be obtained from other computer devices via networks, Bluetooth, infrared, etc. For example, when the focus recognition method is applied to a smart TV, the smart TV can obtain candidate focus information input by the user through input devices such as a mouse and keyboard, and obtain the screenshot URL of the current page by taking a screenshot of the page it displays; when the focus recognition method is applied to a server, the server can obtain the data to be processed from the smart TV via networks, Bluetooth, infrared, etc.

[0067] S202. When the data to be processed contains view tree data, the view tree data is parsed to obtain text parsing result information.

[0068] In this embodiment, the data to be processed may or may not contain view tree data. After obtaining the data to be processed, it can be parsed to obtain the first keyword information corresponding to each parameter in the data to be processed. Based on the first keyword information, it is determined whether the data to be processed contains view tree data. For example, taking the data to be processed in Table 1 as an example, if the data to be processed contains the keyword information "window_dump", then it is determined that the data to be processed contains view tree data.

[0069] Furthermore, the text parsing result information includes first result information and second result information. The first result information indicates that the view tree data does not contain the target text information, and the second result information indicates that the view tree data contains the target text information. The target text information is the text content corresponding to the UI control that is in the system focus state. For example, the target text information is the content of the text property corresponding to the control with the attribute "focused=true" in the view tree data.

[0070] S203. When the text parsing result information is the first result information, the target focus information is determined based on the page image information and the target processing model; the first result information indicates that the view tree data does not contain target text information.

[0071] In this embodiment, when the text parsing result is the first result, it indicates that the view tree data does not contain target text information, meaning the current focus may be an icon or image and there is no text prompt after focusing. Therefore, the target focus information is determined based on the page image information and the target processing model. In this embodiment, when the data to be processed includes view tree data, the view tree data is parsed to obtain text parsing result information. When the text parsing result is the first result, the target focus information is determined based on the page image information and the target processing model, enabling focus recognition without text. For example, refer to... Figure 3As shown, the focus recognition method provided in this application embodiment can identify... Figure 3 The rightmost focus is "+".

[0072] Furthermore, by determining the target focus information through the target processing model, the obtained focus recognition results can be matched with user-defined focus information, and the focus can be identified even when the content is dynamically changing. For example, in a scenario where the Wi-Fi connection is selected, the text content identified by the focus will differ depending on the Wi-Fi network connected. By using the user-defined focus information "connected Wi-Fi", the focus control connected to the Wi-Fi network can be automatically identified.

[0073] Furthermore, the page image information is the page image data currently displayed on the computer device. The page image information can be obtained by obtaining the screenshot URL of the current page from the data information to be processed, and based on the screenshot URL of the current page. The page image information can also be obtained by taking a screenshot of the currently displayed page by the computer device, or by acquiring image data of the currently displayed page by the imaging module.

[0074] Optionally, the target processing model includes any one or more of the second, third, and fourth processing models. The second, third, and fourth processing models are pre-trained artificial intelligence models, which can be based on large language models (LLMs). Large language models refer to artificial neural network models with a very large number of parameters. In the field of artificial intelligence, large models typically refer to models with hundreds of millions to trillions of parameters. These models usually need to be trained on large-scale datasets and require significant computational resources for optimization and tuning. Large models are commonly used to solve complex tasks such as natural language processing, computer vision, and speech recognition.

[0075] In this embodiment of the application, the large language model can be a large-scale pre-trained model such as ChatGPT series, BERT, XLNet, Zhipu model, Claude, Moonshot AI model, ChatGLM model, Tongwen Qianyi model, MiniMax model, Xinghuo model, Llama model, 360GPT model, Qwen model, Baichuan model, Yunque model, vivoLM model, deepseek, Tencent Yuanbao and Wenxin Yiyan, etc. This embodiment of the application does not limit the model.

[0076] Furthermore, in step S203, when determining the target focus information based on the page image information and the target processing model, the page image information can be input into any one of the second, third, and fourth processing models, and the target focus information can be output through any one of the second, third, and fourth processing models. Alternatively, the page image information can be input into the second, third, and fourth processing models, and the target focus information can be determined based on the output results of multiple models.

[0077] In some embodiments, the target processing model includes a second processing model and a third processing model. The second and third processing models can be constructed based on the same large language model, or they can be constructed based on different large language models. The steps for determining target focus information based on page image information and the target processing model specifically include: inputting page image information and second prompt information into the second processing model, and outputting first focus information through the second processing model; inputting page image information and third prompt information into the third processing model, and outputting second focus information through the third processing model; and determining the target focus information based on the first and second focus information. This embodiment comprehensively determines the target focus information based on the output results of the second and third processing models, which can improve the accuracy of focus recognition.

[0078] In this embodiment, the second processing model is a pre-trained artificial intelligence model for outputting first focus information based on page image information and second prompt information. The second prompt information is prompt data used to guide the second processing model to output the first focus information based on page image information. The second prompt information can indicate to the second processing model that the focus area is an icon or image, and the features of the icon or image being focused, thereby improving the accuracy of the determined first focus information. The second processing model is obtained by training a first network model based on first training data. The first training data includes first sample image information and first label focus information corresponding to the first sample image information. The training process of the second processing model is as follows: inputting the first sample image information and the second prompt information into the first network model, outputting first predicted focus information through the first network model; training the first network model based on the first label focus information and the first predicted focus information to obtain the second processing model.

[0079] Furthermore, the third processing model is a pre-trained artificial intelligence model used to output second focus information based on page image information and third prompt information. The third prompt information is prompt data used to guide the third processing model to output second focus information based on page image information. Through the third prompt information, the third processing model can be prompted that the focus area is an icon or image and the focus feature of the icon or image, thereby improving the accuracy of the determined second focus information. The third processing model is obtained by training the second network model based on the second training data. The second training data includes the second sample image information and the second label focus information corresponding to the second sample image information. The training process of the third processing model is the same as that of the second processing model. For details, please refer to the training process of the second processing model described above. To avoid repetition, this embodiment will not repeat it here.

[0080] In some embodiments, the target processing model further includes a fourth processing model, wherein the number of model parameters of the fourth processing model is greater than the number of model parameters of the second processing model and the number of model parameters of the third processing model. Optionally, the step of determining the target focus information based on the first focus information and the second focus information specifically includes: when the first focus information and the second focus information match, determining the first focus information or the second focus information as the target focus information; when the first focus information and the second focus information do not match, inputting the page image information and the fourth prompt information into the fourth processing model, and outputting the target focus information through the fourth processing model. In this embodiment, the first focus information and the second focus information are first determined by the second processing model and the third processing model with smaller model parameters, respectively. When the first focus information and the second focus information match, the first focus information or the second focus information is directly determined as the target focus information, which can improve the efficiency of focus recognition; when the first focus information and the second focus information do not match, the fourth processing model with larger model parameters is used for focus recognition, which can ensure the accuracy of focus recognition.

[0081] Optionally, matching the first focus information and the second focus information can mean that the first focus information and the second focus information are completely identical. Matching the first focus information and the second focus information can also mean that the similarity between the first focus information and the second focus information is greater than a first similarity threshold. The fourth processing model is a pre-trained artificial intelligence model used to output target focus information based on page image information and fourth prompt information. The fourth prompt information is prompt data used to guide the fourth processing model to output target focus information based on page image information. Through the fourth prompt information, the fourth processing model can be prompted that the focus area is an icon or image and the focus feature of the icon or image, thereby improving the accuracy of the determined target focus information. The fourth processing model is obtained by training the third network model based on the third training data. The third training data includes the third sample image information and the third label focus information corresponding to the third sample image information. The training process of the fourth processing model is the same as the training process of the second processing model. To avoid repetition, it will not be described again in this embodiment.

[0082] In some embodiments, refer to Figure 4 As shown, after parsing the view tree data to obtain the text parsing result information in step S202 above, steps S301 to S303 can be included, as detailed below:

[0083] S301. When the text parsing result information is the second result information, match the target text information and the candidate focus information; the second result information indicates that the view tree data contains the target text information.

[0084] In this embodiment, the candidate focus information is a set of user-defined focus information. The candidate focus information can be obtained from the data to be processed, or it can be obtained from other computer devices via network, Bluetooth, infrared, etc. Optionally, when the text parsing result is the second result information, indicating that the view tree data contains target text information, the target text information and the candidate focus information are matched. This embodiment determines the target focus information based on the candidate focus information, which can match the obtained target focus information with user-defined focus information and identify focus areas where content changes.

[0085] S302. When the candidate focus information contains focus information that matches the target text information, the target focus information is determined based on a first quantity; the first quantity is the number of focus information in the candidate focus information that matches the target text information.

[0086] In this embodiment, the inclusion of focus information matching the target text information in the candidate focus information can mean that the candidate focus information contains focus information with a similarity greater than a second similarity threshold to the target text information, or it can mean that the candidate focus information contains focus information identical to the target text information. When the candidate focus information contains focus information matching the target text information, obtaining the number of focus information matching the target text information in the candidate focus information and determining the target focus information based on the number of focus information matching the target text information in the candidate focus information can improve the efficiency and accuracy of focus recognition.

[0087] In some embodiments, the step of determining the target focus information based on the first quantity specifically includes: when the first quantity is less than or equal to a first threshold, determining the focus information that matches the target text information among the candidate focus information as the target focus information; when the first quantity is greater than the first threshold, inputting the target text information, page image information, and first prompt information into a first processing model, and outputting the target focus information through the first processing model. This embodiment directly determines the focus information that matches the target text information among the candidate focus information as the target focus information when the first quantity is less than or equal to the first threshold, and then determines the target focus information through the first processing model when the first quantity is greater than the first threshold, which can improve the efficiency of focus recognition.

[0088] In this embodiment, the first threshold can be set according to actual needs. For example, the first threshold can be set to 1. When the candidate focus information contains focus information that matches the target text information and the focus information that matches the target text information is unique, the unique focus information is determined as the target focus information. When the candidate focus information contains focus information that matches the target text information but the focus information that matches the target text information is not unique, the target focus information is determined based on the first processing model. This can solve the problems of redundant focus text content and the inability to accurately identify the main focus when multiple pages exist simultaneously and each page has its own focus. For example, refer to... Figure 5 As shown, the background page has the focus "TV", and the front page has the focus "All settings". The focus recognition method provided in this embodiment can accurately identify the main focus of the interaction, "All settings".

[0089] Furthermore, the first processing model is a pre-trained artificial intelligence model used to output target focus information based on target text information, page image information, and first prompt information. The first processing model can be constructed based on a Large Language Model (LLM). The first prompt information is prompt data used to guide the first processing model to output target focus information based on target text information and page image information. The first prompt information can indicate to the first processing model that target text information exists in the focus area, thereby improving the accuracy of the determined target focus information. The first processing model is obtained by training a fourth network model based on fourth training data. The fourth training data includes fourth sample image information, the first text information corresponding to the fourth sample image information, and the fourth label focus information corresponding to the fourth sample image information. The training process of the first processing model is as follows: input the fourth sample image information, the first text information corresponding to the fourth sample image information, and the first prompt information into the fourth network model, and output the second predicted focus information through the fourth network model; train the fourth network model based on the fourth label focus information and the second predicted focus information to obtain the first processing model.

[0090] S303. When the candidate focus information does not contain focus information that matches the target text information, the target text information, the page image information, and the first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

[0091] In this embodiment, when the candidate focus information does not contain focus information that matches the target text information, the target text information, page image information, and first prompt information are input into the first processing model, and the target focus information is output through the first processing model. This can solve the problems of redundant focus text content and the inability to accurately identify the main focus when multiple pages exist simultaneously and each page has its own focus.

[0092] In some embodiments, refer to Figure 6 As shown, after obtaining the data to be processed in step S201, steps S401 to S403 may be included, as detailed below:

[0093] S401. When the data to be processed does not contain view tree data, input the page image information and the fifth prompt information into the fifth processing model, and output the third focus information through the fifth processing model.

[0094] In this embodiment, the fifth processing model is a pre-trained artificial intelligence model for outputting third focus information based on page image information and fifth prompt information. The fifth processing model can be constructed based on a Large Language Model (LLM). The fifth prompt information is prompt data used to guide the fifth processing model to output third focus information based on page image information. Through the fifth prompt information, the fifth processing model can be prompted to identify the focus area and match it with the focus information in the candidate focus information, thereby improving the accuracy of the determined third focus information.

[0095] Furthermore, the fifth processing model is obtained by training the fifth network model based on the fifth training data. The fifth training data includes the fifth sample image information and the fifth label focus information corresponding to the fifth sample image information. The training process of the fifth processing model is the same as that of the second processing model. For details, please refer to the training process of the aforementioned second processing model. To avoid repetition, this embodiment will not repeat it here.

[0096] S402. Input the page image information and the sixth prompt information into the sixth processing model, and output the fourth focus information through the sixth processing model.

[0097] In this embodiment, the sixth processing model is a pre-trained artificial intelligence model for outputting fourth focus information based on page image information and sixth prompt information. The sixth processing model can be constructed based on a Large Language Model (LLM). The sixth prompt information is prompt data used to guide the sixth processing model to output fourth focus information based on page image information. The sixth prompt information can prompt the sixth processing model to identify the focus area and match it with the focus information in the candidate focus information, thereby improving the accuracy of the determined fourth focus information.

[0098] Furthermore, the sixth processing model is obtained by training the sixth network model based on the sixth training data. The sixth training data includes the sixth sample image information and the sixth label focus information corresponding to the sixth sample image information. The training process of the sixth processing model is the same as that of the second processing model. For details, please refer to the training process of the aforementioned second processing model. To avoid repetition, this embodiment will not repeat it here.

[0099] S403. Based on the third and fourth focus information, determine the target focus information.

[0100] In this embodiment of the application, when the data to be processed does not contain view tree data, the third focus information and the fourth focus information are output based on the fifth processing model and the sixth processing model, respectively. The target focus information is determined based on the third focus information and the fourth focus information, which can improve the accuracy and efficiency of focus recognition.

[0101] In some embodiments, the step of determining the target focus information based on the third focus information and the fourth focus information specifically includes: when the third focus information and the fourth focus information match, determining the third focus information or the fourth focus information as the target focus information; when the third focus information and the fourth focus information do not match, inputting the page image information and the seventh prompt information into the seventh processing model, and outputting the target focus information through the seventh processing model; wherein, the number of model parameters of the seventh processing model is greater than the number of model parameters of the fifth processing model and the number of model parameters of the seventh processing model is greater than the number of model parameters of the sixth processing model.

[0102] Optionally, matching the third and fourth focus information can mean that the third and fourth focus information are identical, or that the similarity between the third and fourth focus information is greater than a third similarity threshold. In this embodiment, the fifth and sixth processing models with fewer model parameters are used to determine the third and fourth focus information, respectively. When the third and fourth focus information match, either the third or fourth focus information is directly identified as the target focus information, which can improve the efficiency of focus recognition. When the third and fourth focus information do not match, the seventh processing model with more model parameters is used for focus recognition, which can ensure the accuracy of focus recognition.

[0103] To better implement the focus recognition method in the embodiments of this application, a focus recognition device is also provided in the embodiments of this application, such as... Figure 7 As shown, the focus recognition device 600 includes:

[0104] Data acquisition module 610 is used to acquire data information to be processed;

[0105] The data parsing module 620 is used to parse the view tree data to obtain text parsing results when the data to be processed contains view tree data.

[0106] The focus recognition module 630 is used to determine the target focus information based on the page image information and the target processing model when the text parsing result information is the first result information; the first result information indicates that the view tree data does not contain the target text information.

[0107] In this embodiment, by acquiring data information to be processed, and when the data information to be processed contains view tree data, the view tree data is parsed to obtain text parsing result information. When the text parsing result information is the first result information, the target focus information is determined based on the page image information and the target processing model, which can realize focus recognition without text. In addition, by determining the target focus information through the target processing model, the obtained focus recognition result can be matched with the user-defined focus information and the focus of dynamically changing content can be identified.

[0108] In some embodiments of this application, the focus recognition module 630 is further configured to:

[0109] When the text parsing result is the second result information, the target text information and the candidate focus information are matched; the second result information indicates that the target text information is contained in the view tree data;

[0110] When the candidate focus information contains focus information that matches the target text information, the target focus information is determined based on a first quantity; the first quantity is the number of focus information in the candidate focus information that matches the target text information.

[0111] When the candidate focus information does not contain focus information that matches the target text information, the target text information, page image information, and first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

[0112] In some embodiments of this application, the focus recognition module 630 is specifically used for:

[0113] When the first quantity is less than or equal to the first threshold, the focus information that matches the target text information in the candidate focus information is determined as the target focus information;

[0114] When the first quantity exceeds the first threshold, the target text information, page image information, and first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

[0115] In some embodiments, the target processing model includes a second processing model and a third processing model, and the focus recognition module 630 is further configured to:

[0116] The page image information and the second prompt information are input into the second processing model, and the first focus information is output through the second processing model.

[0117] The page image information and the third prompt information are input into the third processing model, and the second focus information is output through the third processing model.

[0118] Based on the first focus information and the second focus information, the target focus information is determined.

[0119] In some embodiments, the target processing model further includes a fourth processing model, wherein the number of model parameters of the fourth processing model is greater than the number of model parameters of the second processing model and the number of model parameters of the fourth processing model is greater than the number of model parameters of the third processing model, and the focus recognition module 630 is further used for:

[0120] When the first focus information and the second focus information match, the first focus information or the second focus information is determined as the target focus information;

[0121] When the first focus information and the second focus information do not match, the page image information and the fourth prompt information are input into the fourth processing model, and the target focus information is output through the fourth processing model.

[0122] In some embodiments, the focus recognition module 630 is further configured to:

[0123] When the data to be processed does not contain view tree data, the page image information and the fifth prompt information are input into the fifth processing model, and the third focus information is output through the fifth processing model.

[0124] Input the page image information and the sixth prompt information into the sixth processing model, and output the fourth focus information through the sixth processing model;

[0125] Based on the third and fourth focus information, the target focus information is determined.

[0126] In some embodiments, the focus recognition module 630 is further configured to:

[0127] When the third and fourth focus information match, the third or fourth focus information is determined as the target focus information;

[0128] When the third and fourth focus information do not match, the page image information and the seventh prompt information are input into the seventh processing model, and the target focus information is output through the seventh processing model; wherein, the number of model parameters of the seventh processing model is greater than the number of model parameters of the fifth processing model and the number of model parameters of the seventh processing model is greater than the number of model parameters of the sixth processing model.

[0129] This application also provides a computer device that integrates any of the focus recognition devices provided in this application, the computer device comprising:

[0130] One or more processors;

[0131] Memory; and

[0132] One or more applications, wherein the applications are stored in memory and configured to be executed by a processor from the steps of the focus recognition method in any of the above-described focus recognition method embodiments.

[0133] This application also provides a computer device that integrates any of the focus recognition mechanisms provided in this application. For example... Figure 8 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0134] The computer device may include components such as a processor 801 with one or more processing cores, a memory 802 with one or more computer-readable storage media, a power supply 803, and an input unit 804. Those skilled in the art will understand that... Figure 8 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0135] The processor 801 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 802, and by calling data stored in the memory 802, it performs various functions of the computer device and processes data, thereby providing overall monitoring of the computer device. Optionally, the processor 801 may include one or more processing cores; preferably, the processor 801 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 801.

[0136] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 802 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 802 may also include a memory controller to provide the processor 801 with access to the memory 802.

[0137] The computer device also includes a power supply 803 that supplies power to the various components. Preferably, the power supply 803 can be logically connected to the processor 801 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 803 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0138] The computer device may also include an input unit 804, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0139] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 801 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 802 according to the following instructions, and the processor 801 runs the application programs stored in the memory 802 to realize various functions, as follows:

[0140] Obtain the data to be processed;

[0141] When the data to be processed contains view tree data, the view tree data is parsed to obtain the text parsing result information;

[0142] When the text parsing result is the first result information, the target focus information is determined based on the page image information and the target processing model; the first result information indicates that the view tree data does not contain target text information.

[0143] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0144] Therefore, embodiments of this application provide a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any of the focus recognition methods provided in embodiments of this application. For example, the computer program loaded by the processor can execute the following steps:

[0145] Obtain the data to be processed;

[0146] When the data to be processed contains view tree data, the view tree data is parsed to obtain the text parsing result information;

[0147] When the text parsing result is the first result information, the target focus information is determined based on the page image information and the target processing model; the first result information indicates that the view tree data does not contain target text information.

[0148] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed descriptions of other embodiments above, which will not be repeated here.

[0149] In practice, each of the above units or structures can be implemented as an independent entity or can be arbitrarily combined to be implemented as the same or several entities. For the specific implementation of each of the above units or structures, please refer to the previous method embodiments, which will not be repeated here.

[0150] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0151] The above provides a detailed description of a focus identification method, apparatus, computer device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A focus recognition method, characterized in that, include: Obtain the data to be processed; When the data to be processed contains view tree data, the view tree data is parsed to obtain text parsing result information; When the text parsing result is the first result, the target focus information is determined based on the page image information and the target processing model; The first result information indicates that the view tree data does not contain target text information.

2. The focus recognition method according to claim 1, characterized in that, After parsing the view tree data to obtain the text parsing result information, the process includes: When the text parsing result is the second result, the target text information and the candidate focus information are matched; the second result indicates that the view tree data contains the target text information. When the candidate focus information includes focus information that matches the target text information, the target focus information is determined based on a first quantity; the first quantity is the number of focus information in the candidate focus information that matches the target text information. When the candidate focus information does not contain focus information that matches the target text information, the target text information, the page image information, and the first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

3. The focus recognition method according to claim 2, characterized in that, The determination of target focus information based on the first quantity includes: When the first quantity is less than or equal to the first threshold, the focus information that matches the target text information in the candidate focus information is determined as the target focus information; When the first quantity is greater than the first threshold, the target text information, the page image information, and the first prompt information are input into the first processing model, and the target focus information is output through the first processing model.

4. The focus recognition method according to claim 1, characterized in that, The target processing model includes a second processing model and a third processing model; The determination of target focus information based on page image information and target processing model includes: The page image information and the second prompt information are input into the second processing model, and the first focus information is output through the second processing model. The page image information and the third prompt information are input into the third processing model, and the second focus information is output through the third processing model. Based on the first focus information and the second focus information, the target focus information is determined.

5. The focus recognition method according to claim 4, characterized in that, The target processing model further includes a fourth processing model, wherein the number of model parameters of the fourth processing model is greater than the number of model parameters of the second processing model and the number of model parameters of the fourth processing model is greater than the number of model parameters of the third processing model. Determining the target focus information based on the first focus information and the second focus information includes: When the first focus information and the second focus information match, the first focus information or the second focus information is determined as the target focus information; When the first focus information and the second focus information do not match, the page image information and the fourth prompt information are input into the fourth processing model, and the target focus information is output through the fourth processing model.

6. The focus recognition method according to claim 1, characterized in that, After obtaining the data information to be processed, the following steps are included: When the data to be processed does not contain view tree data, the page image information and the fifth prompt information are input into the fifth processing model, and the third focus information is output through the fifth processing model. The page image information and the sixth prompt information are input into the sixth processing model, and the fourth focus information is output through the sixth processing model. Based on the third focus information and the fourth focus information, the target focus information is determined.

7. The focus recognition method according to claim 6, characterized in that, Determining the target focus information based on the third focus information and the fourth focus information includes: When the third focus information and the fourth focus information match, the third focus information or the fourth focus information is determined as the target focus information; When the third focus information and the fourth focus information do not match, the page image information and the seventh prompt information are input into the seventh processing model, and the target focus information is output through the seventh processing model; wherein, the number of model parameters of the seventh processing model is greater than the number of model parameters of the fifth processing model and the number of model parameters of the seventh processing model is greater than the number of model parameters of the sixth processing model.

8. A focus recognition device, characterized in that, include: The data acquisition module is used to acquire data information to be processed. The data parsing module is used to parse the view tree data to obtain text parsing result information when the data information to be processed contains view tree data; The focus recognition module is used to determine the target focus information based on the page image information and the target processing model when the text parsing result information is the first result information; The first result information indicates that the view tree data does not contain target text information.

9. A computer device, characterized in that, The computer device includes: One or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the focus recognition method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which is loaded by a processor to perform the steps of the focus recognition method according to any one of claims 1 to 7.

Images