Page reading method and device, terminal and storage medium

Abstract

The embodiment of the invention discloses a page reading method and device, a terminal and a storage medium, and relates to the field of artificial intelligence. Comprising the steps of obtaining a page script of a current page in response to a page reading operation for the current page; under the condition that the page script is obtained, extracting a reading text from the page script; under the condition that the page script is not obtained, obtaining a reading text in the current page through a barrier-free service; under the condition that the reading text cannot be obtained through the barrier-free service, performing optical character recognition on the current page to obtain the reading text in the current page; and playing the reading audio corresponding to the reading text. By adopting the method provided by the embodiment of the invention, the probability of obtaining the reading text corresponding to the current page is increased, and the application scene of the page reading function is expanded.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of artificial intelligence, and in particular to a page reading method, device, terminal and storage medium. Background Art

[0002] When using the terminal, users often have the need to listen to books. For example, when the user's visual fatigue occurs, the user needs to convert the text on the web page into audio for playback, or listen to the content of the public account while driving.

[0003] In related technologies, optical character recognition technology (OCR), page parsing technology, or accessibility services may be used to convert text on a page into audio and play it. Summary of the invention

[0004] The embodiment of the present application provides a method, device, terminal and storage medium for reading a page. The technical solution is as follows:

[0005] On the one hand, an embodiment of the present application provides a page reading method, the method comprising:

[0006] In response to a page reading operation on a current page, obtaining a page script of the current page;

[0007] When the page script is obtained, extracting the reading text from the page script;

[0008] If the page script is not obtained, obtaining the read text in the current page through an accessibility service;

[0009] In the case where the read-aloud text cannot be obtained through the barrier-free service, performing optical character recognition on the current page to obtain the read-aloud text in the current page;

[0010] Play the reading audio corresponding to the reading text.

[0011] On the other hand, an embodiment of the present application provides a page reading device, the device comprising:

[0012] A page script acquisition module, used for acquiring a page script of the current page in response to a page reading operation on the current page;

[0013] A reading text acquisition module, used to extract the reading text from the page script when the page script is acquired;

[0014] The read-aloud text acquisition module is further used to acquire the read-aloud text in the current page through an accessibility service when the page script is not acquired;

[0015] The read-aloud text acquisition module is further used to perform optical character recognition on the current page to obtain the read-aloud text in the current page when the read-aloud text cannot be obtained through the barrier-free service;

[0016] The playing module is used to play the reading audio corresponding to the reading text.

[0017] On the other hand, an embodiment of the present application provides a terminal, which includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the method described in the above aspect.

[0018] On the other hand, an embodiment of the present application provides a computer-readable storage medium, in which at least one instruction is stored, and the instruction is loaded and executed by a processor to implement the method described in the above aspect.

[0019] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the method provided in various optional implementations of the above aspects.

[0020] In an embodiment of the present application, by extracting the reading text from the page script when the page script is obtained; obtaining the reading text through the accessibility service when the page script is not obtained; and obtaining the reading text through optical character recognition when the reading text cannot be obtained through the accessibility service, the probability of obtaining the reading text corresponding to the current page can be increased, thereby expanding the applicable scenarios of the page reading function. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0023] Figure 2is a flow chart of a page reading method provided by an exemplary embodiment of the present application;

[0024] Figure 3 It is a schematic diagram of parsing a page script and extracting aloud text from a page script provided by an exemplary embodiment of the present application;

[0025] Figure 4 is a flowchart of obtaining a read-aloud text through an accessibility service provided by an exemplary embodiment of the present application;

[0026] Figure 5 is a flow chart of obtaining a read-aloud text through optical character recognition provided by an exemplary embodiment of the present application;

[0027] Figure 6 is a flow chart of a page reading method provided by another exemplary embodiment of the present application;

[0028] Figure 7 is a schematic diagram of a reading panel provided by an exemplary embodiment of the present application;

[0029] Figure 8 is a structural block diagram of a page reading device provided by an exemplary embodiment of the present application;

[0030] Fig. 9 It is a structural block diagram of a terminal provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0031] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0032] See also Figure 1 , Figure 1 It is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application.

[0033] The implementation environment includes a terminal 11 , a reading application 12 , and a text-to-speech server 13 .

[0034] The terminal 11 is an electronic device that can run the reading application 12. The electronic device can be a mobile terminal such as a smart phone, a tablet computer, a laptop computer, a vehicle-mounted terminal, or a terminal such as a desktop computer or a projection computer. This specification uses the example of the terminal 11 being a smart phone for schematic description, but does not constitute a limitation thereto.

[0035] The reading application 12 is an application for displaying the reading text corresponding to the current page and playing the reading audio corresponding to the current page. In some embodiments, the reading application 12 can display a reading panel, which includes a play control to play the reading audio corresponding to the current page; the reading panel can also include controls such as a play previous section control, a play next section control, a speed selection control, and a replay control. In some embodiments, the text segment in the reading text corresponding to the reading audio being played can be displayed in the reading panel by highlighting or pinning it.

[0036] The text-to-speech server 13 is used to receive the reading text corresponding to the current page transmitted by the reading application 12, and convert the reading text into reading audio to return to the reading application 12. Optionally, the text-to-speech server 13 is a background server of the reading application 12, or a third-party server. The text-to-speech server 13 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0037] In some embodiments, the terminal 11 may run a reading application 12 in the background. When the reading application 12 receives a page reading operation from the user for the current page, the reading application 12 obtains the page script of the current page, and extracts the reading text from the page script to send the reading text to the text-to-speech server 13. The current page may be a page corresponding to an application running in the foreground of the terminal 11, for example, a page of a news and information application being browsed by the user, or a public account page of a social software, etc.

[0038] In some embodiments, when the page script is not obtained, the reading application 12 can also obtain the reading text in the current page through the accessibility service; when the reading text cannot be obtained through the accessibility service, optical character recognition can be performed on the current page to obtain the reading text in the current page; and the reading text can be sent to the text-to-speech server 13.

[0039] It should be noted that the above embodiment only describes the general architecture of the implementation environment, and the system may also include more or fewer components, or combine certain components. For example, the implementation environment may not include a text-to-speech server, and the text-to-speech function may be implemented through the reading application 12. Such variations are all within the scope of protection of this application.

[0040] See also Figure 2 , Figure 2 1 is a flow chart of a method for reading aloud a page provided by an exemplary embodiment of the present application. In some embodiments, the method is performed by a reading aloud application. In some embodiments, the reading aloud application is run in the background of the terminal. The method comprises the following steps:

[0041] Step 201 , in response to a page reading operation on a current page, obtaining a page script of the current page.

[0042] The current page may be a page running in the terminal foreground, such as a page of a news application that the user is browsing, or a public account page of a social software, etc. The current page may be a page corresponding to a third-party application, or a page corresponding to an application that comes with the terminal system, and this embodiment of the application does not limit this.

[0043] Optionally, the page script of the current page is a Hyper Text Markup Language (HTML) script, or an Extensible Markup Language (XML) script, or other types of page scripts.

[0044] Regarding the triggering method of the page reading operation, in some embodiments, the user can trigger the page reading operation by clicking the wake-up button or sidebar corresponding to the reading application; in other embodiments, the user can trigger the page reading operation through special gestures (such as two-finger click) or shortcut keys; in other embodiments, the user can also wake up the page reading operation based on voice commands. The embodiments of the present application are not limited to this.

[0045] Regarding the method of obtaining the page script, in some embodiments, the page script can be obtained by parsing based on the activity context (Activity Context) corresponding to the current page; in other embodiments, the page script can be obtained based on the uniform resource locator (Uniform Resource Location, URL) corresponding to the current page.

[0046] Step 202: When the page script is obtained, extract the reading text from the page script.

[0047] In some embodiments, all text displayed in the current page can be extracted from the HTML script as the read-aloud text.

[0048] In other embodiments, non-text content such as picture content and advertisement content in the current page may be filtered based on HTML script to obtain the reading text corresponding to the current page.

[0049] Step 203: If the page script is not obtained, obtain the reading text in the current page through the accessibility service.

[0050] The accessibility service is a system service that has the ability to obtain the spoken text in the current page.

[0051] In some embodiments, the accessibility service is a system-level program interface provided by the Android system for simulating operations. When the user authorizes the use of the accessibility service, the accessibility service can control the terminal by simulating operations. For example, it can perform continuous operations on behalf of the user (such as automatically grabbing red envelopes, automatically liking, automatically replying, etc.), or help users who cannot fully interact with the device (such as visually impaired users) perform operations, etc.

[0052] In some embodiments, the webview node of the current page is used to store the text content in the page. Based on the setting principles of the accessibility service, the read-aloud text can be extracted from the webview node through the accessibility service.

[0053] It should be noted that any system service that has the ability to obtain the text to be read aloud in the current page can be referred to as an accessibility service in the embodiments of the present application. For example, a system service that has the ability to obtain the text to be read aloud in the current page in systems other than the Android system can also be referred to as an accessibility service in the embodiments of the present application, or other system services in the Android system that have the ability to obtain the text to be read aloud in the current page can also be referred to as an accessibility service in the embodiments of the present application. Such variations are all within the protection scope of the embodiments of the present application.

[0054] In some cases, the page script of the current page may not be obtained because the application corresponding to the current page blocks the page script or due to limitations in parsing capabilities. In this case, the terminal can enable the Accessibility Service function, and the reading application obtains the reading text in the current page through the Accessibility Service.

[0055] Step 204: When the text to be read aloud cannot be obtained through the barrier-free service, optical character recognition is performed on the current page to obtain the text to be read aloud in the current page.

[0056] Because the application corresponding to the current page does not support accessibility services, in some cases it may not be possible to obtain the read text through the accessibility service; or the accessibility service times out, resulting in too long a time to obtain the read text through the accessibility service. In this case, optical character recognition can be used to determine the read text in the current page.

[0057] Optionally, a screenshot of the current page may be taken, and the text in the screenshot may be recognized by optical character recognition to obtain the text to be read aloud in the current page.

[0058] Optionally, non-text content such as picture content and advertisement content may be removed from the screenshot, and then the text in the screenshot may be recognized by optical character recognition to obtain the text to be read aloud in the current page.

[0059] Step 205: play the reading audio corresponding to the reading text.

[0060] In some embodiments, the reading application can generate and play the reading audio based on the reading text corresponding to the current page.

[0061] In other embodiments, the reading application may pass the reading text to a text-to-speech server, which processes the reading text to obtain reading audio and returns it to the reading application for playback.

[0062] Optionally, a reading panel may be displayed in the reading application, and the reading panel is used to display the reading text of the current page and play the reading audio of the current page.

[0063] To summarize, by extracting the reading text from the page script when the page script is obtained; obtaining the reading text through the accessibility service when the page script is not obtained; and obtaining the reading text through optical character recognition when the reading text cannot be obtained through the accessibility service, the probability of obtaining the reading text corresponding to the current page can be increased, thereby improving the accuracy of page reading.

[0064] Applications corresponding to different pages may have different page parsing rules. For example, application A can obtain the page script of current page A1 through the context.string function based on the activity context corresponding to the activity of current page A1; while application B may need to obtain the page script of current page B1 through other functions in the activity context based on the activity context corresponding to the activity of current page B1.

[0065] Therefore, in some embodiments, to obtain the page script of the current page, the target page parsing rule corresponding to the current page can be obtained from the rule database; based on the target page parsing rule, the activity context corresponding to the current page is parsed to obtain the page script of the current page.

[0066] The target page parsing rule may include: based on which function or functions in the activity context of the current page, the page script of the current page can be parsed.

[0067] Optionally, the rule database is a local database of the terminal, and the rule database stores page parsing rules corresponding to different sample pages. Exemplarily, the rule database stores page parsing rule 1 corresponding to the sample page of the news application, page parsing rule 2 corresponding to the sample page of the browser application, page parsing rule 3 corresponding to the sample page of the public account, page parsing rule 4 corresponding to the sample page of the social application, and so on.

[0068] See also Figure 3 , Figure 3 It is a schematic diagram of parsing a page script and extracting aloud text from a page script provided by an exemplary embodiment of the present application.

[0069] like Figure 3 As shown, the reading application 310 is integrated with a page parsing toolkit 311. The page parsing toolkit 311 is used to provide data interaction services between the reading application 310, the Android underlying framework code 320, the rule database 340 and the non-text filtering model 350 in the process of parsing the page script and extracting the reading text from the page script.

[0070] The page parsing toolkit 311 includes a rule acquisition component (Rule Module) 3111 , a client interface component (Client API Module) 3112 for connecting to the reading application 310 , and a model interface component (AI Interface Module) 3113 for connecting to the non-text filtering model 350 .

[0071] In some embodiments, the rule acquisition component 3111 can obtain page parsing rules corresponding to different sample pages through the page rule server 330 and store them in the local rule database 340 of the terminal.

[0072] In some embodiments, in response to a page reading operation for the current page, the reading application 310 transmits the activity corresponding to the current page to the client interface component 3112. The client interface component 3112 obtains the target page parsing rule corresponding to the current page stored in the rule database 340 through the rule acquisition component 3111.

[0073] In some embodiments, the client interface component 3112 may convert the activity corresponding to the current page to determine the activity context corresponding to the current page.

[0074] Activity is an Android application component that provides a screen for interaction with the user; activity context is the base class of activity, which provides the context of application global information (such as application environment variables, etc.). In some embodiments, the activity context includes functions such as context.string, which can be used to obtain the page script of the current page.

[0075] In some embodiments, the client interface component 3112 passes the activity context and the target page parsing rules to the Android underlying framework code 320, and the Android underlying framework code 320 parses the activity context corresponding to the current page based on the target page parsing rules, obtains the page script of the current page, and returns it to the client interface component 3112.

[0076] In some embodiments, the client interface component 3112 passes the activity context and target page parsing rules to the add-on toolkit (AddOn SDK) 321; the add-on toolkit 321 is used to receive the activity context and target page parsing rules, and pass them to the window management service (OplusWindowManagerServiceEx) 322, the window management service 322 is used to receive the activity context and target page parsing rules, and pass them to the target window (ColorW) 323 through the ViewRootImpl.W interface; the target window 323 is used to parse and obtain the page script corresponding to the current page based on the activity context and target page parsing rules, and pass the page script to the client interface component 3112.

[0077] In this embodiment, the activity context corresponding to the current page is parsed through the target page parsing rules. Compared with the usual method of obtaining the page script based on the page URL, when the current page has been displayed on the terminal through page rendering based on the page URL, there is no need to access the current page again through the page URL and parse the page script, thereby reducing data traffic consumption and improving the speed of obtaining the page script.

[0078] After obtaining the page script of the current page, in order to filter the non-text content such as picture content and advertisement content in the current page, in some embodiments, the rule database 340 also stores filtering rules corresponding to different sample pages.

[0079] The filtering rule is a rule for filtering non-text content in a page. Optionally, the non-text content includes image content, advertising content, and the like.

[0080] In some embodiments, the filtering rules can be determined based on HTML tags in the HTML script of the filtering page. Optionally, HTML tags are keywords enclosed in angle brackets, such as, <script>、、<h1>等。可选的，过滤规则包括过滤页面中非正文内容所对应的HTML标签，例如将标签、<h1>标签对应的内容视作非正文内容。

[0081] 在一些实施例中，客户端接口组件3112通过规则获取组件3111，从规则数据库340中获取当前页面的过滤规则，并将过滤规则和页面脚本传递给模型接口组件3113，由模型接口组件3113输入至非正文过滤模型350中，得到去除了非正文内容的朗读文本。模型接口组件3113再将朗读文本通过客户端接口组件3112返回至朗读应用程序310中，以使朗读应用程序310对该朗读文本进行播放。

[0082] 其中，非正文过滤模型350为预先训练好的机器学习模型。例如，非正文过滤模型是基于大量样本页面的过滤规则和页面脚本，以真实的正文内容为监督训练得到的机器学习模型。

[0083] 在一种可能的场景中，由于未获得当前页面对应的应用程序授权，规则数据库中可能不存在当前页面对应的目标页面解析规则，在这种情况下，可以获取当前页面的页面URL，并基于页面URL获取当前页面的页面脚本，以弥补无法通过目标页面解析规则和活动上下文解析出页面脚本的情况。

[0084] 同时，由于未获得当前页面对应的应用程序授权，因此规则数据库中可能也未存储有当前页面对应的目标过滤规则，此时可以利用与当前页面较为相似的其他页面，完成对当前页面对应的活动上下文的解析。

[0085] 在一些实施例中，在规则数据库中不存在当前页面对应的目标过滤规则的情况下，确定页面脚本与样本页面脚本之间的格式相似度。

[0086] 其中，样本页面脚本是指规则数据库中不同样本页面对应的页面脚本。

[0087] 可选的，基于页面脚本的HTML标签，与样本页面脚本的HTML标签，确定格式相似度。例如，当页面脚本中标签在<body>中的数量和位置，与样本页面脚本接近时，可以确定格式相似度较高；或是当页面脚本中HTML标签的类型与样本页面脚本差异较大时，可以确定格式相似度较低。

[0088] 在一些实施例中，可以将规则数据库中与页面脚本具有最高格式相似度的样本页面脚本对应的过滤规则确定为目标过滤规则。例如，在规则数据库中不存在当前页面A1对应的目标过滤规则的情况下，规则数据库中存在样本页面A2与当前页面A1的格式相似度为90％，且90％为最高格式相似度，则可以基于样本页面A2对应的过滤规则确定为当前页面A1的目标过滤规则，并基于该目标过滤规则以及当前页面A1的页面脚本，通过非正文过滤模型过滤非正文内容，得到当前页面A1的朗读文本。

[0089] 本实施例中，在规则数据库中不存在当前页面对应的目标过滤规则的情况下，通过将最高格式相似度的样本页面脚本对应的过滤规则确定为目标过滤规则，可以实现对当前页面的页面脚本进行较为准确地非正文过滤，从而提升了朗读文本的准确性。

[0090] 由于当前页面对应的应用程序对页面脚本做了屏蔽，或是出于解析能力的限制等原因，在一些情况下可能无法获取到当前页面的页面脚本。在未获取到页面脚本的情况下，可以通过无障碍服务获取当前页面中的朗读文本。

[0091] 无障碍服务是一种应用程序，可以实现通过辅助功能服务来代替用户执行连续性的操作(如自动抢红包，自动点赞，自动回复等)，或是帮助无法和设备完全交互的用户(如视力障碍用户)执行操作。

[0092] 基于无障碍服务的设置原则，当前页面的webview节点中用于存储页面中正文内容，因此可以从webview节点中提取朗读文本。

[0093] 参见图4，图4是本申请一个示例性实施例提供的通过无障碍服务获取朗读文本的流程图。该流程图包括以下步骤。

[0094] 步骤401，开启无障碍服务。

[0095] 在一些实施例中，在未获取到页面脚本的情况下，可以开启无障碍服务，并通过无障碍服务获取当前页面中的朗读文本。

[0096] 可选的，在未获取到页面脚本的情况下，朗读应用程序向终端发送无障碍服务请求，由终端基于无障碍服务请求自动开启无障碍服务。

[0097] 步骤402，通过无障碍服务获取当前页面的根节点。

[0098] 在一些实施例中，在当前页面支持无障碍服务的情况下，通过无障碍服务获取当前页面的根节点。

[0099] 在一些实施例中，在当前页面不支持无障碍服务的情况下，可以确定无法通过无障碍服务获取朗读文本，从而后续可采用其他方式(如光学字符识别)以提取朗读文本，避免获取朗读文本失败的情况。

[0100] 步骤403，判断当前节点是否为webview节点。

[0101] 在当前节点为webview节点的情况下，执行步骤406，结束。webview节点为存储页面中正文内容的节点，因此，可以从webview节点中提取当前页面中的朗读文本。

[0102] 在当前节点不是webview节点的情况下，执行步骤404。

[0103] 步骤404，判断当前遍历深度是否达到深度阈值。

[0104] 可选的，深度阈值为正整数值，如10。

[0105] 在当前遍历深度达到深度阈值的情况下，执行步骤406，结束。由于当前遍历深度达到深度阈值，因此可以确定无法通过无障碍服务获取朗读文本，从而后续可采用其他方式(如光学字符识别)以提取朗读文本，从而避免用户等待时间过长。

[0106] 在当前节点不是webview节点，且当前遍历深度未达到深度阈值的情况下，执行步骤405。

[0107] 步骤405，进行深度遍历。

[0108] 例如，可以通过层次优先遍历等方式，直到查找到webview节点，或当前遍历深度达到深度阈值。

[0109] 步骤406，结束。

[0110] 本实施例中，通过无障碍服务获取webview节点，进而得到页面中的正文内容，一方面可以获得准确的朗读文本；另一方面在当前页面不支持无障碍服务，或，当前遍历深度达到深度阈值的情况下，确定无法通过无障碍服务获取朗读文本，可以避免获取朗读文本失败的情况，或是避免用户等待时间过长。

[0111] 在无法通过无障碍服务获取朗读文本的情况下，可以对当前页面进行光学字符识别，以得到当前页面中的朗读文本。在进行光学字符识别之前，还可以先基于正文识别模型确定当前页面中的正文内容区域，以提升光学字符识别的准确度。

[0112] 参见图5，图5是本申请一个示例性实施例提供的通过光学字符识别获取朗读文本的流程图。该流程图包括以下步骤。

[0113] 步骤501，对当前页面进行截图，得到页面截图。

[0114] 可选的，页面截图可以是当前页面在终端屏幕显示范围内的截图，也可以是整个当前页面的长截图。

[0115] 步骤502，通过正文识别模型对页面截图进行正文区域识别，得到页面截图中的正文内容区域。

[0116] 可选的，正文内容区域为页面截图中包含当前页面的正文内容的区域。

[0117] 可选的，正文识别模型为机器学习模型，用于从页面截图中框选出正文内容区域。例如，可以采集不同应用程序的不同样本页面作为训练样本，框选样本页面中的真实正文内容区域作为监督，训练得到正文识别模型。

[0118] 通过正文识别模型，可以去除页面截图中包括非正文内容区域，如页面顶部包括时间图标、电量图标、蓝牙图标等的导航栏区域，或是广告内容区域、图片内容区域等。

[0119] 步骤503，对页面截图中的正文内容区域进行光学字符识别，得到当前页面中的朗读文本。

[0120] 可选的，当页面截图中有多个的正文内容区域时，可以依据正文内容区域的排列位置，将不同正文内容区域对应的朗读文本进行拼接，得到当前页面对应的朗读文本。例如，将位于下方的正文内容区域对应的朗读文本，拼接于位于上方的正文内容区域对应的朗读文本之后。

[0121] 本实施例中，通过正文识别模型确定正文内容区域，并对正文内容区域进行光学字符识别，得到当前页面中的朗读文本，可以提升朗读文本的准确度，避免朗读文本中包括非正文内容，从而提升朗读音频的质量。

[0122] 为基于朗读文本合成朗读音频，在一些实施例中，朗读应用程序将朗读文本发送给文字转语音服务器，由文字转语音服务器基于朗读文本合成朗读音频之后，将朗读音频返回给朗读应用程序。

[0123] 在一些实施例中，文字转语音服务器对朗读应用程序发送的整段朗读文本进行合成之后，才将朗读音频返回给朗读应用程序，这会导致用户等待的时间过长。

[0124] 在一些实施例中，朗读应用程序可以和文字转语音服务器建立长连接。

[0125] 其中，长连接是指在朗读应用程序与文字转语音服务器的一个连接上可以连续发送多个数据包。

[0126] 可选的，长连接为HTTP持久连接(Hyper Text Transfer Protocol persistentconnection，超文本传输协议长连接)。

[0127] 可选的，长连接为基于TCP(Transmission Control Protocol，传输控制协议)、UDP(User Datagram Protocol，用户数据报协议)、QUIC(Quick UDP InternetConnections protocol，快速UDP网络连接)、WebSocket(双向通信协议)等一个或多个协议建立后可以持续收发消息的数据通路。

[0128] 在一些实施例中，朗读应用程序通过长连接向文字转语音服务器发送朗读文本。

[0129] 在一些实施例中，朗读应用程序通过长连接持续接收文字转语音服务器发送的音频数据包，且音频数据包中包含已转换得到的部分朗读音频，且不同音频数据包对应朗读文本中不同的文本片段。

[0130] 例如，当朗读应用程序向文字转语音服务器发送整段朗读文本时，文字转语音服务器持续合成每一行朗读文本对应的部分朗读音频，并将部分朗读音频持续返回给朗读应用程序。

[0131] 本实施例中，通过与文字转语音服务器建立长连接，朗读应用程序可以实现持续接收文字转语音服务器发送的不同文本片段对应的音频数据包，并对不同文本片段对应的音频数据包依次进行播放，从而减少了用户的等待时间，避免了跳段朗读时延过长的问题，增强了音频播放的体验。

[0132] 在文字转语音服务器对朗读文本进行处理时，若朗读应用程序一次性发送的朗读文本较长，则文字转语音服务器的响应时间也相对较长。因此，当朗读文本的字数较多时，可以对朗读文本进行拆分，以减少文字转语音服务器的响应时间。

[0133] 在一些实施例中，在朗读文本的字数超过字数阈值的情况下，朗读应用程序可以对朗读文本进行文本拆分，得到至少两段朗读子文本。其中，朗读子文本的字数不超过字数阈值。例如，字数阈值为2000，某段朗读文本字数为5300，则朗读应用程序可以将该段朗读文本拆分为三段朗读子文本，三段朗读子文本的字数分别为1500字、2000字和1800字。

[0134] 可选的，可以通过在朗读文本中添加标点符号、换行等方式，对朗读文本进行文本拆分，得到至少两段朗读子文本。

[0135] 在一些实施例中，朗读应用程序可以对各段朗读子文本进行转换，得到朗读文本对应的至少两段朗读子音频。

[0136] 在另一些实施例中，朗读应用程序也可以将各段朗读子文本发送给文字转语音服务器，以使文字转语音服务器依次返回朗读文本对应的至少两段朗读子音频，并对至少两段朗读子音频进行依次播放。

[0137] 本实施例中，在朗读文本的字数超过字数阈值的情况下，通过对朗读文本进行文本拆分，可以减少基于朗读文字生成朗读音频的响应时间，从而也减少了用户的等待时间，避免了跳段朗读时延过长的问题，增强了音频播放的体验。

[0138] 参见图6，图6是本申请另一个示例性实施例提供的页面朗读方法的流程图。该流程图包括以下步骤。

[0139] 步骤601，接收针对当前页面的页面朗读操作。

[0140] 关于页面朗读操作的触发方式，在一些实施例中，用户可以通过点击朗读应用程序对应的唤醒按键或侧边栏触发页面朗读操作；在另一些实施例中，用户可以通过特殊手势(如双指点击)或快捷键触发页面朗读操作；在另一些实施例中，用户还可以基于语音指令唤醒页面朗读操作。本申请实施例对此不作限制。

[0141] 步骤602，判断是否能获取到HTML脚本。

[0142] 可选的，通过当前页面对应的活动上下文或是页面URL，获取HTML脚本。

[0143] 在能获取到HTML脚本的情况下，执行步骤603。

[0144] 步骤603，去除非正文内容。

[0145] 在一些实施例中，可以基于当前页面对应的过滤规则以及该HTML脚本，通过非正文过滤模型过滤页面脚本中的非正文内容，得到当前页面对应的朗读文本。

[0146] 在不能获取到HTML脚本的情况下，执行步骤604。

[0147] 步骤604，判断是否能通过无障碍服务获取到朗读文本。

[0148] 在不能通过无障碍服务获取到朗读文本的情况下，执行步骤605。

[0149] 步骤605，通过光学字符识别，获取朗读文本。

[0150] 在一些实施例中，可以获取当前页面的页面截图，并通过光学字符识别从该页面截图中提取出朗读文本。

[0151] 在一些实施例中，还可以基于正文识别模型，从该页面截图中识别出正文内容区域，并通过光学字符识别从正文内容区域中提取出朗读文本，以减少朗读文本中出现非正文内容的可能性，从而获得更准确的朗读文本。

[0152] 在能通过无障碍服务获取到朗读文本的情况下，执行步骤606。

[0153] 步骤606，对朗读文本进行文本拆分。

[0154] 在朗读文本的字数超过字数阈值的情况下，对朗读文本进行文本拆分，得到至少两段朗读子文本，朗读子文本的字数不超过字数阈值；对各段朗读子文本进行转换，得到朗读文本对应的至少两段朗读子音频。

[0155] 步骤607，与文字转语音服务建立长连接。

[0156] 在一些实施例中，朗读应用程序通过长连接持续接收文字转语音服务器发送的音频数据包，且音频数据包中包含已转换得到的部分朗读音频，且不同音频数据包对应朗读文本中不同的文本片段。

[0157] 例如，当朗读应用程序向文字转语音服务器发送整段朗读文本时，文字转语音服务器持续合成每一行朗读文本对应的部分朗读音频，并将部分朗读音频持续返回给朗读应用程序。

[0158] 步骤608，持续接收文字转语音服务器发送的音频数据包。

[0159] 其中，音频数据包中包含已转换得到的部分朗读音频，且不同音频数据包对应朗读文本中不同的文本片段。

[0160] 步骤609，对音频数据包进行播放。

[0161] 仅作示例的，朗读应用程序可以通过调用AudioTrack或MediaPlayer对音频数据包进行播放。

[0162] 在页面朗读的同时，朗读应用程序可以显示朗读面板。朗读面板可以用于同时展示朗读文本和播放朗读音频，并为用户提供与音频播放相关的交互控件。

[0163] 参见图7，图7是本申请一个示例性实施例提供的朗读面板的示意图。

[0164] 如图7所示，终端通过朗读应用程序显示朗读面板701。

[0165] 在朗读面板701中，设置有音色选择控件711和播放设置栏712。其中，音色选择控件711可以供用户下拉选择不同种类的音色，播放设置栏712中包括播放 / 暂停控件、播放上一段控件、播放下一段控件、倍速选择控件、重播控件等控件，以供用户实现相应的功能。

[0166] 在一些实施例中，朗读面板可以对正在播放的朗读文本所在段落或所在行进行特殊显示(如高亮显示或置顶显示)。例如，正在播放段落721对应的朗读音频时，朗读面板对段落721进行高亮显示，对其他段落进行置灰显示。

[0167] 在一些实施例中，响应于对朗读面板中的目标文字的选择操作，播放该目标文字所在段落或所在行对应的朗读音频。

[0168] 例如，当用户对段落722进行点击操作时，正在播放的朗读音频由段落721对应的朗读音频切换为段落722对应的朗读音频。

[0169] 在一些实施例中，不同段落或不同行的朗读文本可以设置不同的索引值，响应于用户对段落722进行点击操作，基于段落722对应的索引值，播放该索引值对应的部分朗读音频。

[0170] 参见图8，图8是本申请一个示例性实施例提供的页面朗读装置的结构框图。该装置包括：

[0171] 页面脚本获取模块801，用于响应于针对当前页面的页面朗读操作，获取所述当前页面的页面脚本；

[0172] 朗读文本获取模块802，用于在获取到所述页面脚本的情况下，从所述页面脚本中提取朗读文本；

[0173] 所述朗读文本获取模块802，还用于在未获取到所述页面脚本的情况下，通过无障碍服务获取所述当前页面中的所述朗读文本；

[0174] 所述朗读文本获取模块802，还用于在无法通过无障碍服务获取所述朗读文本的情况下，对所述当前页面进行光学字符识别，得到所述当前页面中的所述朗读文本；

[0175] 播放模块803，用于播放所述朗读文本对应的朗读音频。

[0176] 可选的，所述页面脚本获取模块801，用于：

[0177] 从规则数据库中获取所述当前页面对应的目标页面解析规则，所述规则数据库中存储有不同样本页面对应的页面解析规则；

[0178] 基于所述目标页面解析规则，对所述当前页面对应的活动上下文进行解析，得到所述当前页面的所述页面脚本。

[0179] 可选的，所述规则数据库中还存储有不同样本页面对应的过滤规则，所述过滤规则为过滤页面中非正文内容的规则；所述朗读文本获取模块802，用于：

[0180] 从所述规则数据库中获取所述当前页面对应的目标过滤规则；

[0181] 基于所述目标过滤规则以及所述页面脚本，通过非正文过滤模型过滤所述页面脚本中的非正文内容，得到所述朗读文本。

[0182] 可选的，基于所述目标过滤规则以及所述页面脚本，通过非正文过滤模型过滤所述页面脚本中的非正文内容，得到所述朗读文本之前，所述朗读文本获取模块802，用于：

[0183] 在所述规则数据库中不存在所述当前页面对应的所述目标过滤规则的情况下，确定所述页面脚本与样本页面脚本之间的格式相似度；

[0184] 将所述规则数据库中与所述页面脚本具有最高格式相似度的所述样本页面脚本对应的过滤规则确定为所述目标过滤规则。

[0185] 可选的，所述页面脚本获取模块801，用于：

[0186] 在所述规则数据库中不存在所述当前页面对应的所述目标页面解析规则的情况下，获取所述当前页面的页面URL；

[0187] 基于所述页面URL获取所述当前页面的所述页面脚本。

[0188] 可选的，所述朗读文本获取模块802，用于：

[0189] 开启无障碍服务；

[0190] 在所述当前页面支持无障碍服务的情况下，通过无障碍服务获取所述当前页面的根节点；

[0191] 在当前节点为webview节点的情况下，从所述webview节点中提取所述朗读文本，所述webview节点为存储页面中正文内容的节点；

[0192] 在当前节点不是webview节点，且当前遍历深度未达到深度阈值的情况下，进行深度遍历。

[0193] 可选的，所述朗读文本获取模块802，用于：

[0194] 在所述当前页面不支持无障碍服务，或，所述当前遍历深度达到所述深度阈值的情况下，确定无法通过无障碍服务获取所述朗读文本。

[0195] 可选的，所述朗读文本获取模块802，用于：

[0196] 对所述当前页面进行截图，得到页面截图；

[0197] 通过正文识别模型对所述页面截图进行正文区域识别，得到所述页面截图中的正文内容区域；

[0198] 对所述页面截图中的所述正文内容区域进行光学字符识别，得到所述当前页面中的所述朗读文本。

[0199] 可选的，所述装置还包括文本拆分模块，所述文本拆分模块，用于：

[0200] 在所述朗读文本的字数超过字数阈值的情况下，对所述朗读文本进行文本拆分，得到至少两段朗读子文本，所述朗读子文本的字数不超过所述字数阈值；

[0201] 对各段所述朗读子文本进行转换，得到所述朗读文本对应的至少两段朗读子音频。

[0202] 可选的，所述装置还包括长连接模块，所述长连接模块，用于：

[0203] 与文字转语音服务器建立长连接；

[0204] 通过所述长连接向文字转语音服务器发送所述朗读文本；

[0205] 通过所述长连接持续接收所述文字转语音服务器发送的音频数据包，所述音频数据包中包含已转换得到的部分朗读音频，且不同音频数据包对应所述朗读文本中不同的文本片段。

[0206] 图9是本申请一个示例性实施例提供的终端的结构框图。

[0207] 该终端可以执行上述实施例的页面朗读方法。该终端可以是支持运行朗读应用程序的电子设备，例如智能手机、笔记本电脑、平板电脑、导航设备、智能电视等。终端还可能被称为用户设备、便携式终端等其他名称。该终端可以包括一个或多个如下部件：处理器910和存储器920。

[0208] 可选的，处理器910利用各种接口和线路连接整个电子设备内的各个部分，通过运行或执行存储在存储器920内的指令、程序、代码集或指令集，以及调用存储在存储器920内的数据，执行电子设备的各种功能和处理数据。可选地，处理器910可以采用数字信号处理(Digital Signal Processing，DSP)、现场可编程门阵列(Field－Programmable GateArray，FPGA)、可编程逻辑阵列(Programmable Logic Array，PLA)中的至少一种硬件形式来实现。处理器910可集成中央处理器(Central Processing Unit，CPU)、图像处理器(Graphics Processing Unit，GPU)、神经网络处理器(Neural-network Processing Unit，NPU)和基带芯片等中的一种或几种的组合。其中，CPU主要处理操作系统、用户界面和应用程序等；GPU用于负责触摸显示屏所需要显示的内容的渲染和绘制；NPU用于实现人工智能(Artificial Intelligence，AI)功能；基带芯片用于处理无线通信。可以理解的是，上述基带芯片也可以不集成到处理器910中，单独通过一块芯片进行实现。

[0209] 存储器920可以包括随机存储器(Random Access Memory，RAM)，也可以包括只读存储器(Read-Only Memory，ROM)。可选地，该存储器920包括非瞬时性计算机可读介质(non-transitory computer-readable storage medium)。存储器920可用于存储指令、程序、代码、代码集或指令集。存储器920可包括存储程序区和存储数据区，其中，存储程序区可存储用于实现操作系统的指令、用于至少一个功能的指令(比如触控功能、声音播放功能、图像播放功能等)、用于实现下述各个方法实施例的指令等；存储数据区可存储根据电子设备的使用所创建的数据(比如音频数据、电话本)等。

[0210] 除此之外，本领域技术人员可以理解，上述附图所示出的终端的结构并不构成对终端的限定，终端可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。

[0211] 本申请实施例还提供一种计算机可读存储介质，该可读存储介质中存储有至少一条指令，至少一条指令由处理器加载并执行以实现上述实施例所述的方法。可选地，该计算机可读存储介质可以包括：ROM、RAM、固态硬盘(SSD，Solid State Drives)或光盘等。其中，RAM可以包括电阻式随机存取记忆体(ReRAM,Resistance Random Access Memory)和动态随机存取存储器(DRAM，Dynamic Random Access Memory)。

[0212] 本申请实施例还提供了一种计算机程序产品或计算机程序，该计算机程序产品或计算机程序包括计算机指令，该计算机指令存储在计算机可读存储介质中。计算机设备的处理器从计算机可读存储介质读取该计算机指令，处理器执行该计算机指令，使得该计算机设备执行上述方面的各种可选实现方式中提供的方法。

[0213] 以上所述仅为本申请的可选的实施例，并不用以限制本申请，凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。< / script>

Claims

1. A page reading method, characterized in that: The method comprises: In response to a page reading operation on a current page, obtaining a page script of the current page; When the page script is obtained, extracting the reading text from the page script; If the page script is not obtained, obtaining the read text in the current page through an accessibility service; In the case where the read-aloud text cannot be obtained through the barrier-free service, performing optical character recognition on the current page to obtain the read-aloud text in the current page; Play the reading audio corresponding to the reading text.

2. The method according to claim 1, characterized in that The step of obtaining the page script of the current page includes: Acquire a target page parsing rule corresponding to the current page from a rule database, wherein the rule database stores page parsing rules corresponding to different sample pages; Based on the target page parsing rule, the activity context corresponding to the current page is parsed to obtain the page script of the current page.

3. The method according to claim 2, characterized in that The rule database also stores filtering rules corresponding to different sample pages, wherein the filtering rules are rules for filtering non-text content in the page; The extracting the reading text from the page script comprises: Acquire the target filtering rule corresponding to the current page from the rule database; Based on the target filtering rule and the page script, the non-text content in the page script is filtered through a non-text filtering model to obtain the read-aloud text.

4. The method according to claim 3, characterized in that Before filtering the non-text content in the page script by a non-text filtering model based on the target filtering rule and the page script to obtain the read text, the method further includes: In the case that the target filtering rule corresponding to the current page does not exist in the rule database, determining the format similarity between the page script and the sample page script; The filtering rule corresponding to the sample page script having the highest format similarity with the page script in the rule database is determined as the target filtering rule.

5. The method according to claim 2, characterized in that: The step of obtaining the page script of the current page further includes: If the target page parsing rule corresponding to the current page does not exist in the rule database, obtaining the page URL of the current page; The page script of the current page is obtained based on the page URL.

6. The method according to claim 1, characterized in that The obtaining the read text in the current page through the barrier-free service includes: Enable accessibility services; In a case where the current page supports accessibility services, obtaining the root node of the current page through the accessibility services; When the current node is a webview node, extracting the read text from the webview node, where the webview node is a node storing the text content in the page; When the current node is not a webview node and the current traversal depth does not reach the depth threshold, perform a depth traversal.

7. The method according to claim 6, characterized in that The method further comprises: When the current page does not support accessibility services or the current traversal depth reaches the depth threshold, it is determined that the read text cannot be obtained through the accessibility services.

8. The method according to claim 1, characterized in that The performing optical character recognition on the current page to obtain the read text in the current page includes: Taking a screenshot of the current page to obtain a page screenshot; Performing text area recognition on the page screenshot using a text recognition model to obtain the text content area in the page screenshot; Optical character recognition is performed on the text content area in the page screenshot to obtain the read-aloud text in the current page.

9. The method according to claim 1, characterized in that: Before playing the reading audio corresponding to the reading text, the method further includes: When the number of words in the read-aloud text exceeds the word count threshold, the read-aloud text is split to obtain at least two read-aloud sub-texts, and the number of words in the read-aloud sub-texts does not exceed the word count threshold; Each segment of the reading sub-text is converted to obtain at least two segments of reading sub-audio corresponding to the reading text.

10. The method according to claim 1, characterized in that Before playing the reading audio corresponding to the reading text, the method further includes: Establish a persistent connection with the text-to-speech server; Sending the reading text to the text-to-speech server through the long connection; The audio data packets sent by the text-to-speech server are continuously received through the long connection, wherein the audio data packets contain the converted partial reading audio, and different audio data packets correspond to different text segments in the reading text.

11. A page reading device, characterized in that: The device comprises: A page script acquisition module, used for acquiring a page script of the current page in response to a page reading operation on the current page; A reading text acquisition module, used to extract the reading text from the page script when the page script is acquired; The read-aloud text acquisition module is further used to acquire the read-aloud text in the current page through an accessibility service when the page script is not acquired; The read-aloud text acquisition module is further used to perform optical character recognition on the current page to obtain the read-aloud text in the current page when the read-aloud text cannot be obtained through the barrier-free service; The playing module is used to play the reading audio corresponding to the reading text.

12. A terminal, characterized in that: The terminal includes a processor and a memory; the memory stores at least one instruction, and the at least one instruction is used to be executed by the processor to implement the page reading method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, and the instruction is loaded and executed by the processor to implement the page reading method according to any one of claims 1 to 10.

14. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor of a terminal reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the terminal executes the page reading method according to any one of claims 1 to 10.

Images