Intelligent integrated reading system and method

Abstract

According to the intelligent integrated reading system and method, the functions of page turning, recognition and output are integrated, the size is small, the functions are comprehensive, and operation is convenient and fast; the page turning mechanism adopts a mechanical arm and a vacuum adsorption technology and can adapt to various book sizes and materials, so that the applicability and the flexibility are improved; character recognition is based on the recognition model, the accuracy is high, complex layout processing such as multi-column typesetting is supported, the recognition efficiency and quality are improved, image restoration and text recognition are achieved based on the neural network, and the reading precision is improved. Voice output supports multi-language recognition and multi-timbre voice synthesis, and the requirements of different user groups are met; personalized setting such as speech speed adjustment is supported, and humanized reading experience is provided for a user; the automation degree is high, page turning and content output can be rapidly completed, and the reading efficiency is improved; and the design is stable and reliable, various book materials and sizes are supported, and the device is suitable for various use scenes.

Description

Technical Field

[0001] The present invention relates to the technical field of reading auxiliary equipment, and more specifically, to an intelligent integrated reading system and method based on artificial intelligence technology. Background Art

[0002] As the aging of society continues, the number of visually impaired people and people with long-term reading needs continues to increase, and traditional reading methods can no longer meet their needs. Existing reading assistive devices have many shortcomings, such as complex operation, single function, and large size, which bring many inconveniences to use.

[0003] At present, although some reading assistance devices can realize functions such as automatic page turning, text recognition and voice output, there are still some problems. For example, the page turning mechanism cannot adapt to books of different sizes and materials, the text recognition accuracy is low and cannot handle complex layouts, and the voice output lacks multi-language support and personalized settings. These problems seriously affect the practicality and ease of use of the device.

[0004] Therefore, there is an urgent need for an intelligent reading device with high integration, convenient operation and stable functions, which can realize automatic book turning, high-precision text and image recognition and multi-language voice output, so as to provide a more humane reading experience for different user groups, and an intelligent integrated reading system and method integrating automatic book turning, text recognition and voice output. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide an intelligent integrated reading system to solve the technical problems that the existing Braille display devices are complex to operate, single in function, and bulky in size, which bring many inconveniences to use, the page turning mechanism cannot adapt to books of different sizes and materials, the text recognition accuracy is low and cannot handle complex layouts, and the voice output lacks multi-language support and personalized settings, etc., which seriously affect the practicality and ease of use of the equipment.

[0006] The present invention provides an intelligent integrated reading system, which includes a machine, a control system integrated on the machine, a photo acquisition module, a text image recognition module, a voice output module and a page turning mechanism; wherein, A book support frame is arranged on the machine platform, and the book support frame is used to fix books; The control system integrates the control logic of the photo acquisition module, the text image recognition module, the voice output module and the page turning mechanism, and the control system is used to regulate and control the machine; The photographing and collecting module includes an adjustable light source system for lighting the book, a high-resolution camera and a pre-trained image collection model; wherein the high-resolution camera is used to take a picture of the to-be-read page of the fixed book to form a photographed image; and the image collection model is used to perform image stitching and image restoration on the photographed image to form a standard image; The text image recognition module is used to perform text recognition and image recognition on the standard image to determine the text area and the image area, and to perform text extraction on the text area to form a reading text, and to perform image description on the image area to form a description text; The speech output module is used to play the reading text and the description text according to a preset playing rule; The page turning mechanism is used to turn the pages of the book after the text image recognition module acquires the reading text or the description text, so as to use the next page of the current page to be read as a new page to be read.

[0007] Optionally, it also includes a voice input unit that is communicatively connected to the control system; wherein, The voice input unit is used to receive user voice to capture the control command issued by the user, so that the control system can perform customized control on the machine based on the control command.

[0008] Optionally, it further includes a translation module that is communicatively connected to the text image recognition module and the speech output module; The translation module is used to translate the reading text and the description text into a target reading text and a target description text in a target language according to a translation instruction of a user, and to enable the speech output module to play the target reading text and the target description text.

[0009] Optionally, the image acquisition model includes a preprocessing unit, a text recognition unit, a context restoration unit, a post-processing unit and an image output unit; wherein, The preprocessing unit is used to perform grayscale conversion processing, median filtering denoising processing, and histogram equalization processing on the captured image, and use a preset Canny operator to perform edge detection on the captured image to extract a paper page layout, call a preset dividing line detection program to detect a dividing line in the middle of the paper page layout, and cut the paper page according to the dividing line to divide it into two pages, a left page and a right page, and cumulatively number the left page and the right page as preprocessing pages; The text recognition unit is used to recognize the text content in the preprocessed page using a preset OCR model, and to infer missing characters according to the context of the text content using a preset BERT model; The contextual repair unit is used to use a preset generative adversarial network and a diffusion model combined with the context to perform text repair on the missing characters to generate a repair text or use a preset PCA approximate text algorithm to generate a repair text; The post-processing unit is used to cover the missing characters with the repair text to complete the pre-processed page; The image output unit is used to take the completed pre-processed page as a standard image and input the standard image into the text image recognition module.

[0010] Optionally, the text image recognition module includes a region division module, a text extraction model and an image description model; wherein, The region division module is used to use the region where the text content recognized by the OCR model is located as a text region, and use the region other than the text region as an image region; The text extraction model is used to extract text from the text area using OCR text recognition technology to obtain reading text; The image description model is used to generate a natural language description of the image in the image area through a preset visual semantic alignment model, and use the natural language description as the description text; the basic architecture of the visual semantic alignment model includes a convolutional neural network and a bidirectional recurrent neural network.

[0011] Optionally, the page turning mechanism includes a mechanical arm disposed above the book support frame and a sensor embedded in the mechanical arm; The end of the mechanical arm is connected to a vacuum suction cup, which is connected to an external vacuum pump through a vacuum pipe and is used to generate negative pressure with the pages of the book to absorb the pressure surface for turning the pages; The sensor is in communication with the control system and is used to detect the page turning progress, the number of overlapped pages and the suction force of the mechanical arm.

[0012] Optionally, the adjustable light source system and the high-resolution camera are arranged directly above the book support frame via symmetrical brackets; Two pressure bars are arranged on the book support frame; the pressure bars can be slidably fixed on the slide rail to adjust the relative width between the two pressure bars, and knobs are arranged on the two pressure bars, and the knobs are used to adjust the height of the pressure bars to adapt to pages of different thicknesses.

[0013] Optionally, a speech speed button, a volume button and a tone button are arranged on the machine.

[0014] The present invention also provides an intelligent integrated reading method, wherein intelligent reading is realized based on the intelligent integrated reading system as described above, comprising: Fixed books; Lighting the fixed book, photographing the pages to be read of the fixed book to form a photographed image, and performing image stitching and image restoration on the photographed image to form a standard image; Performing text recognition and image recognition on the standard image to determine a text area and an image area, performing text extraction on the text area to form a reading text, and performing image description on the image area to form a description text; Play the reading text and the description text according to a preset playing rule; After the text image recognition module acquires the reading text or the description text, the book is page-turned to use the next page of the current page to be read as a new page to be read.

[0015] Optionally, a control command issued by a user is captured to perform customized control on the machine based on the control command.

[0016] It can be seen from the above technical solutions that the intelligent integrated reading system and method provided by the present invention have a high degree of integration, integrating page turning, recognition, and output functions into one, with a compact size, comprehensive functions, and easy operation; the page turning mechanism adopts a mechanical arm and vacuum adsorption technology, which can adapt to a variety of book sizes and materials, thereby improving applicability and flexibility; text recognition is based on a high accuracy rate of the recognition model, and supports complex layout processing, such as multi-column typesetting, etc., thereby improving recognition efficiency and quality, and image restoration and text recognition are realized based on neural networks to improve reading accuracy; voice output supports multi-language recognition and multi-timbre speech synthesis to meet the needs of different user groups; supports personalized settings such as speech speed adjustment to provide users with a humanized reading experience; has a high degree of automation, can quickly complete page turning and content output, and improves reading efficiency; the design is stable and reliable, supports a variety of book materials and sizes, and is suitable for a variety of usage scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more apparent and easy to understand. In the accompanying drawings: Figure 1 A logical schematic diagram of an intelligent integrated reading system according to an embodiment of the present invention; Figure 2 A schematic diagram of a machine of an intelligent integrated reading system according to an embodiment of the present invention; Figure 3 Flowchart of an intelligent integrated reading method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Existing Braille display devices are complex to operate, have single functions, and are bulky, which bring many inconveniences to use. The page-turning mechanism cannot adapt to books of different sizes and materials, the text recognition accuracy is low and cannot handle complex layouts, and the voice output lacks multi-language support and personalized settings, etc. These technical problems seriously affect the practicality and ease of use of the equipment.

[0019] In view of the above problems, the present invention provides an intelligent integrated reading system and method. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0020] In order to illustrate the intelligent integrated reading system and method provided by the present invention, Figure 1-Figure 3 The embodiments of the present invention are exemplarily indicated.

[0021] The following description of exemplary embodiments is merely illustrative in nature and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technologies and devices should be considered as part of the specification.

[0022] like Figure 1 , Figure 2 As shown together, the intelligent integrated reading system provided by the present invention mainly includes a machine 100, a control system 110 integrated on the machine 100, a photo acquisition module 120, a text image recognition module 130, a voice output module 140 and a page turning mechanism 150; wherein, A book support is provided on the machine platform, and the book support is used to fix the books to overcome the preset wind force in life and the force of the book spine on the book pages; The control system 110 integrates the control logic of the photo acquisition module 120, the text image recognition module 130, the voice output module 140 and the page turning mechanism 150, and the control system 110 is used to regulate and control the machine; The photo acquisition module 120 includes an adjustable light source system for lighting the book, a high-resolution camera 121 and a pre-trained image acquisition model 122; wherein the high-resolution camera 121 is used to take a photo of the to-be-read page of the fixed book to form a photographed image; the image acquisition model 122 is used to perform image stitching and image restoration on the photographed image to form a standard image; The text image recognition module 130 is used to perform text recognition and image recognition on the standard image to determine the text area and the image area, and to extract text from the text area to form a reading text, and to describe the image area to form a description text; The speech output module 140 is used to play the reading text and the description text according to the preset playing rules; The page turning mechanism 150 is used to turn the pages of the book after the text image recognition module obtains the reading text or the description text, so as to use the next page of the current page to be read as a new page to be read.

[0023] Specifically, in this embodiment, regarding the machine, the specific appearance and structure of the machine are as follows: Figure 2 As shown, in a specific embodiment, a switch button ( Figure 2 ), the Pause and Start buttons ( Figure 2 The second button from left to right in the middle), speech speed button ( Figure 2 The third button from left to right in the middle can play at double speed), volume button ( Figure 2 The rightmost volume knob) and tone buttons ( Figure 2 ), and the tone selection buttons ( Figure 2 knob from the right to the left in the middle).

[0024] A book support is provided on the machine, and the book support is used to fix books against a preset life wind force. Figure 2 As shown, in a specific embodiment, an adjustable light source system and a high-resolution camera are arranged directly above a book support frame through symmetrical brackets; two pressure bars are arranged on the book support frame; the pressure bars can be slidably fixed on a slide rail to adjust the relative width between the two pressure bars, and knobs are arranged on the two pressure bars, and the knobs are used to adjust the height of the pressure bars to adapt to pages of different thicknesses. In this way, the design is stable and reliable, supports a variety of book materials and sizes, and is suitable for a variety of usage scenarios.

[0025] Regarding the control system 110, a quad-core ARM processor can be used as the core control unit of the control system, integrating the control logic of automatic page turning, photo collection, text image recognition and voice output. The system can be equipped with a touch screen for remote operation, and the touch screen can be implemented based on a mobile phone, a tablet, etc., that is, the control system can be connected to a remote operation device such as a mobile phone, a tablet, etc. based on a local area network and Bluetooth to remotely control the intelligent integrated reading system of this embodiment. At the same time, a voice control system is equipped, and the user can complete mode switching and parameter setting through voice commands. For example, the intelligent integrated reading system in this embodiment also includes a voice input unit 160 that is connected to the control system 110 in communication; wherein, the voice input unit 160 is used to receive the user's voice to capture the control command issued by the user so that the control system can perform custom control on the machine based on the control command, such as performing custom control by voice control on power on, power off, adjusting the volume, adjusting the speed, adjusting the timbre, etc.

[0026] The photo acquisition module 120 includes an adjustable light source system for lighting the book, a high-resolution camera 121 and a pre-trained image acquisition model 122. In this embodiment, the image acquisition model 122 includes a pre-processing unit 1221, a text recognition unit 1222, a context restoration unit 1223, a post-processing unit 1224 and an image output unit 1225; The preprocessing unit 1221 is used to perform grayscale conversion, median filtering denoising, and histogram equalization on the captured image, and use a preset Canny operator to perform edge detection on the captured image to extract the paper page layout, call a preset dividing line detection program to detect the dividing line in the middle of the paper page layout, and cut it according to the dividing line to divide it into two pages, left page and right page, which are pageXLeft.png and pageXRight.png respectively, and the left page and the right page are cumulatively numbered as preprocessing pages, where X is the book page count; more specifically, the imgPreprocessing() method of the ImgPretreat() class is called to preprocess the image, grayscale conversion, median filtering denoising, and histogram equalization are performed on the image, and the Canny operator is used to perform edge detection to extract the paper page; HoughLines is called to detect the dividing line in the middle of the book, and then it is cut according to the black line into two pictures of the left and right pages, which are pageXLeft.png and pageXRight.png respectively, and X is the book page count; The text recognition unit 1222 is used to use a preset OCR model to recognize the text content in the preprocessed page, and use a preset BERT model to infer missing characters according to the context of the text content; the specific structure and training method of the OCR model and the BERT model are not specifically limited, and the ready-made OCR model and BERT model with perfect functions and good training can be directly used; The contextual repair unit 1223 is used to use a preset generative adversarial network and diffusion model combined with the context to repair the missing characters to generate a repair text or use a preset PCA approximate text algorithm to generate a repair text; the specific structure and specific training method of the adversarial network and diffusion model are not limited as long as they can meet the function; The post-processing unit 1224 is used to cover the missing characters with the repair text to complete the pre-processed page; The image output unit 1225 is used to take the completed pre-processed page as a standard image and input the standard image into the text image recognition module.

[0027] Based on this, the intelligent integrated reading system in this embodiment can overcome the problem of inaccurate reading caused by unclear photography in the prior art, and greatly improve the reading accuracy.

[0028] The text image recognition module 130 includes a region division module 131, a text extraction model 132 and an image description model 133; wherein, The region division module 131 is used to use the region where the text content recognized by the OCR model is located as a text region, and use the region other than the text region as an image region; The text extraction model 132 is used to extract text from the text area using OCR text recognition technology to obtain reading text; The image description model 133 is used to generate a natural language description of the image in the image area through a preset visual semantic alignment model, and use the natural language description as a description text; the basic architecture of the visual semantic alignment model includes a convolutional neural network and a bidirectional recurrent neural network. The specific forms of expression of the convolutional neural network and the bidirectional recurrent neural network are not limited, as long as they can meet the function, they will not be elaborated here.

[0029] Based on this, the intelligent integrated reading system of this embodiment can automatically distinguish the text part and the image part in the book, and can then accurately export and read the text part, and can also accurately describe and read the image part.

[0030] The voice output module 140 is used to play the reading text and the description text according to preset playback rules. The voice output module 140 includes a speaker (speaker) for directly playing the reading sound, and also includes a headphone jack for connecting wired headphones for playback. Of course, it can also be connected to Bluetooth headphones, Bluetooth speakers, etc. for playback via Bluetooth, thereby meeting different usage scenarios.

[0031] The page turning mechanism 150 includes a mechanical arm disposed above the book support frame and a sensor embedded in the mechanical arm; the end of the mechanical arm is connected to a vacuum suction cup, which is connected to an external vacuum pump through a vacuum pipe, and is used to generate negative pressure with the pages of the book to adsorb the pressure surface for page turning; the sensor is connected to the control system 110 for communication, and is used to detect the page turning progress, number of re-sheets, and suction force of the mechanical arm, so as to facilitate the control system 110 to adjust the further page turning progress, number of re-sheets, and suction force. The specific size of the vacuum suction cup is not specifically limited, and can be appropriately adjusted according to the commonly used books of the general public.

[0032] In addition, in this embodiment, a translation module 170 is also included that is communicatively connected to the text image recognition module and the voice output module; wherein the translation module is used to translate the reading text and the description text into a target reading text and a target description text in a target language according to a translation instruction of the user, and enables the voice output module to play the target reading text and the target description text, that is, if the user pre-sets a translation instruction, the reading text and the description text are translated into a target reading text and a target description text in a target language through the translation module 170, and enables the voice output module to play the target reading text and the target description text.

[0033] In one specific embodiment, the page turning mechanism is provided with a mechanical arm (the mechanical arm is provided in the middle of the book support frame so as to flexibly realize left and right page turning) and vacuum adsorption technology in the middle position after the book is opened, which can adapt to books of different sizes and materials and realize accurate page turning. The mechanism can adaptively adjust the parameters such as page turning force and speed, and supports different book sizes such as 16-open and 32-open and different paper thicknesses such as thin paper and coated paper. The page turning mechanism also has a page number monitoring function, and confirms whether the page is turned successfully through a photoelectric sensor. The photo acquisition module is equipped with a 20-megapixel high-resolution camera and an adjustable light source system to ensure uniform exposure of the shooting area and avoid reflections and shadows. It also supports the autofocus function, can dynamically detect changes in the height of the book and adjust the focal length in real time, and the image acquisition model supports multi-page splicing, and integrates the cross-page content into a continuous image through the image processing algorithm.

[0034] The text image recognition module has a built-in text image processing unit and integrates the TesseractOCR model based on deep learning to achieve high-precision text recognition, support complex layouts such as multi-column typesetting, and has real-time correction functions. The module supports Chinese, English and 20 other common languages, with a recognition accuracy rate of 97%. It can also extract image content from books and present them in a structured manner.

[0035] The voice output module includes an audio processor and a 5W power speaker, which converts the recognized text content into voice output. The connected translation module supports speech synthesis in 20 languages ​​including Chinese and English, and provides multiple modes such as single-page step-by-step reading, whole chapter reading, and user-specified content reading. The speech speed can be adjusted within the range of 60-200 words / minute to meet the needs of different users. The module also interfaces with barrier-free information communication devices such as hearing aids.

[0036] When using the intelligent integrated reading system of the present embodiment, first, the books to be read are placed on the book support, the pressure bar is adjusted to fix the books, and after the device is started, the page turning mechanism starts working. After each page is turned, the camera shoots the page content and stores it as an image file. Then, the image acquisition model preprocesses the captured image, including grayscale conversion, median filtering denoising, histogram equalization, edge detection to extract paper pages, detect the dividing line in the middle of the book and cut it into left and right pages of pictures, distortion correction and other steps, and outputs the enhanced left and right page image files. The text image recognition module calls the Tesseract OCR model to detect the text area in the picture, and uses the Mask R-CNN model to segment the image area. Then, through the character recognition and image understanding algorithms, the text and image content are output as text documents respectively, and then the voice output module detects whether there is a text document that has not been read aloud. If so, the text content is converted into a voice signal and played through a speaker according to the speech speed set by the user. During the reading process, the user can control the various functions of the device, such as turning pages, adjusting the speech speed, etc., through physical buttons, touch screens connected to it by Bluetooth, or voice commands. For example, when the user issues a "next page" command, the control system will call the page turning mechanism to turn the page. In addition, the device can also be used in reading scenarios for the elderly. Since it supports the speech speed adjustment function, users can set the appropriate reading speed according to their needs to obtain a comfortable listening experience. At the same time, the design of the large-size touch screen and voice control system enables elderly users to easily operate the device.

[0037] As described above, the intelligent integrated reading system provided by the present invention overcomes the preset life wind force size to fix the book through the book support frame, controls the photo acquisition module 120, the text image recognition module 130, the voice output module 140 and the page turning mechanism 150 through the control system 110, so that the photo acquisition module 120 illuminates the book, takes a photo of the page to be read of the fixed book to form a photographed image, performs image stitching and image restoration on the photographed image through the image acquisition model 130 to form a standard image, enables the text image recognition module 130 to perform text recognition and image recognition on the standard image to determine the text area and the image area, and extracts text from the text area to form a reading text, performs image description on the image area to form a description text, enables the voice output module 140 to play the reading text and the description text according to the preset playback rules, and enables the page turning mechanism 150 to turn the book after the text image recognition module obtains the reading text or the description text, so as to use the next page of the current page to be read as a new page to be read. Thus, the page turning, recognition and output functions are integrated into one, with a compact size, comprehensive functions and easy operation; the page turning mechanism adopts a mechanical arm and vacuum adsorption technology, which can adapt to a variety of book sizes and materials, improving applicability and flexibility; text recognition is based on the high accuracy of the recognition model, and supports complex page layout processing, such as multi-column typesetting, which improves recognition efficiency and quality, and realizes image restoration and text recognition based on neural networks to improve reading accuracy; voice output supports multi-language recognition and multi-timbre speech synthesis to meet the needs of different user groups; supports personalized settings such as speech speed adjustment to provide users with a humanized reading experience; with a high degree of automation, it can quickly complete page turning and content output, improving reading efficiency; the design is stable and reliable, supports a variety of book materials and sizes, and is suitable for a variety of usage scenarios.

[0038] In addition, if Figure 3 As shown, the present invention also provides an intelligent integrated reading method, which realizes intelligent reading based on the intelligent integrated reading system as described above, including: S1: fixed books; S2: lighting the fixed book, photographing the pages to be read of the fixed book to form photographed images, and performing image stitching and image restoration on the photographed images to form a standard image; S4: performing text recognition and image recognition on the standard image to determine a text area and an image area, performing text extraction on the text area to form a reading text, and performing image description on the image area to form a description text; S5: Play the reading text and the description text according to a preset playing rule; S6: After the text image recognition module acquires the reading text or the description text, the book is page-turned to use the next page of the current page to be read as a new page to be read.

[0039] In this embodiment, it also includes capturing control commands issued by the user to perform custom control on the machine based on the control commands, for example, receiving user voice to capture control commands issued by the user to perform custom control on the machine based on the control commands.

[0040] The specific implementation example can refer to the specific implementation method of the above-mentioned intelligent integrated reading system, which will not be described in detail here.

[0041] As shown above, the intelligent integrated reading method provided by the present invention is like the above-mentioned intelligent integrated reading system, which generally integrates page turning, recognition, and output functions into one, with a small size, comprehensive functions, and easy operation; the page turning mechanism adopts a mechanical arm and vacuum adsorption technology, which can adapt to a variety of book sizes and materials, and improves applicability and flexibility; text recognition is based on the recognition model with high accuracy, and supports complex layout processing, such as multi-column typesetting, etc., which improves recognition efficiency and quality, and realizes image restoration and text recognition based on neural networks to improve reading accuracy; voice output supports multi-language recognition and multi-timbre speech synthesis to meet the needs of different user groups; supports personalized settings such as speech speed adjustment to provide users with a humanized reading experience; has a high degree of automation, can quickly complete page turning and content output, and improves reading efficiency; the design is stable and reliable, supports a variety of book materials and sizes, and is suitable for a variety of usage scenarios.

[0042] As described above, the intelligent integrated reading system and method proposed in the present invention are described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the intelligent integrated reading system and method proposed in the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. An intelligent integrated reading system, characterized in that: It includes a machine, a control system integrated on the machine, a photo acquisition module, a text image recognition module, a voice output module and a page turning mechanism; wherein, A book support frame is arranged on the machine platform, and the book support frame is used to fix books; The control system integrates the control logic of the photo acquisition module, the text image recognition module, the voice output module and the page turning mechanism, and the control system is used to regulate and control the machine; The photographing and collecting module includes an adjustable light source system for lighting the book, a high-resolution camera and a pre-trained image collection model; wherein the high-resolution camera is used to take a picture of the to-be-read page of the fixed book to form a photographed image; and the image collection model is used to perform image stitching and image restoration on the photographed image to form a standard image; The text image recognition module is used to perform text recognition and image recognition on the standard image to determine the text area and the image area, and to perform text extraction on the text area to form a reading text, and to perform image description on the image area to form a description text; The speech output module is used to play the reading text and the description text according to a preset playing rule; The page turning mechanism is used to turn the pages of the book after the text image recognition module acquires the reading text or the description text, so as to use the next page of the current page to be read as a new page to be read.

2. The intelligent integrated reading system according to claim 1, characterized in that: It also includes a voice input unit that is in communication with the control system; wherein, The voice input unit is used to receive user voice to capture the control command issued by the user, so that the control system can perform customized control on the machine based on the control command.

3. The intelligent integrated reading system according to claim 1, characterized in that: Also included is a translation module that is communicatively connected to the text image recognition module and the speech output module; The translation module is used to translate the reading text and the description text into a target reading text and a target description text in a target language according to a translation instruction of a user, and to enable the speech output module to play the target reading text and the target description text.

4. The intelligent integrated reading system according to claim 1, characterized in that: The image acquisition model includes a preprocessing unit, a text recognition unit, a context restoration unit, a post-processing unit and an image output unit; wherein, The preprocessing unit is used to perform grayscale conversion processing, median filtering denoising processing, and histogram equalization processing on the captured image, and use a preset Canny operator to perform edge detection on the captured image to extract a paper page layout, call a preset dividing line detection program to detect a dividing line in the middle of the paper page layout, and cut the paper page according to the dividing line to divide it into two pages, a left page and a right page, and cumulatively number the left page and the right page as preprocessing pages; The text recognition unit is used to recognize the text content in the preprocessed page using a preset OCR model, and to infer missing characters according to the context of the text content using a preset BERT model; The contextual repair unit is used to use a preset generative adversarial network and a diffusion model combined with the context to perform text repair on the missing characters to generate a repair text or use a preset PCA approximate text algorithm to generate a repair text; The post-processing unit is used to cover the missing characters with the repair text to complete the pre-processed page; The image output unit is used to take the completed pre-processed page as a standard image and input the standard image into the text image recognition module.

5. The intelligent integrated reading system according to claim 4, characterized in that: The text image recognition module includes a region division module, a text extraction model and an image description model; wherein, The region division module is used to use the region where the text content recognized by the OCR model is located as a text region, and use the region other than the text region as an image region; The text extraction model is used to extract text from the text area using OCR text recognition technology to obtain reading text; The image description model is used to generate a natural language description of the image in the image area through a preset visual semantic alignment model, and use the natural language description as the description text; the basic architecture of the visual semantic alignment model includes a convolutional neural network and a bidirectional recurrent neural network.

6. The intelligent integrated reading system according to claim 1, characterized in that: The page turning mechanism includes a mechanical arm disposed above the book support frame and a sensor embedded in the mechanical arm; The end of the mechanical arm is connected to a vacuum suction cup, which is connected to an external vacuum pump through a vacuum pipe and is used to generate negative pressure with the pages of the book to absorb the pressure surface for turning the pages; The sensor is in communication with the control system and is used to detect the page turning progress, the number of overlapped pages and the suction force of the mechanical arm.

7. The intelligent integrated reading system according to claim 1, characterized in that: The adjustable light source system and the high-resolution camera are arranged directly above the book support frame through symmetrical brackets; Two pressure bars are arranged on the book support frame; the pressure bars can be slidably fixed on the slide rail to adjust the relative width between the two pressure bars, and knobs are arranged on the two pressure bars, and the knobs are used to adjust the height of the pressure bars to adapt to pages of different thicknesses.

8. The intelligent integrated reading system according to claim 1, characterized in that: The machine is provided with a speech speed button, a volume button and a tone button.

9. An intelligent integrated reading method, characterized in that: Intelligent reading is realized based on the intelligent integrated reading system according to any one of claims 1 to 8, including: Fixed books; Lighting the fixed book, photographing the pages to be read of the fixed book to form a photographed image, and performing image stitching and image restoration on the photographed image to form a standard image; Performing text recognition and image recognition on the standard image to determine a text area and an image area, performing text extraction on the text area to form a reading text, and performing image description on the image area to form a description text; Play the reading text and the description text according to a preset playing rule; After the text image recognition module acquires the reading text or the description text, the book is page-turned to use the next page of the current page to be read as a new page to be read.

10. The intelligent integrated reading method according to claim 9, characterized in that: Also includes: The control command issued by the user is captured to perform customized control on the machine based on the control command.

Images