Picture book reading method and related device

Abstract

The invention discloses a picture book reading method and a related device. In the process of resuming reading after the reading of the picture book is interrupted, the device firstly finds the original position of the reader, and then finds the picture book by taking the reader as a main clue. Therefore, the picture book can be continuously read under the condition that the reader is not far away from the picture book, and the speed of finding the picture book by the equipment can be increased. The picture book hit rate after reading recovery can be improved, the reading continuity of a reader can be optimized, and the interactive experience of reading recovery after reading interruption is optimized.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a method and related devices for reading picture books. Background Art

[0002] Picture books are a type of books that are mainly paintings and are accompanied by a small amount of text. Devices such as reading robots can recognize picture books, play the text in the picture books through voice, or play pre-recorded audio files, so that readers can watch picture books while listening to the audio played by the robot, and obtain a dual audio-visual experience.

[0003] In addition to reading picture books, robots and other devices also provide voice interaction functions. During the process of reading picture books, robots and other devices can be awakened by users and interact with users by voice after awakening. Afterwards, the robot can switch from the awakened state back to the reading state and continue to play the audio file. Since the awakening time is uncertain, after returning to the reading state, the position of the reader and the picture book may change compared to before the robot was awakened. How to provide readers with a good audio-visual experience is the direction of current and future research. Summary of the invention

[0004] The present application provides a method and related devices for reading picture books, which can improve the hit rate of picture books after resuming reading, optimize the reading continuity of readers, and enhance the interactive experience of resuming reading after interrupting reading.

[0005] A first aspect provides a method for reading a picture book, which may include:

[0006] Receive a first voice input for reading a picture book; in response to the first voice input, a first image is acquired through the first camera, the first image includes a face of a first user who emits the first voice input; in the first direction, the second image includes a first picture book, and the audio of the first picture book is played;

[0007] Receive a second voice input including a wake-up word; in response to the second voice input, stop playing the audio of the first picture book, turn to face the second user who issued the second voice input, and perform voice interaction with the second user; acquire a third image through the first camera in the second direction, the third image being an image including the face of the first user acquired through the first camera during the voice interaction with the second user;

[0008] Receive a third voice input for reading a picture book; in response to the third voice input, rotate to the second direction; acquire a fourth image through the first camera in the second direction; when the fourth image includes the face of the first user and the angle between the second direction and the first direction is less than a threshold, rotate to the first direction; acquire a fifth image through the second camera in the first direction; when the fifth image includes the first picture book, continue to play the audio of the first picture book.

[0009] A second aspect provides a method for reading a picture book, which may include:

[0010] Receive the first voice input for reading the picture book; in response to the first voice input, a first image is obtained through the first camera, the first image includes the face of the first user who emits the first voice input, and faces the second direction when acquiring the first image; in the first direction, the second image is obtained through the second camera, the second image includes the first picture book, and the audio of the first picture book is played;

[0011] Receive a second voice input containing a wake-up word; in response to the second voice input, stop playing the audio of the first picture book, rotate to the direction facing the second user who emits the second voice input, and perform voice interaction with the second user. The image obtained through the first camera during the voice interaction with the second user does not contain the face of the first user;

[0012] Receive a third voice input for reading a picture book; in response to the third voice input, rotate to the second direction; acquire a fourth image through the first camera in the second direction; when the fourth image includes the face of the first user and the angle between the second direction and the first direction is less than a threshold, rotate to the first direction; acquire a fifth image through the second camera in the first direction; when the fifth image includes the first picture book, continue to play the audio of the first picture book.

[0013] The method of the first aspect is applicable to a scenario in which the device has seen the first user after being awakened.

[0014] The method of the second aspect is applicable to a scenario in which the device has not seen the first user after being awakened.

[0015] Through the method of the first aspect or the second aspect, when the device resumes reading a picture book after being interrupted, the reader's historical location is first found, and then the picture book is found with the reader as the main clue. This ensures that the picture book can be read again when the reader and the picture book are not far apart, and can also increase the speed at which the device finds the picture book. This is equivalent to not only improving the hit rate of the picture book after resuming reading, but also optimizing the reader's reading continuity and improving the interactive experience of resuming reading after interruption.

[0016] In combination with the first or second aspects, in some embodiments, the second image may be an image containing the first picture book acquired through the second camera before receiving the second voice input, or may be an image containing the first picture book newly acquired through the second camera before receiving the second voice input.

[0017] In combination with the first aspect, in some embodiments, the third image may be an image containing the face of the first user acquired through the first camera during voice interaction with the second user, or may be an image containing the face of the first user newly acquired through the first camera during voice interaction with the second user.

[0018] In combination with the first or second aspects, in some embodiments, in the case where the fifth image does not include the first picture book, the first prompt information is output, and the first prompt information is used to prompt the user to place the first picture book.

[0019] In combination with the first or second aspects, in some embodiments, when the fourth image includes the face of the first user, and the angle between the second direction and the first direction is greater than a threshold, the sixth image is acquired through the second camera in the second direction; when the sixth image includes the first picture book, the audio of the first picture book is continued.

[0020] In combination with the previous embodiment, when the sixth image does not include the first picture book, the second prompt information is output, and the second prompt information is used to prompt the user to place the first picture book.

[0021] In combination with the first aspect or the second aspect, in some implementations, when the fourth image does not include the face of the first user,

[0022] If the seventh image is acquired through the first camera in the third direction, and the seventh image includes the face of the first user, the eighth image is acquired through the second camera in the third direction; if the eighth image contains the first picture book, the audio of the first picture book is continued.

[0023] In combination with the previous embodiment, when the eighth image does not include the first picture book, the third prompt information is output, and the third prompt information is used to prompt the user to place the first picture book.

[0024] In combination with the first or second aspects, in some embodiments, in the case where the fourth image does not include the face of the first user, if the image containing the face of the first user is not obtained through the first camera within the first time period, the sleep state or the shutdown state is entered.

[0025] In combination with the first aspect or the second aspect, in some embodiments, before acquiring the second image through the second camera in the first direction, the method further includes: turning off the first camera.

[0026] In combination with the first aspect or the second aspect, in some implementations, the second camera is a wide-angle camera.

[0027] In combination with the first aspect or the second aspect, in some implementations, the steering gear can be rotated to the second direction by adjusting the rotation angle of the steering gear.

[0028] The third aspect provides an apparatus, including: memory, one or more processors; a memory is coupled to one or more processors, and the memory is used to store program code, the program code includes instructions, one or more processors invoke instructions to cause the device to execute a method of any one of the first aspect, or the second aspect, or any one of the first aspects, or any other of the second aspect.

[0029] The fourth aspect provides a readable storage medium, including instructions, when the instructions are run on the device, causing the device to perform a method such as any one of the first aspects, or the second aspects, or any one of the first aspects, or any other of the second aspects.

[0030] The fifth aspect provides a program product, when the program product is running on, a method of causing the device to perform such as any one of the first aspects, or the second aspects, or any one of the first aspects, or any one of the second aspects.

[0031] The sixth aspect provides a chip system, the chip system including at least one processor for implementing a method such as the first aspect, or the second aspect, or any one embodiment of the first aspect, or any one embodiment of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A diagram of a device in the form of a robot provided in an embodiment of the present application;

[0033] Figure 2 A scenario in which the process of reading a picture book by a device provided in an embodiment of the present application is interrupted;

[0034] Figure 3 A flowchart of a method for reading picture books provided in an embodiment of the present application;

[0035] Figure 4 Schematic diagrams of several scenarios in which the device provided in the embodiments of the present application is located;

[0036] Figure 5 A hardware structure diagram of the device provided in the embodiment of the present application;

[0037] Figure 6 A software structure block diagram of the device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0039] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0040] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on the device and finally presented as content that can be recognized by the user. The commonly used form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be a visual interface element such as text, icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets, etc. displayed on the display screen of the device.

[0041] The method of reading picture books provided in this application is applied to the device. The device can be implemented in a variety of product forms, such as robots, smart speakers, smart desk lamps, smart furniture, smart transportation, etc. The device is equipped with a camera and is able to rotate in a direction, and the position of the camera in the space changes after the rotation direction.

[0042] Figure 1 The device 100 is shown as an example in a robot form.

[0043] like Figure 1As shown, the head of the robot may include cameras, such as camera 193-1 and camera 193-2. Camera 193-1 and camera 193-2 can both be used to obtain images within their own framing range. Camera 193-1 can be a wide-angle camera for obtaining images containing picture books; camera 193-2 can be used to obtain images containing human faces. The default initial orientation of camera 193-1 can be below the horizontal line and at a certain angle (such as 20 degrees) to the horizontal line, so that camera 193-1 can obtain images of picture books laid flat on the desktop. The default initial orientation of camera 193-2 can be on the horizontal line. The head of the robot can be rotated to the left, right, up, down or in other directions by a servo. After the position of the robot's head changes, the positions of cameras 193-1 and 193-2 in space change, and the images they can obtain also change accordingly. Equivalently, the robot can adjust the framing of cameras 193-1 and 193-2 by a servo.

[0044] Of course, in some other embodiments, the robot may also include only one camera.

[0045] The robot may also include a microphone array for receiving sound so that the robot can determine the location of the sound source.

[0046] Figure 2 The example shows a scenario in which the process of reading a picture book by a device is interrupted.

[0047] like Figure 2 As shown, the reader ( Figure 2 user1) and the picture book are both located at position P1.

[0048] First, the device 100 faces P1, acquires an image containing the picture book through the camera 193-1 on the head, and plays audio (voice synthesized in the picture book in the picture book, or an audio file of the picture book pre-recorded). When the device 100 recognizes that the picture book exists, it will determine the current picture book location P1, and can continuously refresh the location of the picture book in the future to determine the last location of the picture book it recognizes.

[0049] Then, while the device 100 is playing the audio, the waker ( Figure 2 The device 100 can receive a voice input containing a wake-up word, determine the location P2 of the wake-up person who issued the voice input, turn the head to face P2, and start the voice assistant in response to the wake-up word, and perform voice interaction with the wake-up person based on the voice assistant. After being awakened, the device 100 can turn off the camera 193-1 and start the camera 193-2 to find the wake-up person, so as to face the wake-up person during the conversation with the wake-up person.

[0050] Afterwards, if the waker outputs a voice input instructing the device 100 to read the picture book (such as "read the picture book") or does not output any instructions for a long time, the device 100 resumes the reading mode. Specifically, the reading mode is resumed in the following manner: the device 100 turns off the camera 193-2 and starts the camera 193-1; first faces P2 and obtains an image through the camera 193-1, and recognizes whether there is a picture book based on the image; if there is, the corresponding audio is played; if not, it turns to the position P1 where the picture book last appeared, and recognizes whether there is a picture book again.

[0051] Figure 2 The picture book reading process shown involves the following concepts:

[0052] 1. Reader

[0053] Readers refer to users who have the need to read picture books, such as children. There can be one or more readers.

[0054] 2. The Awakener

[0055] The waker refers to the user who issues the wake-up word. The waker and the reader can be the same user or different users.

[0056] 3. Voice Assistant

[0057] The voice assistant is a function provided by the device, which supports the device to receive user's voice input, use voice recognition technology, natural language processing (such as semantic understanding) technology, etc. to identify user's intentions, and respond in a targeted manner, such as performing voice responses, starting applications, changing device configurations, etc. The device may activate the voice assistant after receiving the wake-up word (which may be a wake-up word that binds the user input).

[0058] Figure 2 The scenario of resuming reading after the picture book reading process is interrupted has the following shortcomings: if the picture book position has not moved, the device 100 first faces P2 and cannot recognize the picture book at P2, and then turns to P1 to recognize the picture book. If a recognition cycle takes about 1.5 seconds, it takes 4 seconds or more to resume reading. Since the picture book position has not moved, the process of the device 100 facing P2 and recognizing the picture book is meaningless to the user, and this part of the time is wasted. In addition, if the reader has left P1, the device 100 turns to P1 to continue reading the picture book, which is a poor experience for the user.

[0059] The following embodiments of the present application provide a method for reading picture books. When the device resumes reading a picture book after the reading is interrupted, the original position of the reader is first found, and then the picture book is found with the reader as the main clue. This ensures that the picture book can be read again when the reader and the picture book are not far apart, and can also improve the speed of the device finding the picture book. This is equivalent to not only improving the hit rate of the picture book after resuming reading, but also optimizing the reading continuity of the reader and optimizing the interactive experience of resuming reading after interruption.

[0060] refer to Figure 3 , Figure 3 The process of the method for reading picture books provided by the present application is exemplified. Figure 3 , the method may include the following steps:

[0061] Stage 1: Reading picture books

[0062] S11, the device 100 starts the camera 193-2 and acquires an image through the camera 193-2.

[0063] Optionally, the camera 193 - 2 can be always on after the device 100 is turned on.

[0064] S12, the device 100 receives a voice input for reading a picture book.

[0065] The device 100 may receive voice input through a microphone or other sound pickup device. The voice input for reading picture books may be preset by the device 100 or set by the user themselves, for example, "Reading picture books" may be input for voice.

[0066] After receiving the voice input for reading the picture book, the device 100 may enter the reading mode. The reading mode may refer to the device 100 executing S13-S15.

[0067] S13, the facial image of the reader is acquired through the camera 193-2, and the device 100 determines the reader's position P2.

[0068] The reader refers to a user who issues a voice input for reading a picture book in S12.

[0069] After receiving the voice input for reading the picture book, the device 100 can search for a reader in response to the voice input.

[0070] Specifically, the device 100 can use microphone array sound source localization technology to determine the direction of the reader according to the voice input issued by the reader in S12, and then rotate to the direction, obtain an image in the direction through the camera 193-2, and determine the face in the image as the face of the reader. The device 100 can rotate the direction by adjusting the posture, adjusting the direction of the head, adjusting the rotation angle of the servo, etc.

[0071] If the image acquired by the camera 193-2 in the direction where the reader is located contains multiple faces, the device 100 can first extract the voiceprint from the voice input, then find the reader's face from the correspondence between the pre-stored voiceprint and the user's face, and then find the reader's face from the multiple faces of the image.

[0072] The reader's position reflects the relative position of the reader in the device 100, and can be reflected by any one or more of the following data: when a face image is obtained, the position of the device 100 (such as the position of the robot), the direction facing the device 100 (such as the camera 193-2), the state of the servo, etc.

[0073] S14, the device 100 turns off the camera 193-2, starts the camera 193-1, and obtains an image through the camera 193-1.

[0074] Camera 193 - 1 may be a wide-angle camera, which is convenient for framing a large area for subsequent identification of picture books.

[0075] S15, the device 100 recognizes the picture book in the image captured by the camera 193-1, and interacts with the reader based on the recognized picture book.

[0076] After the device 100 starts the camera 193-1, it can rotate the direction to change the viewing range of the camera 193-1 until the camera 193-1 obtains an image containing the picture book. When the picture book is within the viewing range of the camera 193-1, the device 100 can obtain the image containing the picture book through the camera 193-1. If the device 100 does not obtain the image containing the picture book, it can output a prompt message to prompt the user to place the picture book within the viewing range of the camera 193-1 until the camera 193-1 obtains the image containing the picture book. The prompt message can be implemented as voice or text on the display screen.

[0077] The device 100 may identify picture books in an image through picture book recognition technology. The embodiment of this application does not limit the picture book recognition technology. For example, a QR code may be printed on the picture book, and the device 100 may identify the QR code in the acquired image to obtain picture book information (such as a logo).

[0078] The ways in which the device 100 interacts with the reader based on the identified picture book may include the following:

[0079] 1. The device 100 plays the audio corresponding to the picture book. For example, the device 100 finds the audio file corresponding to the picture book from the pre-made audio files of the stored multiple picture books and plays it. For example, the device 100 may recognize the text in the acquired image through optical character recognition (OCR), and then combine the text into voice playback.

[0080] 2. The device 100 plays the interactive content corresponding to the picture book, and after the reader's feedback, plays the subsequent interactive content based on the reader's feedback. For example, the device 100 in the form of a robot can play the questions corresponding to the picture book, and the reader can answer by touching the robot's head, left hand, right hand, belly, etc. The sensors at the above positions of the robot can detect the reader's touch operation and continue to play further questions based on the reader's answer.

[0081] 3. The device 100 can perform real-time voice interaction with the reader based on the picture book. The voice interaction process is based on the content of the picture book. The device 100 can make different feedbacks based on different voice inputs of the user.

[0082] During the process of device 100 interacting with the reader based on the recognized picture book, if the location of the picture book changes, device 100 may turn to a new location facing the picture book to find the picture book through the camera 193-1. It is equivalent to that during this process, the location of the picture book may be updated in real time.

[0083] Stage 2: Interrupted reading

[0084] S16, the device 100 receives a voice input including a wake-up word output by the waker.

[0085] The device 100 can receive voice input through a sound pickup device such as a microphone. The wake-up word can be preset by the device 100 or set by the user.

[0086] The device 100 may start the voice assistant in response to the wake-up word in the voice input, and stop interacting with the reader based on the recognized picture book, for example, by stopping playing the audio of the picture book.

[0087] S17, the device 100 determines the location P1 of the picture book.

[0088] In some embodiments, the device 100 may periodically or non-periodically determine the location of the picture book during the process of reading the picture book in stage 1, and after receiving the wake-up word, record the location of the picture book determined last in stage 1 as P1. This is equivalent to S17 being executed at the end of stage 1 and before stage 2. In other embodiments, the location of any picture book determined during the process of reading the picture book in stage 1 may also be recorded as P1. S17 may be executed before S16.

[0089] In other implementations, the device 100 may determine the current location of the picture book and record it as P1 after receiving the wake-up word.

[0090] The location of the picture book reflects the relative position of the picture book in the device 100, and can be reflected by any one or more of the following data: when reading the picture book, the position of the device 100 (such as the position of the robot), the direction facing the device 100 (such as the camera 193-1), the state of the servo, etc.

[0091] S18, the device 100 turns off the camera 193-1 and starts the camera 193-2.

[0092] S19, the device 100 rotates to face the waker and performs voice interaction with the waker.

[0093] The device 100 can use microphone array sound source localization technology to determine the direction of the waker according to the voice input from the waker, and can rotate to that direction. Voice interaction refers to the process in which the waker outputs voice input and the device 100 responds to the voice input through a voice assistant. During the voice interaction process, the waker can move, and the device 100 can also continuously update the position of the waker and adjust the direction in real time to find the waker. The device 100 faces the waker, which can make the voice interaction experience of the waker more friendly.

[0094] S20, during the voice interaction, if the image obtained by the camera 193-2 contains the reader's face image, the reader's location is determined, and the reader's location P2 is refreshed to the reader's location when the camera 193-2 acquires the reader's face image.

[0095] The device 100 may use face recognition technology to determine whether the image acquired by the camera 193 - 2 contains the reader's face in S18 .

[0096] Since the device 100 can turn the direction to find the wakeup during the voice interaction of S19, the camera 193-2 can also acquire images in different directions. If the reader moves the position during the voice interaction, and the moved position is also within the view range of the device 100 , it can be captured and refreshed by the device 100 .

[0097] In some other implementations, the reader position P2 may also be refreshed to any position where the camera 193 - 2 obtains the reader's facial image during the voice interaction process.

[0098] Of course, during the voice interaction, there may also be cases where the images acquired by the camera 193-2 do not contain the reader's face image. In this case, S20 is not required. If the image acquired by the camera 193-2 does not contain the reader's face image during the voice interaction, that is, the device 100 does not capture the reader's face image, then P2 will still be the original data determined in S16.

[0099] Stage 3: Resume reading

[0100] S21, the device 100 receives a voice input for reading a picture book, or does not receive any voice input for a period of time.

[0101] It should be understood that the voice input used to read picture books in S21 and the voice input used to read picture books in S12 may be the same. For example, the voice input in S21 may be "Read picture book", "Read picture book", "Continue reading picture book", etc.

[0102] The voice input for reading picture books in S21 can be issued by the waker, the reader, or other users, without limitation here.

[0103] S22, the device 100 rotates to a position P2 facing the reader.

[0104] The device can face the reader position P2 by adjusting the position, adjusting the direction of the head, adjusting the rotation angle of the servo, etc. The reader location P2 may be the location of the reader determined by the device 100 in S13, or may be the location of any reader determined during the voice interaction of S20, or may also be the latest or last determined reader location during the voice interaction of S20.

[0105] S23, the device 100 determines whether the image acquired by the camera 193-2 when facing the reader's position P2 contains the reader's face.

[0106] If yes, execute S24, if no, execute S30.

[0107] S24, the device 100 determines whether the position angle θ between P2 and P1 is less than a threshold.

[0108] The angle between P1 and P2 refers to the angle between the direction in which the device 100 faces P1 and the direction in which the device 100 faces P2.

[0109] The threshold value may be determined based on the angle θ of the viewing range of the camera 193-1, which may be less than or equal to the angle θ. This threshold may also be continuously updated by the device 100 according to actual conditions. For example, the threshold may be 20°.

[0110] If yes, execute S29, if no, execute S25.

[0111] When the position angle θ of P2 and P1 is smaller than the threshold, assuming that the picture book is still at the position P1 determined in S17, it means that the current picture book and the reader are not far away. Then, executing S29 to face the picture book, the picture book can be more accurately identified.

[0112] When the position angle θ of P2 and P1 is greater than the threshold value, assuming that the picture book is not at the position P1 determined in S17 but moves with the reader, then S25-S26 is executed to face the reader, and the picture book moving with the reader can be recognized.

[0113] S25, the device 100 turns off the camera 193-2 and turns on the camera 193-1.

[0114] S26, the device 100 determines whether a picture book is recognized from the image acquired by the camera 193-1.

[0115] The device 100 may set a preset time (eg, 2 seconds) and determine whether the picture book is recognized within the preset time after the camera 193-1 is activated. The preset time may be referred to as a first time.

[0116] If yes, execute S27, if no, execute S28.

[0117] S27, the device 100 interacts with the reader based on the recognized picture book.

[0118] For the method of identifying picture books and interacting with readers based on the identified picture books, please refer to S15.

[0119] In some embodiments, the device 100 in S26 can further determine whether the same picture book in stage one S15 is recognized. If so, the reading progress at the moment when stage one is interrupted starts to continue playing the audio. Correspondingly, when the device 100 receives the wake-up word in stage two S16, the reading progress can be recorded so as to restore the reading progress later. This solution allows readers to continue reading the same picture book continuously and smoothly. Of course, if the device in S26 does not recognize the same picture book in stage one S15, the device 100 can prompt the user to place the same picture book, or the user can directly read the recognized new picture book.

[0120] In other implementations, if the device 100 recognizes a picture book in S26, but the picture book is not the same picture book as in stage 1 S15, the device 100 may also interact with the reader based on the recognized new picture book.

[0121] S28, the device 100 outputs prompt information, where the prompt information is used to prompt the user to place the picture book.

[0122] The prompt information can be implemented as voice output or text on a display screen.

[0123] The prompt information can be used to prompt the user to place the picture book near the device 100, for example, within the current viewing range of the device 100. For example, the prompt information can be implemented as a voice output "Please put the picture book right in front of me".

[0124] In some implementations, the prompt information may further prompt the user of a specific picture book, for example, prompting the user to place the same picture book in stage one S15.

[0125] If the user places the picture book based on the prompt information, the device 100 can recognize the picture book and continue reading.

[0126] S29, the device 100 rotates to face the position P1 where the picture book is located.

[0127] The picture book location P1 here may be any picture book location determined by the device 100 during the picture book reading process in stage one, or may be the picture book location most recently or finally determined by the device 100 in stage one.

[0128] After executing S29, the device returns to S25.

[0129] If the device 100 enters stage 3 and sequentially executes S21-S24, S29, S25-S27, optionally, a prompt message may be output before S27 to prompt the reader to get close to the picture book. The prompt message may be implemented as a voice output or text on a display screen. When the reader gets close to the picture book, he or she can listen to the audio played by the device 100 while watching the picture book, thereby receiving information from both the visual and auditory senses.

[0130] S30, the device 100 searches for a reader.

[0131] The device 100 can adjust the direction it faces by adjusting the posture, the direction of the head, the rotation angle of the servo, etc., so that different images can be obtained through the camera 193-2, and it can be determined whether these images contain the reader's face. If so, the device finds the reader.

[0132] There are many ways and strategies for the device 100 to adjust the facing direction, which are not limited here. For example, the device 100 can rotate the head 360° in the horizontal direction from the facing direction P2 to find the reader; or, it can first adjust to the position where the reader appears second to last, and then adjust to the position where the reader appears third to last, and so on.

[0133] S31, the device 100 determines whether a reader is found.

[0134] After the device 100 starts searching, if the reader is found, S31 may be executed immediately, and the judgment result is yes.

[0135] After the device 100 starts searching, if the reader is not found after a period of time, S31 may be executed immediately and the judgment result may be negative.

[0136] If the judgment result of S31 is yes, the device 100 returns to S25, or the device 100 updates P2 to the position of the reader when the reader is found and returns to S24.

[0137] If the judgment result of S31 is no, then S32 is executed.

[0138] S32, the device 100 exits the reading mode.

[0139] Exiting the reading mode may include the device 100 turning off the camera 193 - 2 .

[0140] In some implementations, exiting the reading mode may include the device 100 entering a sleep state or a shutdown state.

[0141] The following describes the technical effects of the method for reading picture books provided by the present application in combination with various actual picture book reading scenarios.

[0142] refer to Figure 4 , Figure 4 Five scenarios in which a robot-shaped device may be located are shown as examples.

[0143] The main difference between the scenarios is that when reading resumes in stage 3, the positions of the reader and the picture book are different. Figure 4 There are 5 scenarios in the figure. There are more scenarios in actual applications, which are not listed here one by one.

[0144] exist Figure 4 In the illustrated scenarios, user1 is the reader and user2 is the awakener. The reader in stage 1 is represented by a dotted line, and the reader in stage 3 is represented by a solid line.

[0145] Scenario 1

[0146] In stage 1, user1 and the picture book are located at L1, and the robot faces L1;

[0147] In stage 2, user2 wakes up the robot at L2, and the robot faces L2;

[0148] In stage 3, user1 and the picture book are still in L1.

[0149] In scenario 1, the position P1 of the picture book determined by the robot is L1, the last position P2 of the reader determined by the robot in stage 1 is L1, and the reader's position is not updated during the wake-up process.

[0150] The method of the present application is implemented in scenario 1. After entering phase 3, the robot executes S21-S24, S29, S25-S27 in sequence. Therefore, the robot faces P2 (i.e., L1), can find the reader, recognize the picture book, and interact with the reader based on the picture book. It can be seen that in scenario 1, the robot can quickly find the picture book and resume reading, which can optimize the user's reading continuity, and the picture book and the reader after resuming reading are located in one place to ensure the reader's audio-visual experience.

[0151] Scenario 2

[0152] In stage 1, user1 and the picture book are located at L1, and the robot faces L1;

[0153] In stage 2, user2 wakes up the robot at L2, and the robot faces L2;

[0154] In stage three, the picture book is still at L1, user1 moves to L3, and the position angle θ between the robot and L1 and L3 is less than the threshold.

[0155] In scenario 2, the position P1 of the picture book determined by the robot is L1, the robot sees the reader during the wake-up process, and the latest determined reader position P2 is L3.

[0156] The method of the present application is implemented in Scenario 2. After entering Phase 3, the robot executes S21-S24, S29, S25-S27 in sequence. The robot first faces P2 (i.e., L3) and can find the reader at P2, but because the angle θ between P2 (i.e., L3) and P1 (i.e., L1) is less than the threshold, it turns to P1 (i.e., L1), and then can accurately identify the picture book and interact with the reader based on the picture book. It can be seen that in Scenario 2, after resuming reading, the picture book and the reader are not far apart, the robot can quickly find the picture book and resume reading, optimize the user's reading continuity, and after resuming reading, the picture book and the reader are not far apart, which can also ensure the reader's audio-visual experience.

[0157] Scene 3

[0158] In stage 1, user1 and the picture book are located at L1, and the robot faces L1;

[0159] In stage 2, user2 wakes up the robot at L2, and the robot faces L2;

[0160] In stage three, the picture book and user1 both move to L4, and the position angle θ between the robot and L1 and L4 is greater than the threshold.

[0161] In scenario 3, the position P1 of the picture book determined by the robot is L1, the robot sees the reader during the wake-up process, and the latest determined reader position P2 is L4.

[0162] The method of the present application is implemented in scenario 3. After entering phase three, the robot executes S21-S27 in sequence. The robot first faces P2 (i.e., L4) and can find the reader at P2. Since the angle θ between P2 (i.e., L4) and P1 (i.e., L1) is greater than the threshold, the picture book can be directly recognized at P2 and the reader can interact based on the picture book. It can be seen that in scenario 3, when the reader moves with the picture book and moves a large angle, the robot can also quickly find the picture book and resume reading, optimizing the user's reading continuity.

[0163] Scene 4

[0164] In stage 1, user1 and the picture book are located at L1, and the robot faces L1;

[0165] In stage 2, user2 wakes up the robot at L2, and the robot faces L2;

[0166] In stage 3, the picture book is still at L1, and user1 moves to L5.

[0167] In scenario 4, the position P1 of the picture book determined by the robot is L1. The robot does not see the reader during the wake-up process, and the latest determined reader position P2 is still L1.

[0168] The method of the present application is implemented in scenario 4. After entering stage three, the robot executes S21-S23, S30, S31, S25-S27 in sequence, or executes S21-S23, S30, S31, S25, S26, and S28 in sequence. The robot first faces P2 (i.e., L1) and fails to find the reader at P2, then turns to find the reader. After finding the reader facing L5, it determines whether the picture book can be recognized at this position. If the picture book can be recognized, it is based on the interaction between the picture book and the reader. If not, the user is prompted to place the picture book. It can be seen that in scenario 4, the robot can quickly read and find the reader, thereby resuming reading and optimizing the user's reading continuity.

[0169] Scene 5

[0170] In stage 1, user1 and the picture book are located at L1, and the robot faces L1;

[0171] In stage 2, user2 wakes up the robot at L2, and the robot faces L2;

[0172] In stage 3, the picture book is still located at L1, and user1 moves to L6.

[0173] In scenario 5, the position P1 of the picture book determined by the robot is L1. The robot does not see the reader during the wake-up process, and the latest determined reader position P2 is still L1.

[0174] The method of the present application is implemented in scenario 5. After entering stage 3, the robot executes S21-S23, S30, S31, and S32 in sequence. The robot first faces P2 (i.e., L1) and does not find the reader at P2, then turns to look for the reader, but because L6 where the reader is located is not within the robot's field of view, the robot cannot find the reader. It can be seen that in scenario 5, the robot did not see the reader during the wake-up process and failed to find the reader after resuming reading, indicating that the reader has left. The robot can then exit reading to avoid continuing to read the picture book in a rigid manner, providing users with a flexible usage experience.

[0175] based on Figure 3 In the method shown, in the embodiment of the present application, camera 193-2 can be called the first camera, camera 193-1 can be called the second camera, the reader can be called the first user, and the waker can be called the second user. The picture book recognized by the device 100 can be called the first picture book.

[0176] In Phase 1:

[0177] The voice input for reading the picture book in S12 may be referred to as a first voice input. In response to the first voice input, the image containing the reader's face acquired by the camera 193 - 2 in S13 may be referred to as a first image.

[0178] The image containing the picture book acquired by the camera 193-1 in S15 may be referred to as the second image. The second image may be any image containing the picture book acquired by the camera 193-1 in S15, or may be the last or most recently acquired image containing the picture book in stage 1. The direction facing the device 100 when acquiring the second image may be referred to as the first direction.

[0179] In Phase 2:

[0180] The voice input including the wake-up word in S16 may be referred to as a second voice input.

[0181] If during the voice interaction process, the image acquired by the camera 193 - 2 does not include the reader's face, that is, S20 is not executed, the direction facing the device 100 when acquiring the first image can be referred to as the second direction.

[0182] If during the voice interaction process, the image acquired by the camera 193-2 contains the reader's face, that is, S20 is executed, then the image containing the reader's face acquired by the device 100 through the camera 193-2 during the voice interaction process can be called a third image. The third image can be any image containing the reader's face acquired by the camera 193-2 during the voice interaction process, or it can be the latest or last image containing the reader's face acquired by the camera 193-2. The direction facing the device 100 when acquiring the third image can be called a second direction.

[0183] In Phase Three:

[0184] The voice input for reading the picture book in S21 may be referred to as the third voice input.

[0185] The image acquired by the camera 193 - 2 in S23 may be referred to as a fourth image.

[0186] If S21-S24, S29, S25-S26, S27 / S28 are executed in sequence after entering stage 3, the image captured by camera 193-1 in S25 can be called the fifth image. If the result of S26 is no, the prompt information input in S28 can be called the first prompt information.

[0187] If S21-S26, S27 / S28 are executed in sequence after entering stage 3, the image captured by the camera 193-1 in S25 can be called the sixth image. If the result of S26 is no, the prompt information input in S28 can be called the second prompt information.

[0188] If S21-S23, S30, S31, S25-S26, S27 / S28 are executed in sequence after entering stage 3, the image containing the reader's face acquired by camera 193-2 in S30 can be called the seventh image, and the direction facing when acquiring the seventh image can be called the third direction. Afterwards, the image acquired by camera 193-1 in the third direction in S25 can be called the eighth image. If the result of S26 is no, the prompt information input in S28 can be called the third prompt information.

[0189] based on Figure 3 The method for reading picture books, this application also provides the following extension solutions:

[0190] Extension solutions

[0191] The device 100 can obtain an image through a camera, and use the image to identify the picture book and to identify or find the reader. In other words, the camera 193-1 and the camera 193-2 are combined into one camera.

[0192] refer to Figure 5 , Figure 5 A hardware structure diagram of the device 100 provided in an embodiment of the present application.

[0193] like Figure 5 As shown, the device 100 may include: the device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a sensor module 180, a button 190, an indicator 192, a camera 193, a display screen 194, etc. The sensor module 180 may include a gyroscope sensor 180B, an acceleration sensor 180E, etc.

[0194] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the device 100. In other embodiments of the present application, the device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0195] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0196] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0197] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0198] The charging management module 140 is used to receive charging input from the charger. The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160.

[0199] The wireless communication function of the device 100 can be implemented through an antenna, a wireless communication module 160, a modem processor, a baseband processor, etc.

[0200] The device 100 implements display functions through a GPU, a display screen 194, and an application processor.

[0201] The device 100 can implement a shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0202] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also perform algorithm optimization on the noise and brightness of the image. The ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0203] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to the ISP for conversion into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0204] The internal memory 121 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM).

[0205] The device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0206] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0207] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0208] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The device 100 can be provided with at least one microphone 170C. In other embodiments, the device 100 can be provided with two microphones 170C, which can not only receive sound signals but also realize noise reduction function. In other embodiments, the device 100 can also be provided with three, four or more microphones 170C to realize receiving sound signals, noise reduction, identifying the sound source, realizing directional recording function, etc.

[0209] The gyroscope sensor 180B can be used to determine the motion posture of the device 100. In some embodiments, the angular velocity of the device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the device 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0210] The acceleration sensor 180E can detect the magnitude of acceleration of the device 100 in all directions (generally three axes). When the device 100 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the device posture and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0211] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The device 100 may receive key input and generate key signal input related to user settings and function control of the device 100.

[0212] In the embodiments of the present application:

[0213] The number of cameras 193 may be one or more. For example, the device 100 may include cameras 193-1 and 193-2, wherein the camera 193-1 may be a wide-angle camera. The image acquired by the camera 193-1 may be used by the processor 110 to identify and find picture books, and the image acquired by the camera 193-2 may be used by the processor 110 to identify and find readers.

[0214] The device 100 may also include a steering gear ( Figure 5 (not shown in the figure), the steering gear can adjust the rotation angle to adjust the direction facing the camera 193. In other embodiments, the steering gear can also be implemented as other types of steering devices for adjusting the direction facing the camera 193.

[0215] Microphone 170C may be used to receive voice input from the waker, such as voice input containing a wake-up word, voice input during voice interaction, etc. Microphone 170C may be implemented as a microphone array, and the sound received by the microphone array is used by processor 110 to determine the direction of the waker using the principle of sound source localization.

[0216] The speaker 170A may be used to play the audio corresponding to the picture book, and may also be used to output voice-type prompt information.

[0217] The internal memory 121 can be used to store preset picture book data, such as picture book information (such as identification) and the audio file corresponding to the picture book. In some embodiments, the wireless communication module 160 can also be used to obtain the audio file corresponding to the picture book from the network.

[0218] The internal memory 121 is also used to store the implementation code of the method for reading picture books provided in the present application, and the processor 110 is used to read the code and implement the processing logic of the method. For example, the processor 110 can be used to identify the picture book in the image in stage 1, identify whether the voice input contains a wake-up word in stage 2, determine the picture book position P1, identify the reader's face image, determine the direction in which the device 100 faces, and so on.

[0219] For specific operations performed by each device of the device 100, please refer to Figure 3 The relevant descriptions will not be expanded here one by one.

[0220] The software system of device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture.

[0221] Figure 6 It is a software structure block diagram of the device 100 according to an embodiment of the present application.

[0222] refer to Figure 6 The device 100 may include: a face detection module, a picture book recognition module, a picture book position determination module, a picture book tracking module, a face tracking module, and a direction control module.

[0223] The face detection module is used to obtain the image stream reported by the camera (such as camera 193-2) and identify whether the image stream contains a face.

[0224] The picture book recognition module is used to obtain the image stream reported by the camera (such as camera 193-1), extract the picture book features in the image stream, and recognize the picture book information.

[0225] The picture book position determination module is used to determine the position P1 of the picture book after the picture book recognition module recognizes the picture book.

[0226] The picture book tracking module is used to determine the direction in which the device 100 is facing and whether to look for a reader based on the detection result of the face detection module and the recognition result of the picture book recognition module.

[0227] The face tracking module is used to find the reader according to the decision result of the picture book tracking module, including deciding in which direction the device 100 should search for the reader, determining whether the image reported by the camera contains the reader's face, and refreshing the reader's location P2.

[0228] The direction control module is used to adjust the rotation angle according to the decision result of the picture book tracking module or the face tracking module, thereby adjusting the direction in which the camera 193 faces.

[0229] For the specific functions of the above modules, please refer to Figure 3 The various steps of the method shown will not be repeated here.

[0230] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0231] The present application also provides a device, which may include: a memory and a processor. The memory may be used to store programs; the processor may be used to call the programs in the memory, so that the device executes the method executed on the device side in any of the above embodiments.

[0232] The present application also provides a chip system, which includes at least one processor for implementing the functions involved in the device side in any of the above embodiments.

[0233] In one possible design, the chip system also includes a memory, which is used to store program instructions and data, and the memory is located inside or outside the processor.

[0234] The chip system may be composed of the chip, or may include the chip and other discrete devices.

[0235] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0236] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0237] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0238] The present application also provides a program product, which includes: a program (also referred to as code, or instruction), which, when executed, enables a device to execute the method executed by the device side in any of the above embodiments.

[0239] The present application also provides a readable storage medium, wherein the readable storage medium stores a program (also referred to as code or instruction). When the program is executed, the device executes the method executed by the device side in any of the above embodiments.

[0240] The various implementation modes of the present application can be combined arbitrarily to achieve different technical effects.

[0241] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by a program to instruct the relevant hardware, and the program can be stored in a readable storage medium. When the program is executed, it can include the processes of the above-mentioned method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0242] In short, the above is only an embodiment of the technical solution of this application, and is not intended to limit the protection scope of this application. Any modification, equivalent replacement, improvement, etc. made according to the disclosure of this application shall be included in the protection scope of this application.

Claims

1. A method for reading a picture book, characterized in that: The method comprises: receiving a first voice input for reading a picture book; In response to the first voice input, acquiring a first image through a first camera, wherein the first image includes a face of a first user who issued the first voice input; Acquire a second image through a second camera in a first direction, where the second image includes the first picture book, and play the audio of the first picture book; Receiving a second voice input including a wake-up word; In response to the second voice input, stop playing the audio of the first picture book, turn to face the direction of the second user who issued the second voice input, and perform voice interaction with the second user; Acquire a third image through the first camera in a second direction, where the third image is an image including the face of the first user acquired through the first camera during voice interaction with the second user; receiving a third voice input for reading the picture book; In response to the third voice input, rotating to the second direction; Acquire a fourth image through the first camera in the second direction; When the fourth image includes the face of the first user and the angle between the second direction and the first direction is smaller than a threshold, rotate to the first direction; Acquire a fifth image through the second camera in the first direction; When the fifth image includes the first picture book, the audio of the first picture book continues to be played.

2. A method for reading a picture book, characterized in that: The method comprises: receiving a first voice input for reading a picture book; In response to the first voice input, acquiring a first image through a first camera, wherein the first image includes a face of a first user who issues the first voice input, and the first image is acquired facing a second direction; Acquire a second image through a second camera in a first direction, where the second image includes the first picture book, and play the audio of the first picture book; Receiving a second voice input including a wake-up word; In response to the second voice input, stop playing the audio of the first picture book, turn to face the second user who issued the second voice input, and perform voice interaction with the second user, wherein the image acquired by the first camera during the voice interaction with the second user does not include the face of the first user; receiving a third voice input for reading the picture book; In response to the third voice input, rotating to the second direction; Acquire a fourth image through the first camera in the second direction; When the fourth image includes the face of the first user and the angle between the second direction and the first direction is smaller than a threshold, rotate to the first direction; Acquire a fifth image through the second camera in the first direction; When the fifth image includes the first picture book, the audio of the first picture book continues to be played.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In a case where the fifth image does not include the first picture book, first prompt information is output, where the first prompt information is used to prompt the user to place the first picture book.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When the fourth image includes the face of the first user and the angle between the second direction and the first direction is greater than a threshold, acquiring a sixth image in the second direction through the second camera; When the sixth image includes the first picture book, the audio of the first picture book continues to be played.

5. The method according to claim 4, characterized in that The method further comprises: In a case where the sixth image does not include the first picture book, second prompt information is output, where the second prompt information is used to prompt the user to place the first picture book.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: In the case where the fourth image does not include the face of the first user, If a seventh image is acquired through the first camera in a third direction, and the seventh image includes the face of the first user, an eighth image is acquired through the second camera in the third direction; When the eighth image includes the first picture book, the audio of the first picture book continues to be played.

7. The method according to claim 6, characterized in that The method further comprises: In a case where the eighth image does not include the first picture book, third prompt information is output, where the third prompt information is used to prompt the user to place the first picture book.

8. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: In the case where the fourth image does not include the face of the first user, If no image containing the face of the first user is acquired through the first camera within a first period of time, the device enters a sleep state or a shutdown state.

9. The method according to any one of claims 1 to 8, characterized in that: Before acquiring the second image through the second camera in the first direction, the method further includes: Turn off the first camera.

10. A device, characterized in that: include: memory, one or more processors; The memory is coupled to the one or more processors, and the memory is used to store program codes, where the program codes include instructions, and the one or more processors call the instructions to enable the device to perform the method according to any one of claims 1 to 9.

11. A readable storage medium, comprising instructions, characterized in that: When the instructions are executed on a device, the device executes the method according to any one of claims 1 to 9.

Images