Electronic equipment and interaction method

Abstract

The invention provides an electronic device, comprising: a housing comprising a first body and a second body; the processor is used for determining a target display screen from the plurality of display screens according to the position relationship between the first body and the second body and the audio direction; and the target display screen is used for displaying response information, and the response information is interaction information with the audio. The invention further provides an interaction method.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic devices, and in particular, to an electronic device and an interaction method. Background Art

[0002] Electronic devices usually have multiple display screens. For example, foldable mobile phones, foldable tablets, dual-screen portable computers, etc. The switching of multiple screens is based on the folding state of the electronic device. For example, when a foldable mobile phone is in a fully folded state, the outer display screen is switched to the main display screen of the foldable mobile phone; when the foldable mobile phone is in a fully unfolded state, the inner display screen is switched to the main display screen.

[0003] However, automatically switching the display screen only according to the folding state of the electronic device reduces the user experience. Summary of the Invention

[0004] One aspect of the present disclosure provides an electronic device, including: a housing, the housing includes a first body and a second body; a processor, configured to determine a target display screen from a plurality of display screens according to the positional relationship between the first body and the second body and the audio direction, the plurality of display screens are respectively disposed on different surfaces of the outer surface of the housing; a target display screen, configured to display a response message, the response message is an interaction message with the audio.

[0005] According to an embodiment of the present disclosure, the interaction message includes at least one of the following: an application interaction message obtained by processing semantic information corresponding to the audio by a target application in the electronic device, a preset interaction message corresponding to the audio.

[0006] According to an embodiment of the present disclosure, the target application processes the semantic information corresponding to the audio through a target model to obtain an application interaction message; the number of weight parameters of the target model is greater than one hundred million, and / or the output result of the target model is obtained from characteristic information generated one by one by the target model.

[0007] According to an embodiment of the present disclosure, a plurality of display screens are respectively disposed on different surfaces or the same surface of the outer surface of the housing, and the positional relationship includes at least one of the following: relative direction relationship, topological relationship; determining a target display screen from the plurality of display screens according to the positional relationship between the first body and the second body and the audio direction includes: determining the walking direction of the user according to the audio direction, and determining a target display screen that matches the walking direction from the plurality of display screens according to the positional relationship; or determining the walking speed of the user according to the time-domain characteristics of the audio, and when the walking speed is greater than a preset speed threshold, determining a target display screen that matches the walking speed according to the walking speed, and displaying interaction information at a target display speed on the target display screen; or determining the orientations of the plurality of display screens according to the relative direction relationship, and when the audio direction is perpendicular to the orientations of the plurality of display screens respectively, using an image acquisition device to identify the eye position of the user, and determining a target display screen that matches the eye position according to the orientations of the plurality of display screens.

[0008] According to an embodiment of the present disclosure, the target display screen is one or more parts of the plurality of display screens, and the interaction information includes first interaction information and second interaction information; the target display screen is used to display the first interaction information, and the first interaction information is used to instruct the user to go to other display screens to view the second interaction information, and the second interaction information is interaction information corresponding to the user operation instruction characterized by the audio.

[0009] According to an embodiment of the present disclosure, the plurality of display screens include a first display screen disposed on the first surface of the first body; determining a target display screen from the plurality of display screens according to the positional relationship between the first body and the second body and the audio direction includes: when the positional relationship represents that the first included angle between the first body and the second body is less than a preset included angle threshold, determining the target display screen as the first display screen; when the positional relationship represents that the first included angle is greater than or equal to the preset included angle threshold, determining a target display screen from the plurality of display screens according to the audio direction, and the preset included angle threshold is determined according to the height of the user relative to the electronic device.

[0010] According to an embodiment of the present disclosure, the plurality of display screens further includes a second display screen disposed on the second surface of the first body; determining a target display screen from the plurality of display screens according to the audio direction includes: determining the second display screen as the target display screen when the first included angle is greater than or equal to the second included angle between the audio direction and the second body; determining the first display screen as the target display screen when the first included angle is less than the second included angle. According to an embodiment of the present disclosure, the first included angle is determined based on the following operations: determining the first included angle according to the first angle of the first body relative to the reference direction and the second angle of the second body relative to the reference direction; the second included angle is determined based on the following operations: determining the second included angle according to the third angle of the audio direction relative to the reference direction and the second angle; or determining the sound source direction according to the first spatial position relationship between the user and the audio collection device, where the audio collection device is used to collect audio, determining the intermediate direction according to the sound source direction and the audio direction, converting the intermediate direction to the same coordinate system as the first included angle according to the second spatial position relationship between the audio collection device and the second body to obtain the target direction, and determining the included angle between the target direction and the second body as the second included angle.

[0011] According to an embodiment of the present disclosure, the first display screen is covered by a transparent cover plate, and the cover plate includes a first part and a second part, the second part covers the display area of the first display screen, and the second part extends outward from the first part to form a protruding part; or a camera module and / or elements supporting the display of the first display screen are embedded in the protruding part.

[0012] Another aspect of the present disclosure provides an interaction method, including: determining a target display area from a plurality of display areas according to the positional relationship between the first body and the second body in the electronic device and the audio direction, where the plurality of display areas include display areas located on different surfaces of the electronic device; displaying a response message in the target display area, and the response message is an interaction message corresponding to the audio.

[0013] According to an embodiment of the present disclosure, the interaction information includes at least one of the following: application interaction information obtained by processing semantic information corresponding to the audio using a target application in the electronic device, and preset interaction information corresponding to the audio.

[0014] According to an embodiment of the present disclosure, the plurality of display areas includes a first display area disposed on the first surface of the first body; determining a target display area from the plurality of display areas according to the positional relationship between the first body and the second body in the electronic device and the audio direction includes: determining the first display area as the target display area when the positional relationship indicates that the first included angle between the first body and the second body is less than a preset included angle threshold; when the positional relationship indicates that the first included angle is greater than or equal to the preset included angle threshold, determining the target display area from the plurality of display areas according to the audio direction.

[0015] Another aspect of the present disclosure provides an electronic device for performing an interaction method, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the interaction method as described above.

[0016] Another aspect of the present disclosure provides a non-volatile storage medium storing computer-executable instructions that are used to implement the interaction method as described above when executed.

[0017] Another aspect of the present disclosure provides a computer program, the computer program including computer-executable instructions that are used to implement the interaction method as described above when executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more fully understand the present disclosure and its advantages, reference will now be made to the following description taken in conjunction with the accompanying drawings, in which:

[0019] Figure 1 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown;

[0020] Figure 2 A schematic diagram of an electronic device according to an embodiment of the present disclosure is schematically shown;

[0021] Figure 3 A schematic diagram of a first included angle according to an embodiment of the present disclosure is schematically shown;

[0022] Figure 4A A schematic diagram of a second included angle according to an embodiment of the present disclosure is schematically shown;

[0023] Figure 4B A schematic illustration of a target direction according to an embodiment of the present disclosure is schematically shown;

[0024] Figure 5 A schematic diagram of a reference direction according to an embodiment of the present disclosure is schematically shown;

[0025] Figure 6 A schematic diagram of a control according to an embodiment of the present disclosure is schematically shown;

[0026] Figure 7 A schematic diagram of a cover plate according to an embodiment of the present disclosure is schematically shown;

[0027] Figure 8 A flowchart of an interaction method according to an embodiment of the present disclosure is schematically shown;

[0028] Figure 9 A block diagram of an information processing device according to an embodiment of the present disclosure is schematically shown;

[0029] Figure 10 Schematically shown is a schematic block diagram of an electronic device for performing an interaction method according to an embodiment of the present disclosure. Detailed implementation manners

[0030] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily obscuring the concepts of the present disclosure.

[0031] The terms used herein are merely for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0032] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those of ordinary skill in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0033] In the description of this specification, descriptions with reference to terms such as "an embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, without conflict, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples.

[0034] In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of the indicated technical features. Thus, features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present application, "a plurality" means two or more, unless otherwise specifically defined.

[0035] Some block diagrams and / or flowcharts are shown in the accompanying drawings. It should be understood that some blocks in the block diagrams and / or flowcharts, or combinations thereof, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when executed by the processor, these instructions can create a device for implementing the functions / operations illustrated in these block diagrams and / or flowcharts.

[0036] Therefore, the technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). Additionally, the technology of the present disclosure can take the form of a computer program product on a computer-readable medium storing instructions, which can be used by or in conjunction with an instruction execution system. In the context of the present disclosure, a computer-readable medium can be any medium that can contain, store, transmit, propagate, or transport instructions. For example, a computer-readable medium can include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, components, or propagation media. Specific examples of computer-readable media include: magnetic storage devices, such as magnetic tapes or hard disk drives (HDDs); optical storage devices, such as compact discs (CD-ROMs); memories, such as random access memories (RAMs) or flash memories; and / or wired / wireless communication links.

[0037] In the technical solution of the present disclosure, the user information involved (including but not limited to user personal information, user image information, user voice information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or reject.

[0038] An electronic device usually has multiple display screens. For example, foldable mobile phones, foldable tablets, dual-screen portable computers, etc. The switching of multiple screens is based on the folding state of the electronic device. For example, when a foldable mobile phone is in a fully folded state, the outer display screen is switched to be the main display screen of the foldable mobile phone; when the foldable mobile phone is in a fully unfolded state, the inner display screen is switched to be the main display screen.

[0039] However, automatically switching the display screen only according to the folding state of the electronic device results in a reduced user experience.

[0040] Figure 1 A block diagram of an electronic device according to an embodiment of the present disclosure is schematically shown.

[0041] Figure 2Schematically shows a schematic diagram of an electronic device according to an embodiment of the present disclosure.

[0042] As Figure 1 shown, the electronic device 100 may include a housing 110, a processor 120, and a plurality of display screens 130.

[0043] As Figure 2 shown, the housing 110 may include a first body 111 and a second body 112.

[0044] The processor 120 may be configured to determine a target display screen from a plurality of display screens according to the positional relationship between the first body 111 and the second body 112 and the audio direction. The plurality of display screens may be the display screen 131, the display screen 132, and the display screen 133 respectively, or the display screen 131 and the display screen 132, etc. There is no limitation on the position and number of the display screens.

[0045] The target display screen may be used to display response information, and the response information may be interaction information with the audio.

[0046] According to an embodiment of the present disclosure, the positional relationship between the first body 111 and the second body 112 may be a spatial position representing the relationship between the two bodies. For example, the first body 111 is in the northeast direction of the second body 112, the first body 111 is adjacent to the second body 112, the first body 111 is perpendicular to the second body 112, the first body is h meters to the left of the second body, etc.

[0047] By using the gyroscope built in the electronic device, the spatial attitude of the electronic device can be obtained; according to the spatial attitude of the electronic device, the relative orientation between the first body and the second body can be deduced (such as the first body is in the southeast direction of the second body, or any direction among the four cardinal directions and the center). According to the continuous change of the user's audio direction, the target display screen can be determined from a plurality of display screens.

[0048] By using the hinge angle sensor and / or the gravity sensor in the electronic device, the relative angle between the first body and the second body can be measured. When the first body and the second body are at 180 degrees, the first body and the second body may be in an unfolded adjacent state; when the first body and the second body are less than 90 degrees, the first body and the second body may be in a flipped intersecting state.

[0049] In some embodiments, when the first body and the second body are in the flipped intersecting mode, the target display screen corresponding to the audio direction can be determined from a plurality of display screens, and the orientation of the target display screen is the same as the audio direction. For example, the orientation of the target display screen and the audio direction can both be the northeast direction.

[0050] In some embodiments, such as Figure 2As shown, the display screen 131 on the surface of the first body 111 and the display screen 132 on the surface of the first body 111 can be back-to-back with each other. The orientation of the display screen 132 can be the positive direction of the y-axis, and the orientation of the display screen 131 can be the negative direction of the y-axis. The display screen 133 is disposed on one surface of the second body, and the orientation of the display screen 133 can be the positive direction of the x-axis. The second body 112 is adjacent to the first body 111. When the first body 111 and the second body 112 are unfolded to 180 degrees, the third display screen 133 and the second display screen 132 can form a display screen with a larger display area. The electronic device in this embodiment can be a folding device. For example, a folding mobile phone, a folding notebook, a folding tablet, etc.

[0051] According to an embodiment of the present disclosure, the audio can be an electrical signal of sound. For example, the audio can be an electrical signal of user voice, door opening sound, knocking sound, etc.

[0052] According to an embodiment of the present disclosure, in the case where the user voice represents an operation instruction, the response information can be the execution status of the operation instruction. For example, the user voice represents an operation instruction of "turn on the camera". The electronic device detects that the user voice comes from the negative direction of the z-axis. According to the positional relationship between the negative direction of the z-axis, the first body 111 and the second body 112, the target display screen is determined from multiple display screens 130. The response information can be "the image taken after the camera is turned on" or the image for the user to preview the shooting effect before shooting after the camera is turned on, etc. The "image taken after the camera is turned on" or the image for the user to preview the shooting effect before shooting after the camera is turned on is displayed on the target display screen.

[0053] For example, the user voice represents an operation instruction of "briefly describe the advantages of a notebook over a mobile phone". The electronic device detects that the user voice comes from the y direction, and can use the display screen in the y direction to display the answer information of "briefly describe the advantages of a notebook over a mobile phone".

[0054] For example, the user voice represents an operation instruction of "generate a landscape painting". Since the electronic device can be folded, the user's sight is affected by the folding condition of the electronic device. For example, the user is at a position on one side of the display screen 131. The direction of the user relative to the electronic device can be determined according to the audio direction; the display directions of multiple display screens can be determined according to the positional relationship between the first body 111 and the second body 112. The target display screen is determined from multiple display screens according to the direction of the user relative to the electronic device and the multiple display directions. The generated landscape painting is displayed on the target display screen.

[0055] For example, when the electronic device detects that the sound of opening the door comes from the x direction, it determines the target body facing the x direction according to the positional relationship between the first body 111 and the second body 112, and determines the display screen facing the x direction on the target body as the target display screen. The target display screen is used to display preset information such as a clock or a welcome message.

[0056] According to an embodiment of the present disclosure, the audio can be obtained by using at least one audio acquisition device, and the audio acquisition device can be disposed on the electronic device or at a spatial position adjacent to the electronic device.

[0057] According to an embodiment of the present disclosure, the audio direction can be the propagation direction of the audio signal, generally the direction starting from the audio acquisition device. The audio direction can be determined according to the audio parameters of the audio. The audio parameters can include at least one of the following: the time difference of the audio reaching multiple audio acquisition devices respectively, the angles at which the multiple audio acquisition devices respectively receive the audio, and the intensities of the audio received by the multiple audio acquisition devices respectively.

[0058] For example, when sound waves propagate to multiple audio acquisition devices, due to the difference in distance, the time for the audio to reach each audio acquisition device is different. The audio direction can be deduced by the time difference method according to the time difference of the audio reaching multiple audio acquisition devices respectively.

[0059] For example, based on beamforming technology, signal weighting delay is performed on the audio respectively received by multiple audio acquisition devices to synthesize the audio from a preset direction together. Delay analysis is performed on the audio from the preset direction to obtain the audio direction.

[0060] For example, the greater the intensity of the audio, the closer the sound source is to the audio acquisition device. By comparing the intensities of the audio received by multiple audio acquisition devices respectively, the audio direction can be estimated.

[0061] According to an embodiment of the present disclosure, multiple display screens are disposed on different surfaces of the outer surface of the housing of the electronic device. The target display screen can be determined from the multiple display screens according to the positional relationship between the first body and the second body and the audio direction, and the target display screen is used to display response information. Not only the positional relationship between the first body and the second body is considered, but also the audio direction is considered, so the display screen for displaying the response information is more in line with the user's perspective.

[0062] According to an embodiment of the present disclosure, the interaction information can include at least one of the following: application interaction information obtained by processing semantic information corresponding to the audio by a target application in the electronic device, and preset interaction information corresponding to the audio.

[0063] According to an embodiment of the present disclosure, the audio can be an electrical signal representing sound. For example, the audio can be a user voice signal, a preset sound signal, etc. The preset interaction information of the preset sound signal can be a welcome message, a clock display, etc.

[0064] For example, the preset sound can be a tapping sound, and the electronic device can be a folding mobile phone. The folding mobile phone can have three display screens. The first body and the second body of the folding mobile phone can be in a state where they are not fully unfolded (for example, the angle between the first body and the second body is 50 degrees, 90 degrees, 120 degrees, etc.). When receiving the preset sound signal, according to the positional relationship between the first body and the second body and the audio direction of the tapping sound, the target display screen corresponding to the audio direction of the tapping sound can be determined from the three screens. For example, the positional relationship between the first body and the second body can be that the angle between the first body and the second body is 135 degrees. When the angle between the first body and the second body is 135 degrees, the orientations of the three display screens of the folding mobile phone are southeast, northwest, and northeast respectively. The orientation of the display screen is the same as the audio direction, and the user can view the interaction information on the display screen. The audio direction of the tapping sound is northwest, and the display screen corresponding to the audio direction is the one with the northwest orientation. Therefore, the display screen with the northwest orientation is determined as the target display screen. And the clock is displayed on the target display screen to improve the user experience. According to an embodiment of the present disclosure, the semantic information can be determined based on the following operation steps: performing feature fusion on multiple audios to obtain a fused feature; performing speech recognition on the multiple audios based on the fused feature to obtain the semantic information corresponding to the audios.

[0065] According to an embodiment of the present disclosure, performing feature fusion on multiple audios to obtain a fused feature can include: adjusting the first audio and the second audio to the same signal energy threshold range respectively to obtain a first target audio and a second target audio; performing feature fusion on the first target audio and the second target audio.

[0066] For example, adjusting the first audio and the second audio to the same signal energy threshold range respectively to obtain a first target audio and a second target audio can include: performing Fourier transform on the first audio and the second audio respectively to obtain the first amplitude of the first audio and the second amplitude of the second audio; performing normalization processing on the first amplitude and the second amplitude to obtain the first target audio and the second target audio.

[0067] For example, performing feature fusion on the first target audio and the second target audio can include: respectively extracting the Mel-frequency cepstral coefficients of the first target audio and the second target audio to obtain a first audio feature and a second audio feature. Concatenating the first audio feature and the second audio feature to obtain a fused feature.

[0068] For example, the fused features are input into a speech recognition model to obtain semantic information corresponding to the audio. The speech recognition model can be a neural network model, a hidden Markov model, or the like.

[0069] The electronic device can be a foldable notebook. The user can wake up a target application in the foldable notebook by voice. In the related art, it is usually detected whether there is a user on the keyboard side of the electronic device. If there is a user, the display screen on the keyboard side is used to display the target application interface. However, the user who wakes up the target application may not be on the side where the electronic device is used, making it difficult for the user who wakes up the target application to see the information on the display screen, resulting in a reduced user experience.

[0070] At the same time, the electronic device can also be a foldable mobile phone. In the related art, when the first body and the second body of the foldable mobile phone coincide (the angle between the first body and the second body is 0 degrees) or the first body and the second body are fully unfolded to 180 degrees, the target application can be woken up by sound, making it difficult for the user to start the target application when the foldable mobile phone is in a semi-folded state (such as the angle between the first body and the second body is 30 degrees, 90 degrees, 120 degrees, etc.), resulting in a reduced user experience.

[0071] In view of this, the present disclosure can determine a target display screen from multiple display screens according to the positional relationship between the first body and the second body and the audio direction, and use the target display screen to display interaction information corresponding to the audio. The interaction information can include application interaction information obtained by processing the semantic information corresponding to the audio using the target application in the electronic device. Therefore, when the positional relationship between the first body and the second body represents that the electronic device is in a semi-folded state (such as the angle between the first body and the second body is 30 degrees, 90 degrees, 120 degrees, etc.), the target application can also be started; the user who wakes up the target application can see the displayed application interaction information on the target display screen, thereby improving the user experience.

[0072] According to an embodiment of the present disclosure, the target application can instruct the target model to work; the number of weight parameters of the target model is greater than 100 million, and / or the output result of the target model is obtained from the characteristic information generated one by one by the target model.

[0073] According to an embodiment of the present disclosure, the target application can be an intelligent application, which can communicate with the user, perform intelligent writing, language translation, image generation, chart drawing, etc.

[0074] The target model can be a large model, and the number of weight parameters of the large model is greater than 100 million. For example, the large model can be a convolutional neural network, a large language model, or the like.

[0075] The target model can be a generative model, and the output result of the generative model can be obtained from the characteristic information generated one by one by the target model. For example, the generative model can be a text generation model (such as a retrieval generation model, a Transformer-based text generation model, etc.), an image generation model (such as a generative adversarial network, a variational autoencoder, etc.), a speech synthesis model (such as a speech synthesis model based on the Transformer architecture, an end-to-end speech synthesis model, etc.), etc. The large language model can be used to generate accurate answers according to the context of semantic information and the content of the question. For example, for the operation instruction of the user voice representation "Please briefly describe the advantages of a laptop compared to a mobile phone", the large language model can be used to analyze the semantic information of the audio "Please briefly describe the advantages of a laptop compared to a mobile phone" to obtain key information; calculate the similarity between the keywords and the retrieved text, and obtain similar texts. The retrieved text can be obtained by calling a search engine from multiple databases; generate an answer text according to the similar text. The response information can be the answer text.

[0076] The convolutional neural network can be used for image recognition. For the operation instruction of the user voice representation "Turn on the camera", the response information can be "the image captured after the camera is turned on". In response to receiving the user's recognition instruction for "the image captured after the camera is turned on", the convolutional neural network can be used to extract features from "the image captured after the camera is turned on" to obtain image features. Identify the target object from the image based on the image features.

[0077] The generative adversarial network can be used to generate high-quality images. In response to receiving the operation instruction of the user voice representation "Generate a landscape painting", the pre-stored target image can be obtained. The Transformer model is used to perform image processing on the target image based on the noise of the target image to obtain a high-quality landscape painting.

[0078] According to an embodiment of the present disclosure, the target application can process the semantic information corresponding to the audio through the target model to obtain application interaction information. The number of weight parameters of the target model is greater than 100 million, and it can include most deep learning models, such as convolutional neural networks, large language models, etc. Through a large amount of training data, it can have good performance in various complex tasks to accurately lock the needs reflected by the user in the audio; and / or the output result of the target model is obtained from the characteristic information generated one by one by the target model, and it can include text generation models, image generation models, speech synthesis models, etc. Therefore, the target application is an intelligent application integrating multiple functions, such as generating images, videos, music, etc., which can meet the personalized needs of users.

[0079] According to an embodiment of the present disclosure, multiple display screens can be respectively arranged on the outer surface of the housing. For example, multiple display screens can be arranged on different surfaces of the outer surface of the housing, or can be respectively arranged on the same surface of the outer surface of the housing.

[0080] In some embodiments, as Figure 2 shown, multiple display screens can be respectively located on one or more sides of the first body 111 or the second body 112. The display screen 131 on the side of the first body 111 and the display screen 132 on the side of the first body 111 are back to back with each other. The display screen 133 can be arranged on one side of the second body. When the first body and the second body are unfolded to 180 degrees, the third display screen 133 and the second display screen 132 can form a display screen with a larger display area.

[0081] It can also be that multiple display screens are all located on the A side of the first body, multiple display screens are respectively located on different sides of the second body, etc.

[0082] According to the embodiments of the present disclosure, the positional relationship can include at least one of the following: relative direction relationship, topological relationship.

[0083] The relative direction relationship can be that the first body is in the east, south, west, north, southeast, northeast, etc. directions of the second body.

[0084] The topological relationship can include the connection relationship and display relevance between the first body and the second body.

[0085] For example, the first body is adjacent to the second body, and the A display screen in the first body and the B display screen in the second body jointly display the picture.

[0086] For example, the first body and the second body can be bent through multiple intermediate bodies, so that the first body is adjacent to the second body, and the display screens of the first body, the second body and the multiple intermediate bodies jointly display the same video, image, etc.

[0087] The topological relationship between the first body and the second body can be measured by using the hinge angle sensor in the electronic device. For example, the electronic device includes multiple bodies (such as a three-fold mobile phone, a curved screen, etc.), and the topological relationship between the multiple bodies can be "Z"-shaped, wavy, scroll-shaped, etc.

[0088] According to the embodiments of the present disclosure, determining a target display screen from multiple display screens according to the positional relationship and audio direction between the first body and the second body can include: determining the walking direction of the user according to the audio direction; determining the target display screen matching the walking direction from multiple display screens according to the positional relationship.

[0089] In one embodiment, the user's voice is continuously collected within a specified time to obtain the audio direction. If it is determined according to the audio direction that the user is walking from east to west, the display screen can be dynamically switched according to the user's real-time position.

[0090] In one embodiment, according to the continuous change of the audio direction, it can be predicted that the user will walk to the target display screen (for example, the display screen facing west), so that the interactive information can be displayed on the target display screen facing west even when the user does not walk to the target display screen. Therefore, when the user sees the target display screen, the interactive information can also be seen.

[0091] According to an embodiment of the present disclosure, determining a target display screen from multiple display screens according to the positional relationship and audio direction between a first body and a second body may include: determining the walking speed of the user according to the time-domain characteristics of the audio; when the walking speed is greater than a preset speed threshold, determining a target display screen matching the walking speed according to the walking speed, and displaying the interactive information on the target display screen at a target display speed.

[0092] According to an embodiment of the present disclosure, the preset speed threshold may be determined according to the original display speed of the interactive information. The preset speed threshold may be 0, 2, 10, etc. When the walking speed is greater than the preset speed threshold, it may indicate that there is an obvious difference in the walking speed of the user relative to the original display speed of the interactive information. When the walking speed is less than or equal to the preset speed threshold, it may indicate that the difference in the walking speed of the user relative to the original display speed is relatively small.

[0093] In one embodiment, the topological relationship between multiple bodies may be in a scroll shape, and the display pictures between the display screens on adjacent bodies may be continuous. Based on determining the target display screen from multiple display screens according to the user's walking direction judged from the audio direction, feature extraction may be performed on the audio to obtain the time-domain characteristics of the footsteps. The walking speed of the user may be determined according to the time-domain characteristics of the footsteps.

[0094] For example, the preset speed threshold may be 0. When the walking speed of the user is 0, the target display screen is used to scroll and display the interactive information at the original display speed.

[0095] When the walking speed of the user is not zero, the target display speed of the interactive information in each display screen may be determined according to the walking speed of the user, so as to match the speed at which the user reads the interactive information with the walking speed of the user. For example, if the walking direction of the user is opposite to the display direction of the interactive information and the user walks fast, the original display speed may be reduced to obtain the target display speed, so that the user can continuously and comfortably view the interactive information. If the walking direction of the user is the same as the display moving direction of the interactive information, the original display speed may be adjusted to the walking speed to obtain the target display speed, and the interactive information is displayed on the target display screen at the target display speed. The display speed may also be adaptively adjusted according to the rotation direction of the user's eyes, so that the user can view comfortably.

[0096] According to an embodiment of the present disclosure, determining a target display screen from multiple display screens according to the positional relationship and audio direction between the first body and the second body may include: determining the orientation of each of the multiple display screens according to the relative direction relationship; when the audio direction is perpendicular to the orientation of each of the multiple display screens respectively, using an image acquisition device to identify the eye position of the user; determining a target display screen that matches the eye position according to the orientation of each of the multiple display screens.

[0097] For example, the operation instruction of the user's voice representing "turn on the camera" is as Figure 2 , and the electronic device detects that the user's voice comes from the negative direction of the z-axis. The orientation of display screen 132 can be the positive direction of the y-axis, the orientation of display screen 131 can be the negative direction of the y-axis, and the orientation of display screen 133 can be the positive direction of the x-axis. Therefore, the orientations of display screens 131, 132, and 133 are respectively perpendicular to the negative direction of the z-axis, and image acquisition devices can be installed on the first body and the second body respectively, and the image acquisition devices can be used to identify the eye position of the user.

[0098] If the user's eyes are located in a side area of the negative direction of the x-axis and the negative direction of the y-axis, and the orientation of display screen 131 is the negative direction of the x-axis, display screen 131 can be within the user's line of sight, so the target display screen can be display screen 131.

[0099] According to an embodiment of the present disclosure, the target display screen is one or more parts of the multiple display screens shown, and the interaction information includes first interaction information and second interaction information; the target display screen is used to display the first interaction information, and the first interaction information is used to instruct the user to go to other display screens to view the second interaction information, and the second interaction information is the interaction information corresponding to the user operation instruction represented by the audio.

[0100] In an embodiment, when the relative positional relationship between the first body and the second body can be 180 degrees, both the first body and the second body may have display screens, and the screens of the first body and the second body can jointly display a complete landscape painting. And when the user is not on the side of the screen presenting the "landscape painting", the target display screen that the user can see can be determined according to the audio direction. The first interaction information displayed on the target display screen can instruct the user to go to other screens to view the second interaction information, and the second interaction information can be the interaction information corresponding to the operation instruction, such as viewing the landscape painting on other screens. Therefore, a more suitable display screen can be selected according to the specifications of the landscape painting to improve the user's viewing experience.

[0101] According to an embodiment of the present disclosure, a plurality of display screens may include a first display screen disposed on a first surface of a first body; determining a target display screen from the plurality of display screens according to the positional relationship between the first body and a second body and the audio direction may include: when the positional relationship indicates that a first included angle between the first body and the second body is less than a preset included angle threshold, determining the target display screen as the first display screen; when the positional relationship indicates that the first included angle is greater than or equal to the preset included angle threshold, determining the target display screen from the plurality of display screens according to the audio direction.

[0102] According to an embodiment of the present disclosure, the preset included angle threshold may be an included angle at which a user cannot see a display screen on a second surface of the first body and / or a display screen on a first surface of the second body. The preset included angle threshold may be 10 degrees, 30 degrees, 45 degrees, etc.

[0103] Figure 3 A schematic diagram of the first included angle according to an embodiment of the present disclosure is schematically shown.

[0104] As Figure 3 shown, the first body 112 is placed flat on a horizontal plane (xoy plane), and the first body 111 and the second body 112 are respectively projected onto the yoz plane. The first surface 1111 of the first body 111 may be a surface facing the positive z-axis direction with respect to the xoy plane. The first included angle α1 between the first body 111 and the second body 112 is less than the preset included angle threshold (for example, the included angle between the first body 111 and the second body 112 is less than 10 degrees, and the display screens on the second surface 1112 of the first body 111 and the first surface 1121 of the second body 112 cannot be seen by the user), and it is determined that the first display screen 1111a disposed on the first surface 1111 of the first body 111 is the target display screen, and the target display screen is used to display response information to the user.

[0105] According to an embodiment of the present disclosure, the preset included angle threshold may be determined according to the height of the user relative to the electronic device.

[0106] For example, when the height of the user relative to the electronic device is greater than a preset height threshold (such as when the user stands near the electronic device), the preset included angle threshold may be 45 degrees. The height of the user relative to the electronic device can be obtained by means such as an infrared sensor and a depth camera.

[0107] When the height of the user relative to the electronic device is less than the preset height threshold (such as when the user sits near the electronic device), the preset included angle threshold may be 30 degrees.

[0108] Determining the preset included angle threshold according to the height of the user relative to the electronic device improves the user experience.

[0109] According to an embodiment of the present disclosure, when the first included angle is greater than or equal to a preset included angle threshold, multiple display screens in the electronic device can all display response information to the user. To improve the user experience, a target display screen can be determined from the multiple display screens according to the audio direction, so that the display screen for displaying the response information is more in line with the user's line of sight.

[0110] According to an embodiment of the present disclosure, the multiple display screens may further include a second display screen disposed on the second surface of the first body; determining a target display screen from the multiple display screens according to the audio direction may include: when the first included angle is greater than or equal to a second included angle between the audio direction and the second body, determining the second display screen as the target display screen; when the first included angle is less than the second included angle, determining the first display screen as the target display screen.

[0111] Figure 4A A schematic diagram of the second included angle according to an embodiment of the present disclosure is schematically shown.

[0112] As Figure 4A shown, the first body 112 is placed flat on the horizontal plane (xoy plane), and the first body 111 and the second body 112 are respectively projected on the yoz plane. The first surface 1111 of the first body 111 may be the surface facing the positive z-axis direction relative to the xoy plane. The second surface 1112 of the first body 111 may be the surface facing the negative z-axis direction relative to the xoy plane. The second included angle α2 and the first included angle α1 are in the yoz plane.

[0113] When the first included angle α1 is greater than or equal to the second included angle α2 between the audio direction r and the second body 112, it is determined that the audio direction points to the side of the second surface 1112 of the first body 111, and the second display screen 1112a located on the second surface 1112 of the first body 111 is determined as the target display screen.

[0114] When the first included angle α1 is less than the second included angle α2, it is determined that the audio direction points to the side of the first surface 1111 of the first body 111, and the first display screen located on the first surface 1111 of the first body 111 is determined as the target display screen.

[0115] According to an embodiment of the present disclosure, the sound source direction is determined according to the first spatial position relationship between the user and the audio collection device. According to the sound source direction and the audio direction, an intermediate direction is determined; according to the second spatial position relationship between the audio collection device and the second body, the intermediate direction is converted to the same coordinate system as the first included angle to obtain a target direction; the included angle between the target direction and the second body is determined as the second included angle.

[0116] To control the balance between the sound source direction and the audio direction, the sound source direction and the audio direction can be weighted and averaged to obtain an intermediate direction r z, thereby improving the accuracy of the target direction.

[0117] Figure 4B Schematically shows a schematic diagram of the target direction according to an embodiment of the present disclosure.

[0118] As Figure 4B shown, the plane where the first included angle α1 is located is the yoz plane. The starting point of the first included angle α1 is the origin 0, and the coordinate system of the first included angle α1 is two-dimensional (y, z). The audio collection device 2 may be located near the electronic device. The intermediate direction r z can be a unit vector starting from the audio collection device, and the coordinate system of the intermediate direction r z is three-dimensional (x, y, z). Project the intermediate direction r z onto the yoz plane where the first included angle is located to obtain the intermediate projection direction r z’ , such that the intermediate projection direction r z’ has the same two-dimensional coordinate system (y, z) as the first included angle α1.

[0119] According to the spatial position relationship between the audio collection device 2 and the second body 112, translate the intermediate projection direction r z’ to the starting point of the first included angle (such as Figure 4B the origin 0), to obtain the target direction r'. Determine the included angle between the target direction r' and the second body as the second included angle

[0120] In the case where there are multiple users (such as Figure 4A ), what this embodiment obtains is the sound source position, such as the user making a sound. For example, the electronic device is a folding notebook, the sound source is located on one side of the first surface of the first body 111, that is, the first included angle α1 is less than the second included angle α2, and the first display screen is the target display screen. Even if a user is detected on the keyboard side of the folding notebook, the second display screen on the second surface of the first body 111 does not display the response information.

[0121] According to an embodiment of the present disclosure, by determining whether the first included angle is greater than or equal to the second included angle, the general direction of the audio direction can be determined, that is, the spatial area where the sound source is located. According to the spatial area where the sound source is located, determine the target display screen from the first display screen and the second display screen, thereby realizing personalized display screen control according to the user's perspective.

[0122] According to an embodiment of the present disclosure, the first included angle can be determined based on the following operations: determine the first included angle according to the first angle of the first body relative to the reference direction and the second angle of the second body relative to the reference direction; the second included angle can be determined based on the following operations: determine the second included angle according to the third angle of the audio direction relative to the reference direction and the second angle.

[0123] According to an embodiment of the present disclosure, the first body and the second body can be connected via a hinge. The first length direction of the first body and the second length direction of the second body can be perpendicular to the hinge length direction respectively.

[0124] According to an embodiment of the present disclosure, the reference direction can be located in the target plane, and the first length direction of the first body and the second length direction of the second body are also included in the target plane.

[0125] Figure 5 A schematic diagram of the reference direction according to an embodiment of the present disclosure is schematically shown.

[0126] As Figure 5 shown, the first body 111 and the second body 112 can be connected via a hinge 113. Both the first length direction and the second length direction are in the yoz plane, and the first length direction and the second length direction are perpendicular to the hinge length direction respectively. The reference direction is also located in the yoz plane, that is, the target plane can be the yoz plane.

[0127] According to the first angle Q1 between the first length direction and the reference direction and the second angle Q2 between the second length direction and the reference direction, determining the first included angle can include: determining the difference between the first angle Q1 and the second angle Q2 as the first included angle.

[0128] According to an embodiment of the present disclosure, according to the third angle and the second angle of the audio direction relative to the reference direction, determining the second included angle can include: determining the second included angle according to the third angle Q3 and the second angle Q2 between the target direction r' and the reference direction. For example, determining the difference between the third angle Q3 and the second angle Q2 as the second included angle.

[0129] According to an embodiment of the present disclosure, according to the first angle of the first body relative to the reference direction and the second angle of the second body relative to the reference direction, the first included angle can be determined; according to the third angle and the second angle of the audio direction relative to the reference direction, the second included angle can be determined. Therefore, both the first included angle and the second included angle can be determined based on the reference direction, so that the first included angle and the second included angle can be compared, and further the target display screen can be accurately determined from the first display screen and the second display screen, so that the display screen of the response information is more in line with the user's line of sight.

[0130] According to an embodiment of the present disclosure, a control can be provided on the outer surface of the housing, and the control can be used to start the target application and the audio acquisition device in response to a user's interaction operation. For example, the control can be provided on the first surface of the second body or any surface of the first body.

[0131] The electronic device can further include an input device, and the input device can be provided on the first surface of the second body ( Figure 3The first surface 1121 of the second body 112). The second display screen can be used to display the content obtained from the control instructions input via the input device when the first included angle is greater than or equal to the operation included angle threshold (the operation included angle threshold can be any value between 0 and 10 degrees). For example, if the electronic device is a laptop computer, the input device can be a keyboard. When the electronic device is in the open cover state (the first included angle is greater than or equal to the operation included angle threshold), the second display screen can be used to display the content obtained from the control instructions input via the input device.

[0132] When the first included angle is less than the operation included angle threshold (the electronic device is in the closed cover state), the input device is in a non-operating state. The control can be set on the first surface of the second body. When the electronic device is in the closed cover state, the target application can be directly started using the control, and the response information obtained by processing the semantic information using the target application can be displayed on the first display screen. Therefore, when the electronic device is in the closed cover state, the target application can also be started through the control, and the response information can be presented to the user using the first display screen, thereby improving the user experience.

[0133] Figure 6 Schematically shows a schematic diagram of a control according to an embodiment of the present disclosure.

[0134] As Figure 6 shown, the control can include an adhesive layer 601, a light-emitting button 602, an icon substrate 603, a light homogenizing sheet 604, a light-emitting diode 606, and a key switch 607.

[0135] The application icon of the target application can be set on the control. The adhesive layer 601 has the application icon. Below the adhesive layer 601 is the icon substrate 603. The icon substrate 603 can provide physical support for the display element, and the light-emitting button 602 can be provided on the icon substrate 603. For example, the light-emitting button 602 can be a part of the application icon. Such as the words and patterns in the icon.

[0136] Below the icon substrate 603 is the light homogenizing sheet 604. The light homogenizing sheet 604 can evenly distribute the light emitted by the light-emitting diode 606 installed on the circuit board 605. When the user uses the target application, different light-emitting effects are presented to improve the display effect of the application icon.

[0137] The key switch 607 can be installed on the main board 605. In response to the user's key operation on the control, it is connected to the processor of the electronic device through the main board 605 to start the target application and the audio acquisition device, thereby realizing one-key start of the target application. The processor can determine the target display screen from multiple display screens according to the positional relationship between the first body and the second body and the audio direction.

[0138] According to an embodiment of the present disclosure, the first display screen may be covered by a transparent cover plate, and the cover plate may include a first part and a second part. The second part may cover the display area of the first display screen, and the second part may be a protruding part extending outward from the first part; or a camera module and / or an element supporting the display of the first display screen may be embedded in the protruding part.

[0139] Figure 7 FIG. schematically shows a schematic diagram of a cover plate according to an embodiment of the present disclosure.

[0140] As Figure 7 shown, the cover plate may include a first part 701 and a second part 702. The first part 701 may cover the display screen to be a part of the display area of the first display screen.

[0141] The second part 702 may also cover the display area of the first display screen. The second part 702 may extend outward from the first part 701 to form a protruding part. The second part 702 may be located on the upper side, left side, right side, upper left corner or lower right corner of the first part to increase the display area of the first display screen.

[0142] A camera module and / or an element supporting the display of the first display screen may be embedded in the protruding part. The camera module may be an infrared camera module, a wide-angle camera module, a dual camera module, etc., which are not limited herein. The element supporting the display of the first display screen may include a liquid crystal display, a touch screen sensor, etc.

[0143] An audio collection device may also be embedded in the second part 702. For example, the audio collection device may be a microphone.

[0144] According to an embodiment of the present disclosure, the second part may extend outward from the first part to form a protruding part, and the second part may cover the display area of the first display screen, thereby increasing the display area of the display screen. At the same time, a camera module and / or an element supporting the display of the first display screen are embedded in the protruding part, so as to efficiently configure electronic components in the effective space.

[0145] Figure 8 FIG. schematically shows a flowchart of an interaction method according to an embodiment of the present disclosure.

[0146] As Figure 8 shown, the method includes operations S810 to S820.

[0147] In operation S810, according to the positional relationship between the first body and the second body in the electronic device and the audio direction, a target display area is determined from a plurality of display areas, and the plurality of display areas include display areas located on different surfaces of the electronic device.

[0148] In operation S820, response information is displayed in the target display area, and the response information is interaction information corresponding to the audio.

[0149] According to an embodiment of the present disclosure, the interaction information may include at least one of the following: application interaction information obtained by processing semantic information corresponding to the audio using a target application in the electronic device, and preset interaction information corresponding to the audio.

[0150] According to an embodiment of the present disclosure, the target application processes the semantic information corresponding to the audio through a target model to obtain application interaction information; the number of weight parameters of the target model is greater than 100 million, and / or the output result of the target model is obtained from characteristic information generated one by one by the target model.

[0151] According to an embodiment of the present disclosure, multiple display areas may be respectively disposed on different surfaces or the same surface of the outer surface of the housing, and the positional relationship may include at least one of the following: relative direction relationship, topological relationship; determining a target display area from multiple display areas according to the positional relationship between the first body and the second body and the audio direction may include: determining the walking direction of the user according to the audio direction, and determining a target display area matching the walking direction from multiple display areas according to the positional relationship; or determining the walking speed of the user according to the time-domain characteristics of the audio, and in the case where the walking speed is greater than a preset speed threshold, determining a target display area matching the walking speed, and displaying the interaction information in the target display area at a target display speed; or determining the orientations of the multiple display areas according to the relative direction relationship, and in the case where the audio direction is perpendicular to the respective orientations of the multiple display areas, identifying the eye position of the user using an image acquisition device, and determining a target display area matching the eye position according to the respective orientations of the multiple display areas.

[0152] According to an embodiment of the present disclosure, the target display area may be one or a part of multiple display areas, and the interaction information may include first interaction information and second interaction information; the target display area is used to display the first interaction information, and the first interaction information is used to instruct the user to go to other display areas to view the second interaction information, and the second interaction information is interaction information corresponding to the user operation instruction represented by the audio.

[0153] According to an embodiment of the present disclosure, the multiple display areas may include a first display area disposed on a first surface of the first body; determining a target display area from the multiple display areas according to the positional relationship between the first body and the second body in the electronic device and the audio direction may include: when the positional relationship indicates that a first included angle between the first body and the second body is less than a preset included angle threshold, determining the first display area as the target display area; when the positional relationship indicates that the first included angle is greater than or equal to the preset included angle threshold, determining the target display area from the multiple display areas according to the audio direction, and the preset included angle threshold is determined according to the height of the user relative to the electronic device.

[0154] According to an embodiment of the present disclosure, the multiple display areas may further include a second display area disposed on a second surface of the first body; determining a target display area from the multiple display areas according to the audio direction may include: when the first included angle is greater than or equal to a second included angle between the audio direction and the second body, determining the second display area as the target display area; when the first included angle is less than the second included angle, determining the first display area as the target display area.

[0155] According to an embodiment of the present disclosure, the above method may further include: determining the first included angle according to a first angle of the first body relative to a reference direction and a second angle of the second body relative to the reference direction; determining the second included angle according to a third angle of the audio direction relative to the reference direction and the second angle; or determining the sound source direction according to a first spatial position relationship between the user and the audio collection device, where the audio collection device is used to collect audio, determining an intermediate direction according to the sound source direction and the audio direction, and converting the intermediate direction to a coordinate system same as that of the first included angle according to a second spatial position relationship between the audio collection device and the second body to obtain a target direction, and determining an included angle between the target direction and the second body as the second included angle.

[0156] It should be noted that the interaction method is the same as the operation steps executed by the processor in the above electronic device, and will not be elaborated here.

[0157] Figure 9 A block diagram of an interaction device according to an embodiment of the present disclosure is schematically shown.

[0158] As Figure 9 shown, the information processing device 900 may include a first determination module 910 and a display module 920.

[0159] The first determination module 910 can be used to determine a target display area from multiple display areas according to the positional relationship between the first body and the second body in the electronic device and the audio direction, where the multiple display areas include display areas located on different surfaces of the electronic device. In an embodiment, the first determination module 910 can be used to perform the operation S810 described above, which will not be elaborated here.

[0160] The display module 920 can be used to display response information in the target display area, where the response information is interaction information corresponding to the audio. In an embodiment, the display module 920 can be used to perform the operation S820 described above, which will not be elaborated here.

[0161] According to an embodiment of the present disclosure, the interaction information may include at least one of the following: application interaction information obtained by processing semantic information corresponding to the audio using a target application in the electronic device, and preset interaction information corresponding to the audio.

[0162] According to an embodiment of the present disclosure, the target application can process the semantic information corresponding to the audio through a target model to obtain application interaction information; the number of weight parameters of the target model is greater than 100 million, and / or the output result of the target model is obtained from characteristic information generated one by one by the target model.

[0163] According to an embodiment of the present disclosure, the multiple display areas can be respectively arranged on different surfaces or the same surface of the outer surface of the housing, and the positional relationship may include at least one of the following: relative direction relationship, topological relationship. The first determination module may include a first determination sub-module and a second determination sub-module, or a third determination sub-module, a fourth determination sub-module, and a display sub-module, or a fifth determination sub-module, an identification sub-module, and a sixth determination sub-module. The first determination sub-module is used to determine the walking direction of the user according to the audio direction, and the second determination sub-module is used to determine a target display area that matches the walking direction from the multiple display areas according to the positional relationship; or the third determination sub-module is used to determine the walking speed of the user according to the time-domain characteristics of the audio, and the fourth determination sub-module is used to determine a target display area that matches the walking speed according to the walking speed when the walking speed is greater than a preset speed threshold, and the display sub-module is used to display the interaction information in the target display area at a target display speed; or the fifth determination sub-module is used to determine the orientation of each of the multiple display areas according to the relative direction relationship, the identification sub-module is used to identify the eye position of the user using an image acquisition device when the audio direction is perpendicular to the orientation of each of the multiple display areas respectively, and the sixth determination sub-module is used to determine a target display area that matches the eye position according to the orientation of each of the multiple display areas.

[0164] According to an embodiment of the present disclosure, the target display area is a part of one or more display areas among a plurality of display areas, and the interaction information includes first interaction information and second interaction information; the target display area is used to display the first interaction information, and the first interaction information is used to instruct the user to view the second interaction information in other display areas, and the second interaction information is interaction information corresponding to the user operation instruction represented by the audio.

[0165] According to an embodiment of the present disclosure, the plurality of display areas may include a first display area provided on a first surface of the first body; the first determination module 910 may include a seventh determination sub-module and an eighth determination sub-module. The seventh determination sub-module may be configured to determine the first display area as the target display area when the first included angle between the first body and the second body represented by the position relationship is less than a preset included angle threshold; the eighth determination sub-module may be configured to determine the target display area from the plurality of display areas according to the audio direction when the position relationship represents that the first included angle is greater than or equal to the preset included angle threshold.

[0166] According to an embodiment of the present disclosure, the plurality of display areas may further include a second display area provided on a second surface of the first body; the eighth determination sub-module may include a first determination unit and a second determination unit. The first determination unit may be configured to determine the second display area as the target display area when the first included angle is greater than or equal to the second included angle between the audio direction and the second body; the second determination unit may be configured to determine the first display area as the target display area when the first included angle is less than the second included angle. The preset included angle threshold is determined according to the height of the user relative to the electronic device.

[0167] According to an embodiment of the present disclosure, the above device may further include a second determination module, a third determination module, or a fourth determination module, a fifth determination module, a conversion module, and a sixth determination module. The second determination module may be configured to determine the first included angle according to the first angle of the first body relative to the reference direction and the second angle of the second body relative to the reference direction; the third determination module may be configured to determine the second included angle according to the third angle of the audio direction relative to the reference direction and the second angle, or the fourth determination module is configured to determine the sound source direction according to the first spatial position relationship between the user and the audio collection device, and the audio collection device is used to collect audio; the fifth determination module is used to determine the intermediate direction according to the sound source direction and the audio direction; the conversion module is used to convert the intermediate direction to the coordinate system same as the first included angle according to the second spatial position relationship between the audio collection device and the second body to obtain the target direction; the sixth determination module is used to determine the included angle between the target direction and the second body as the second included angle.

[0168] It can be understood that the first determination module 910 and the display module 920 can be implemented in one module, or any one of them can be split into multiple modules. Or, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present invention, at least one of the first determination module 910 and the display module 920 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented in any other reasonable way of integrating or packaging circuits, etc., in hardware or firmware, or implemented in an appropriate combination of software, hardware, and firmware. Or, at least one of the first determination module 910 and the display module 920 can be at least partially implemented as a computer program module, and when the program is run by a computer, it can execute the functions of the corresponding module.

[0169] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0170] Figure 10 Schematically shown is a schematic block diagram of an electronic device for performing an interaction method according to an embodiment of the present disclosure.

[0171] The electronic device for performing the interaction method is intended to represent various forms of digital computers, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0172] As Figure 10 shown, the electronic device 1000 for performing the interaction method includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1007 is also connected to the bus 1004.

[0173] Multiple components in the electronic device 1000 that executes the interaction method are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, an optical disc, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0174] The computing unit 1001 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 executes the various methods and processes described above, such as the virtual avatar driving method. For example, in some embodiments, the virtual avatar driving method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 1000 that executes the interaction method via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the virtual avatar driving method described above can be executed. Alternatively, in other embodiments, the computing unit 1001 can be configured to execute the object determination method by any other suitable means (e.g., by means of firmware).

[0175] The various embodiments of the systems and technologies described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip systems (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0176] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0177] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0178] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and the input received from the user can be in any form (including acoustic input, voice input, or tactile input).

[0179] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0180] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client - server relationship is created by computer programs that run on respective computers and have a client - server relationship with each other. Among them, the server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services (“Virtual Private Server”, or simply “VPS”). The server can also be a server of a distributed system, or a server combined with a blockchain.

[0181] Those skilled in the art can understand that the features recited in the various embodiments and / or claims of the present disclosure can be combined or / and combined in various ways, even if such combinations or combinations are not explicitly recited in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features recited in the various embodiments and / or claims of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0182] Although the present disclosure has been shown and described with reference to specific exemplary embodiments thereof, those skilled in the art should understand that various changes in form and detail can be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents. Therefore, the scope of the present disclosure should not be limited to the above - described embodiments, but should be determined not only by the appended claims but also by the equivalents of the appended claims.

Claims

1. An electronic device, comprising: a housing, the housing including a first body and a second body; a processor, configured to determine a target display screen from a plurality of display screens according to a positional relationship between the first body and the second body and an audio direction; the target display screen, configured to display a response message, the response message being interaction information corresponding to the audio.

2. The electronic device according to claim 1, wherein the interaction information includes at least one of the following: application interaction information obtained by processing semantic information corresponding to the audio by a target application in the electronic device, and preset interaction information corresponding to the audio.

3. The electronic device according to claim 2, wherein the target application can process the semantic information corresponding to the audio through a target model to obtain application interaction information; the number of weight parameters of the target model is greater than one hundred million, and / or the output result of the target model is obtained from characteristic information generated one by one by the target model.

4. The electronic device according to claim 1, wherein the plurality of display screens are respectively disposed on different surfaces or the same surface of the outer surface of the housing, and the positional relationship includes at least one of the following: relative direction relationship, topological relationship; determining the target display screen from a plurality of display screens according to the positional relationship between the first body and the second body and the audio direction includes: determining a walking direction of a user according to the audio direction, determining the target display screen matching the walking direction from the plurality of display screens according to the positional relationship; or determining a walking speed of the user according to a time domain feature of the audio, when the walking speed is greater than a preset speed threshold, determining the target display screen matching the walking speed, and displaying the interaction information on the target display screen at a target display speed; or determining orientations of the plurality of display screens according to the relative direction relationship, when the audio direction is perpendicular to the orientations of the plurality of display screens respectively, identifying an eye position of the user by using an image acquisition device, determining the target display screen matching the eye position according to the orientations of the plurality of display screens.

5. The electronic device according to claim 1, wherein the target display screen is one or a part of a plurality of display screens, and the interaction information includes first interaction information and second interaction information; the target display screen is configured to display the first interaction information, the first interaction information being used to instruct the user to view the second interaction information on other display screens, and the second interaction information being interaction information corresponding to a user operation instruction represented by the audio.

6. The electronic device according to claim 1, wherein the plurality of display screens include a first display screen disposed on a first surface of the first body; determining the target display screen from a plurality of display screens according to the positional relationship between the first body and the second body and the audio direction includes: when the positional relationship represents that a first included angle between the first body and the second body is less than a preset included angle threshold, determining the target display screen as the first display screen; When the positional relationship represents that the first included angle is greater than or equal to the preset included angle threshold, determine the target display screen from the plurality of display screens according to the audio direction, and the preset included angle threshold is determined according to the height of the user relative to the electronic device.

7. The electronic device according to claim 6, wherein the plurality of display screens include a second display screen disposed on a second surface of the first body; The determining the target display screen from the plurality of display screens according to the audio direction includes: When the first included angle is greater than or equal to a second included angle between the audio direction and the second body, determine the second display screen as the target display screen; When the first included angle is less than the second included angle, determine the first display screen as the target display screen.

8. The electronic device according to any one of claims 6 or 7, wherein the first included angle is determined based on the following operations: Determine the first included angle according to a first angle of the first body relative to a reference direction and a second angle of the second body relative to the reference direction; The second included angle is determined based on the following operations: Determine the second included angle according to a third angle of the audio direction relative to the reference direction and the second angle; or Determine the sound source direction according to a first spatial position relationship between the user and the audio collection device, where the audio collection device is used to collect the audio, Determine an intermediate direction according to the sound source direction and the audio direction, Convert the intermediate direction to the same coordinate system as the first included angle according to a second spatial position relationship between the audio collection device and the second body to obtain the target direction, Determine the included angle between the target direction and the second body as the second included angle.

9. The electronic device according to claim 6, wherein the first display screen is covered by a transparent cover plate, and the cover plate includes a first part and a second part, the second part covers the display area of the first display screen, and the second part extends outward from the first part to form a protruding part; or The protruding part is embedded with a camera module and / or elements supporting the display of the first display screen.

10. An interaction method, comprising: Determine a target display area from a plurality of display areas according to a positional relationship between a first body and a second body in an electronic device and an audio direction, where the plurality of display areas include display areas located on different surfaces of the electronic device; Display a response message in the target display area, where the response message is an interaction message corresponding to the audio.

Images