Display method and device and electronic equipment

Abstract

The embodiment of the invention provides a display method and device and electronic equipment, and relates to the technical field of terminals. The method comprises the steps that the electronic equipment can determine the posture of the electronic equipment according to sensor data, and when the posture of the electronic equipment is a first posture, a three-dimensional (3D) model is displayed in a first display state. The electronic equipment can also respond to the posture of the electronic equipment to be switched from the first posture to the second posture, and the 3D model is displayed in a second display state; wherein the display state comprises at least one of the posture, the display position and the display size of the 3D model. The playability of the user is effectively improved, the operation process of the user is reduced, and the user can control the 3D model in an immersive mode. And furthermore, the use experience of the user is improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on September 28, 2023, with application number 202311283015.9 and application name “3D model interaction method, device and electronic device”, all contents of which are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of terminal technology, and in particular to a display method, device and electronic device. Background Art

[0003] Three-dimensional (3D) models have cool presentation effects and can provide users with a full range of interactive experiences. Typical application scenarios of 3D models include 3D digital humans, 3D model products, 3D motion effects, and virtual reality (VR) / augmented reality (AR) experiences that rely heavily on 3D.

[0004] Usually, users hope to be able to conveniently operate 3D models on the touch screen of electronic devices. For example, users can rely on a series of gesture operations on the touch screen to control the 3D model. Specifically, a single finger moves on the touch screen to control the translation of the 3D model; or, two fingers move on the touch screen to control the zoom of the 3D model; or, a single finger moves on the touch screen to control the rotation of the 3D model, etc. Although the above operation method can achieve the purpose of operating the 3D model, it requires users to rely on gestures all the time, which is not only complicated to operate, but also lacks playability. In turn, the user's immersive experience is reduced. Summary of the invention

[0005] The embodiments of the present application provide a display method, device, and electronic device, which effectively improve the user's playability and reduce the user's operation process by adjusting the display state of a 3D model on an electronic device according to changes in the corresponding posture of the electronic device.

[0006] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, an embodiment of the present application provides a display method, which is applied to an electronic device, including: the electronic device can determine the posture of the electronic device according to sensor data, and when the posture of the electronic device is a first posture, display a three-dimensional 3D model in a first display state. The electronic device can also display the 3D model in a second display state in response to the posture of the electronic device switching from the first posture to the second posture; wherein the display state includes at least one of the posture, display position and display size of the 3D model.

[0008] In this way, the embodiment of the present application can determine the posture of the electronic device according to the sensor data, and adjust the display state of the 3D model on the electronic device according to the change of the corresponding posture of the electronic device. It effectively improves the playability of the user, reduces the user's operation process, and enables the user to immersively control the 3D model. In turn, it improves the user's experience.

[0009] In one achievable manner, the sensor data is collected by a first sensor and / or a second sensor, the first sensor is used to detect the movement of the electronic device, and the second sensor is used to detect the distance between the electronic device and the user.

[0010] In this way, the sensor data of the embodiment of the present application can be collected by the first sensor and / or the second sensor, the first sensor is used to detect the movement of the electronic device, and the second sensor is used to detect the distance between the electronic device and the user. In this way, the posture of the electronic device can be determined based on the movement of the electronic device and / or the distance between the electronic device and the user, so that the user can control the 3D model only by changing the posture of the mobile phone, without the user having to perform complex and tedious operations, which effectively improves the user's playability and enhances the user's immersive experience.

[0011] In one achievable manner, the method further includes: the electronic device may also acquire audio data, the audio data being used to adjust the display state of the 3D model, and displaying the 3D model in a third display state according to the audio data.

[0012] In this way, the embodiment of the present application can not only adjust the 3D model by a certain type of sensor data alone, but also adjust the 3D model by audio data input by the user. Of course, the 3D model can also be adjusted by combining the first sensor data, the second sensor data and the audio data at the same time to comprehensively represent the information of different aspects of the user, and further determine whether the 3D model needs to be adjusted and how to adjust the 3D model specifically. The flexibility and accuracy of controlling the 3D model are effectively improved, and the user experience is improved.

[0013] In one achievable manner, when the electronic device displays the 3D model in the third display state according to the audio data, the audio data can be input into the 3D language model to obtain the 3D model motion matrix corresponding to the audio data. The electronic device can also display the 3D model in the third display state according to the 3D model motion matrix.

[0014] Thus, in the embodiment of the present application, the 3D language model can be a model trained according to the language instruction samples and the 3D model motion matrix samples. In the process of adjusting the 3D model according to the audio data, the electronic device can input the audio data into the 3D language model, obtain the 3D model motion matrix, and control the 3D model according to the 3D model motion matrix. It provides specific methods for adjusting the 3D model according to the audio data, and improves the flexibility of adjusting the 3D model. It enables the user to directly input the audio data to adjust the 3D model, thereby improving the user experience.

[0015] In one achievable manner, when the electronic device displays the 3D model in the second display state, it can determine the rotation parameters of the electronic device when switching from the first posture to the second posture, and rotate the displayed 3D model according to the rotation parameters, wherein the rotation parameters include the rotation direction and / or the rotation angle.

[0016] In this way, the user can control the rotation of the 3D model by changing the posture of the mobile phone. The electronic device can obtain the specific parameters after the posture of the mobile phone is switched, and rotate the displayed 3D model according to the specific parameters. While experiencing the user's visual experience, the user's operation is reduced, the user's control threshold is lowered, and the user's immersive experience is improved.

[0017] In one feasible manner, in the process of determining the rotation parameters when the electronic device switches from a first posture to a second posture, the electronic device can determine the rotation direction when the electronic device switches from a first posture to a second posture based on sensor data and a first rule, and the first rule is used to specify the mapping relationship between the sensor data and the posture of the electronic device.

[0018] Thus, the embodiment of the present application provides a specific implementation method for determining the rotation direction of the electronic device according to the sensor data and the first rule, thereby improving the flexibility of determining the rotation direction of the electronic device and facilitating the subsequent adjustment of the displayed 3D model according to the rotation direction. Thus, the user's immersion and control flexibility are enhanced.

[0019] In an achievable manner, the method further includes: the electronic device can determine the target offset parameter according to the preset initial parameter, the preset threshold parameter and the sensor data currently collected in the sensor data, and calculate the rotation angle of the 3D model according to the target offset parameter. The electronic device can also determine the rotation direction of the 3D model according to the rotation direction of the electronic device and the second rule; wherein the second rule is used to specify the mapping relationship between the posture change direction of the electronic device and the rotation direction of the 3D model.

[0020] In this way, the electronic device can also determine the rotation direction of the 3D model according to the rotation direction of the electronic device and the second rule. The rotation angle of the 3D model can also be calculated according to the preset initial parameters, preset threshold parameters and sensor data. Not only can the rotation be performed based on the rotation direction, but also the corresponding rotation angle can be determined to accurately control the rotation of the 3D model. While ensuring the user's visual experience, the user operation is reduced, the user's control threshold is lowered, and the user's immersive experience is improved.

[0021] In one achievable manner, in the process of determining the target offset parameter according to the preset initial parameter, the preset threshold parameter and the currently collected sensor data in the sensor data, the electronic device may add the preset initial parameter and the preset threshold parameter to obtain a first value. When the currently collected sensor data is less than or equal to the first value, the currently collected sensor data and the product of the offset speed and the offset time in the sensor data are added to obtain a second value. The minimum value between the first value and the second value is determined as the target offset parameter.

[0022] When the currently collected sensor data is greater than the first value, the difference between the preset threshold parameter and the preset initial parameter is calculated to obtain a third value. The difference between the currently collected sensor data and the product of the offset speed and the offset time in the sensor data is calculated to obtain a fourth value. The minimum value between the third value and the fourth value is determined as the target offset parameter.

[0023] Thus, the embodiment of the present application provides a specific implementation method for determining the target offset parameter according to the preset initial parameter, the preset threshold parameter and the sensor data, which improves the flexibility and accuracy of the subsequent determination of the rotation angle, and facilitates the subsequent adjustment of the displayed 3D model according to the rotation angle. Thus, the user's immersion and control flexibility are enhanced.

[0024] In one achievable manner, the 3D model is presented based on the virtual camera perspective, and the electronic device can determine the rotation angle of the virtual camera according to the target offset parameter when calculating the rotation angle corresponding to the 3D model according to the target offset parameter. The electronic device can also determine the rotation transformation matrix of the virtual camera according to the rotation angle of the virtual camera.

[0025] In the process of displaying the 3D model according to the rotation parameters, the electronic device can control the virtual camera to perform a rotation action according to the rotation transformation matrix of the virtual camera.

[0026] In this way, in the process of controlling the 3D model, the embodiment of the present application can directly adjust the virtual camera to perform corresponding actions to further adjust the posture of the 3D model. As mentioned above, the rotation angle and rotation transformation matrix of the virtual camera can be determined according to the target offset parameter. The flexibility of adjusting the posture of the 3D model is improved, and the viewing angle adjustment of the virtual camera is also made more flexible and intelligent.

[0027] In one achievable manner, when the electronic device displays the 3D model in the second display state, the electronic device can determine based on the sensor data that the electronic device is close to the user and enlarge the display size of the 3D model. The electronic device can also determine based on the sensor data that the electronic device is far away from the user and reduce the display size of the 3D model.

[0028] Usually, users bring electronic devices close to their faces in order to see the images displayed on the screen more clearly. In this way, the electronic device can control the displayed 3D model to be enlarged when it determines that the mobile phone is close to the user based on the sensor data. Conversely, in order to increase the playability of the user, when the user moves the electronic device away from the face, the electronic device can control the displayed 3D model to be reduced when it determines that the electronic device is far away from the user based on the sensor data. In this way, the flexibility of the user to control the 3D model can be improved, and the playability and immersive experience of the user can be enhanced.

[0029] In an achievable manner, the method further includes: the electronic device may obtain a distance change value between the electronic device and the user, and when the distance change value is greater than a distance threshold, determine that the electronic device is close to the user. When the distance change value is less than or equal to the distance threshold, determine that the electronic device is far away from the user.

[0030] In this way, the embodiment of the present application can determine the distance change between the mobile phone and the user through the numerical relationship between the distance change value and the distance threshold, thereby adjusting the display posture of the 3D model. The user does not need to perform tedious operations, which effectively improves the user's playability and enables the user to immersively control the 3D model.

[0031] In one achievable manner, the 3D model is presented based on the perspective of a virtual camera. When the electronic device enlarges the display size of the 3D model, it can enlarge the perspective of the virtual camera corresponding to the 3D model. When the electronic device reduces the display size of the 3D model, it can reduce the perspective of the virtual camera corresponding to the 3D model.

[0032] In this way, in the process of controlling the 3D model, the embodiment of the present application can directly adjust the perspective of the virtual camera to further adjust the posture of the 3D model. As mentioned above, the 3D model can be controlled by adjusting the zoom in or zoom out of the perspective of the virtual camera. The flexibility of adjusting the posture of the 3D model is improved, and the perspective adjustment of the virtual camera is also made more flexible and intelligent.

[0033] In an achievable manner, the method further includes: the electronic device receiving an adjustment instruction from the linkage device, the adjustment instruction being used to adjust a display state of the 3D model, and displaying the 3D model in a fourth display state.

[0034] In this way, the embodiment of the present application can also adjust the display state of the 3D model in the mobile phone through the linkage device, further improving the flexibility of controlling the 3D model and increasing the device interactivity between the user and the linkage device and the electronic device, thereby improving the user's usage experience.

[0035] In a second aspect, an embodiment of the present application further provides a communication device, which includes an acquisition module and a display module.

[0036] The acquisition module is used to determine the posture of the communication device according to the sensor data.

[0037] The display module is used to display the three-dimensional 3D model in a first display state when the posture of the communication device is a first posture.

[0038] The display module is also used to display the 3D model in a second display state in response to the posture of the communication device switching from the first posture to the second posture; wherein the display state includes at least one of the posture, display position and display size of the 3D model.

[0039] In this way, the communication device provided in the embodiment of the present application can determine its own posture according to the sensor data, and adjust the display state of the 3D model according to the change of its own posture. This effectively improves the playability of the user, reduces the user's operation process, and enables the user to immersively control the 3D model. In turn, it improves the user's experience.

[0040] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, one or more processors, and a display screen; the memory is coupled to the processor; wherein the display screen is used to display a 3D model, and the memory stores computer program code, the computer program code comprising computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the display method as described in the first aspect.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer can execute the display method as described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the hardware structure of a mobile phone provided in an embodiment of the present application;

[0043] Figure 2A schematic diagram of the software structure of a mobile phone provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of a main screen interface provided in an embodiment of the present application;

[0045] Figure 4 A schematic diagram of an application interface of a target application provided in an embodiment of the present application;

[0046] Figure 5 A flow chart of a display method provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of a coordinate system provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of a display state switching scenario provided in an embodiment of the present application;

[0049] Figure 8 A scene diagram of a 3D model rotation process provided in an embodiment of the present application Figure 1 ;

[0050] Fig. 9 A scene diagram of a 3D model rotation process provided in an embodiment of the present application Figure 2 ;

[0051] Fig.10 A scene diagram of a 3D model rotation process provided in an embodiment of the present application Figure 3 ;

[0052] Fig.11 A scene diagram of a 3D model rotation process provided in an embodiment of the present application Figure 4 ;

[0053] Fig.12 A schematic diagram of a scene of a 3D model zooming in and out process provided in an embodiment of the present application;

[0054] Fig.13 A schematic diagram of the structure of a display system provided in an embodiment of the present application;

[0055] Fig.14 A schematic diagram of the structure of a communication device provided in an embodiment of the present application;

[0056] Fig.15 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Among them, in the description of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the present application is only a kind of association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. And, in the description of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects.

[0058] Those skilled in the art will appreciate that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in some embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0059] In addition, the network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are equally applicable to similar technical problems.

[0060] With the rapid development of three-dimensional (3D) models in electronic devices, users hope to be able to conveniently operate 3D models on the touch screen of electronic devices. However, currently users can only rely on a series of gesture operations on the touch screen of the electronic device to control the 3D model. In this way, users need to rely on gestures to click or slide on the touch screen all the time, which is not only complicated to operate, but also lacks playability. In turn, the user's immersive experience is reduced.

[0061] Based on the above, an embodiment of the present application provides a display method that can be applied in an electronic device, the method comprising: the electronic device can determine the posture of the electronic device according to sensor data, and when the posture of the electronic device is a first posture, display a three-dimensional 3D model in a first display state. The electronic device can also display the 3D model in a second display state in response to the posture of the electronic device switching from the first posture to the second posture; wherein the display state includes at least one of the posture, display position, and display size of the 3D model.

[0062] The embodiment of the present application can determine the posture of the electronic device according to the sensor data, and adjust the display state of the 3D model on the electronic device according to the change of the corresponding posture of the electronic device. The display state includes at least one of the posture, display position and display size of the 3D model. In this way, at least one of the posture, display position and display size of the 3D model can be adjusted according to the change of the corresponding posture of the electronic device.

[0063] In the related art, when a user performs gesture operation on a touch screen and controls the rotation of a 3D model, the user cannot immersively control the 3D model and needs to operate frequently. Thus, compared with the user performing gesture operation on a touch screen, the embodiment of the present application can control the 3D model according to the posture change of the electronic device, which effectively improves the user's playability, reduces the user's operation process, and enables the user to immersively control the 3D model. In turn, the user's experience is improved.

[0064] The electronic device provided in the embodiment of the present application may include at least one of a mobile phone, a smart watch, a tablet computer, a foldable electronic device, a desktop computer, a laptop computer, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR) device, a virtual reality (VR) device, an artificial intelligence (AI) device, a wearable device, and an in-vehicle device. The embodiment of the present application does not impose any special restrictions on the specific type of the electronic device.

[0065] The operating system installed in the electronic device provided in the embodiment of the present application includes but is not limited to Or other operating systems. This application does not limit the specific type of electronic device and the type of operating system when the operating system is installed.

[0066] For example, taking the electronic device as a mobile phone, Figure 1 A schematic structural diagram of the mobile phone 100 is shown.

[0067] The mobile phone 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. Among them, the sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, a sound sensor 180N, an image sensor 180O, etc.

[0068] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the mobile phone 100. In other embodiments of the present application, the mobile phone 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0069] The wireless communication function of the mobile phone 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0070] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the mobile phone 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0071] In some embodiments, the antenna 1 of the mobile phone 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the mobile phone 100 can communicate with the network and other devices through wireless communication technology.

[0072] In some embodiments of the present application, the display screen 194 is used to display 3D models, images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc. In some embodiments, the mobile phone 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0073] The mobile phone 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor.

[0074] The pressure sensor 180A is used to sense pressure signals and convert the pressure signals into electrical signals.

[0075] The gyroscope sensor 180B can be used to determine the motion posture of the mobile phone 100. In some embodiments, the angular velocity of the mobile phone 100 around three axes (i.e., x, y, and z axes) can be determined by the gyroscope sensor 180B. The gyroscope sensor 180B can be used for anti-shake shooting. Exemplarily, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shaking of the mobile phone 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the mobile phone 100 through reverse movement to achieve anti-shake. The gyroscope sensor 180B can also be used for navigation and somatosensory game scenes.

[0076] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the mobile phone 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation. The magnetic sensor 180D includes a Hall sensor. The mobile phone 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case.

[0077] The acceleration sensor 180E can detect the magnitude of the acceleration of the mobile phone 100 in various directions (generally three axes). When the mobile phone 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.

[0078] The distance sensor 180F is used to measure the distance. The mobile phone 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the mobile phone 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0079] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The mobile phone 100 emits infrared light outward through the light emitting diode. The mobile phone 100 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the mobile phone 100. When insufficient reflected light is detected, the mobile phone 100 can determine that there is no object near the mobile phone 100.

[0080] The ambient light sensor 180L is used to sense the ambient light brightness. The mobile phone 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The fingerprint sensor 180H is used to collect fingerprints. The mobile phone 100 can use the collected fingerprint characteristics to achieve fingerprint unlocking, access application locks, fingerprint photography, fingerprint answering calls, etc.

[0081] The temperature sensor 180J is used to detect temperature. The touch sensor 180K is also called a "touch panel". The touch sensor 180K can be set on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, also called a "touch screen". The touch sensor 180K is used to detect touch operations acting on or near it. The bone conduction sensor 180M can obtain vibration signals.

[0082] The sound sensor 180N is used to collect audio data. For example, the user can issue a voice command to the 3D model displayed in the mobile phone, and the sound sensor 180N is used to collect the voice data issued by the user.

[0083] The image sensor 180O usually measures the depth data of the measured object (or the measured target) by the time-of-flight method. Specifically, the time-of-flight method (TOF) measures the time interval T from the emission to the reception of the pulse signal actively emitted by the measuring instrument (often referred to as the pulse ranging method) or the phase difference (phase difference ranging method) generated by the laser traveling back and forth to the measured object once, and converts it into the distance of the photographed scene, which is used to generate depth data to achieve the measurement of the three-dimensional structure or three-dimensional contour of the measured object (or the detection area of ​​the measured object), and then obtain the grayscale image and depth data of the measured object. Similarly, the image sensor 180O converts the infrared light signal in the received reflected light into depth information, which exists in the form of an electrical signal, and then generates a grayscale image and depth data based on the depth information. In an embodiment of the present application, the image sensor 180O can be used to collect depth data between the user and the mobile phone 100.

[0084] In other embodiments, the image sensor 180O may also be disposed in one or more cameras 193. The embodiment of the present application does not specifically limit the arrangement of components.

[0085] The software system of the mobile phone 100 may adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present invention takes the Android system of the layered architecture as an example to exemplify the software structure of the mobile phone 100.

[0086] Figure 2 It is a software structure block diagram of the mobile phone 100 according to an embodiment of the present application.

[0087] The layered architecture divides the software into several layers, each with clear roles and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system library, and the kernel layer.

[0088] The application layer can include a series of application packages.

[0089] like Figure 2 As shown, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, target application, etc.

[0090] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0091] like Figure 2 As shown, the application framework layer may include a window manager, a content provider, a view system, a phone manager, a resource manager, a notification manager, and the like.

[0092] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc.

[0093] Content providers are used to store and retrieve data and make it accessible to applications. The data can include videos, images, audio, calls made and received, browsing history and bookmarks, phone books, etc.

[0094] The view system includes visual controls, such as controls for displaying text, controls for displaying images, etc. The view system can be used to build applications. A display interface can be composed of one or more views. For example, a display interface including a text notification icon can include a view for displaying text and a view for displaying images.

[0095] The phone manager is used to provide communication functions of the mobile phone 100, such as management of call status (including answering, hanging up, etc.).

[0096] The resource manager provides various resources for applications, such as localized strings, icons, images, layout files, video files, and so on.

[0097] The notification manager enables applications to display notification information in the status bar. It can be used to convey notification-type messages and can disappear automatically after a short stay without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or scroll bar text, such as notifications of applications running in the background, or a notification that appears on the screen in the form of a dialog window. For example, a text message is displayed in the status bar, a prompt sound is emitted, an electronic device vibrates, an indicator light flashes, etc.

[0098] Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system.

[0099] The core library consists of two parts: one part is the function that needs to be called by the Java language, and the other part is the Android core library.

[0100] The application layer and the application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0101] The system library may include multiple functional modules, such as a display module, an acquisition module, etc. In some examples, the display module is used to display the 3D model corresponding to the target application, and the acquisition module is used to acquire the posture of the mobile phone.

[0102] The kernel layer is the layer between hardware and software. The kernel layer contains at least display driver, camera driver, audio driver, and sensor driver.

[0103] For example, the above only uses a mobile phone as an example to illustrate the structure of the electronic device in the embodiment of the present application, but does not constitute a limitation on the structure and form of the electronic device. In some examples, the electronic device may include Figure 1 or Figure 2 More or fewer components are shown, or some components are combined, or some components are separated, or some components are replaced, or different component arrangements are made. The embodiments of the present application do not limit the structure and form of the electronic device.

[0104] The following will be combined with the accompanying drawings and the above description to describe the method provided by the embodiment of the present application. In the embodiment of the present application, each subject can perform some or all of the steps in the embodiment of the present application, and these steps or operations are only examples. The embodiment of the present application can also perform other operations or variations of various operations. In addition, each step can be performed in a different order presented in the embodiment of the present application, and it is possible not to perform all the operations in the embodiment of the present application.

[0105] The following will take a mobile phone as an example of an electronic device to illustrate the display scenario of a 3D model.

[0106] In some embodiments of the present application, a target application may be installed in a mobile phone, and the target application can present a 3D model based on a virtual camera perspective to a user. In response to the user's operation on the target application, the mobile phone displays the 3D model. In this way, the user can view the 3D model through the target application in the mobile phone.

[0107] For example, the target application may include a desktop application. The user may set the mobile phone desktop wallpaper as a dynamic wallpaper. The dynamic wallpaper may include some 3D virtual objects, such as 3D models. In this way, after the mobile phone is started, the desktop application may display the corresponding 3D model in response to the user's setting. Figure 3 As shown, the mobile phone can display a home screen interface 400 (or desktop), which includes one or more application icons, such as a shopping application icon, a setting application icon, etc., which are not described one by one here. The icons of different applications can be used to open the running interface of the corresponding application and implement the functions of the corresponding application. At the same time, the desktop can display a 3D model corresponding to the dynamic wallpaper, which is a 3D town image.

[0108] It should be noted that a 3D model refers to a virtual object with a three-dimensional display effect, and the virtual object includes but is not limited to people, objects, animals, plants, etc.

[0109] As another example, the target application may also include a shopping application. Figure 4 As shown in (A) of FIG. 4 , the user can click the icon corresponding to the shopping application in the main screen interface 400 to trigger the entry into the shopping page corresponding to the shopping application. In response to the user's click operation on the icon corresponding to the shopping application, the mobile phone displays the following Figure 4 The shopping page shown in (B) in FIG. The shopping page may include a search bar, product information, and function entrances. For example, the product information may include product A, product B, product C, product D, product E, and product F. And the user can click on the product icon corresponding to the product information to trigger the product details page corresponding to the product icon. The mobile phone can respond to the user's click operation on the product icon corresponding to product A and display the following information: Figure 4 In the product details page shown in (C), taking product A as headphones as an example, the product details page can not only include relevant information about the headphones, but also display the 3D model corresponding to product A (headphones).

[0110] As another example, the target application may also include 3D modeling applications, social applications, and life service applications (such as car service applications), etc. It should be noted that the embodiment of the present application does not specifically limit the application type of the target application installed above.

[0111] In the embodiment of the present application, when a user uses a target application in a mobile phone, the target application in the mobile phone can present an application interface including a 3D model to the user. The application interface can include one or more 3D models, and the 3D model can be presented in an initial display state. Figure 4 In (C), the earphones may be presented in an initial display state, such as the earphone cap in the earphone body is oriented away from the mobile phone display screen.

[0112] The display method proposed in the embodiment of the present application will be described in detail below with reference to the accompanying drawings. Figure 5 , the method may include:

[0113] S501. The mobile phone obtains its own posture.

[0114] In some embodiments of the present application, the mobile phone can adjust the displayed 3D model according to its own posture change. Specifically, the mobile phone can collect corresponding sensor data according to one or more sensors to determine its own posture and posture change. Thus, the posture of the presented 3D model can be adjusted according to the sensor data, so as to realize the experience of the user adjusting the posture of the 3D model presented in the mobile phone by moving the mobile phone.

[0115] In some examples, the mobile phone can obtain its own posture. Specifically, the mobile phone can obtain sensor data and determine its own posture based on the sensor data. The sensor data is used to characterize the posture of the mobile phone, and the sensor data is collected by a first sensor and / or a second sensor. The first sensor is used to detect the movement of the mobile phone, and the second sensor is used to detect the distance between the mobile phone and the user.

[0116] In one possible implementation, the sensor data includes first sensor data and / or second sensor data, the first sensor data is collected by a first sensor, and the first sensor may include at least one of a gyroscope sensor, an acceleration sensor, a direction sensor, a gravity sensor, and a magnetic sensor. Among them, the acceleration sensor can determine the translation direction and translation speed of the mobile phone by measuring the force of the component in a certain axis. Among them, the force condition is expressed in the form of direction (x-axis, y-axis, z-axis direction) and the magnitude of acceleration in the corresponding direction. The gyroscope sensor measures the angle between the vertical axis of the gyro rotor and the mobile phone in a three-dimensional coordinate system (the three-dimensional coordinate system includes the x-axis, y-axis, and z-axis), and calculates the angular velocity, and determines the motion posture of the mobile phone in three-dimensional space by the angle and angular velocity. The magnetic sensor measures the angle between the mobile phone and the four directions of southeast, northwest, and northeast. The direction sensor and gravity sensor can detect the direction of the device, and determine the direction of the mobile phone by detecting changes in physical quantities such as magnetic field, gravity, and light.

[0117] In some examples, after acquiring the first sensor data, the mobile phone can perform a low-pass filter on the first sensor data to eliminate the interference of high-frequency noise. The first sensor data includes at least one of acceleration data and angular velocity data. In addition, the acceleration data can be acceleration data of at least one of the three axes x, y, and z, or acceleration data after merging the acceleration data of the three axes x, y, and z. In addition, the angular velocity data can be three angular velocity data of yaw, pitch, and roll, or angular velocity data after merging the three angular velocity data of yaw, pitch, and roll.

[0118] The first sensor provided in the embodiment of the present application can be a sensor or a type of sensor. The first sensor data can be collected by one sensor or multiple sensors. Exemplarily, the first sensor data can be collected by a gyroscope sensor, the first sensor data can also be collected by a gyroscope sensor and a direction sensor, and the first sensor data can be collected by a gyroscope sensor and a gravity sensor. The embodiment of the present application does not specifically limit the type and quantity of the first sensor and the implementation method of the first sensor data.

[0119] The acquisition of the first sensor data is a continuously executed step. Usually, the first sensor can continuously collect sensor data. Exemplarily, the first sensor can collect the first sensor data at preset time intervals (preset time), such as 10ms, 100ms, 1s, 2s, 10s, 20s, 50s, 60s, 100s, etc. In this way, the updated first sensor data can be continuously acquired to continuously detect the movement changes of the mobile phone through the changes in the data in the first sensor data.

[0120] In some embodiments of the present application, the second sensing data can be collected by a second sensor, and the second sensor can include at least one of a distance sensor and an image sensor (such as a TOF sensor). The distance sensor can include an ultrasonic distance sensor, a laser distance sensor, and an infrared distance sensor. Exemplarily, the infrared sensor can emit an infrared light beam at a certain angle, and the light beam will be reflected back after encountering an object. After the reflected light beam is detected, the depth data is calculated through the geometric relationship of the structure. Similarly, the TOF sensor can obtain a target image and determine the depth data based on the target image.

[0121] In some examples, the mobile phone can also extract the target area in the target image, including the face area, and divide the target area into multiple sub-areas, and calculate the depth value of each sub-area respectively. Then, the depth data, i.e., the second sensing data, is obtained by calculating the average value of the target depth values ​​corresponding to the multiple sub-areas. Then, the mobile phone can sense the change in the distance between the mobile phone and the user by calculating at least two sets of depth data in the second sensing data. For example, the depth data collected by the second sensor at the current moment is 0.2 meters, and the depth data collected at the historical moment (10s before the current moment) is 0.3 meters, then the distance change between the mobile phone and the user is 0.1 meters.

[0122] It should be noted that the target region refers to an image region selected from the target image, which is the focus of image analysis. The target region can be a rectangular region, a circular region or a region of any shape.

[0123] The second sensor provided in the embodiment of the present application can be a sensor or a type of sensor. The second sensing data can be collected by one sensor or multiple sensors. Exemplarily, the second sensing data can be collected by an infrared sensor, or by an image sensor, or by a combination of an infrared sensor and an image sensor. The embodiment of the present application does not specifically limit the type and quantity of the second sensor, or the implementation method of the second sensing data.

[0124] Similarly, the above-mentioned acquisition of the second sensor data is a continuously executed step, and the second sensor can also continuously collect sensor data at a preset time interval (preset time), so that updated second sensor data can be continuously acquired to continuously detect the change in the distance between the mobile phone and the user through the change in the data in the second sensor data.

[0125] That is, the first sensor data and the second sensor data may be a plurality of sets of raw data collected by the sensor at preset time intervals, or may be change data calculated from the plurality of sets of raw data. This embodiment of the present application does not specifically limit this.

[0126] The sensor data of the embodiment of the present application can be collected by the first sensor and / or the second sensor. In this way, the posture of the electronic device can be determined according to the movement of the electronic device and / or the distance between the electronic device and the user, so that the user can control the 3D model only by changing the posture of the mobile phone, without the user having to perform complex and tedious operations, which effectively improves the playability of the user and enhances the user's immersive experience.

[0127] S502: When the posture of the mobile phone is a first posture, display the 3D model in a first display state.

[0128] S503: In response to the posture of the mobile phone being switched from the first posture to the second posture, displaying the 3D model in a second display state.

[0129] In some embodiments of the present application, the mobile phone can determine its own posture according to the sensor data. At the same time, the mobile phone can also adjust the display state of the 3D model after determining that its own posture has changed according to the sensor data.

[0130] like Figure 6 As shown in (A), the embodiment of the present application can establish a mobile phone coordinate system. When the mobile phone is in a horizontal position, the center point of the mobile phone is the origin O, the direction from the origin O to the right is the positive direction of the x-axis, the x-axis is parallel to the upper and lower sides of the mobile phone screen, the vertical upward direction from the origin O is the positive direction of the y-axis, the y-axis is parallel to the left and right sides of the mobile phone screen, and the horizontal forward direction from the origin O to the mobile phone screen is the positive direction of the z-axis. Figure 6 As shown in (B) in the figure, in the user's visible dimension, the side located above the mobile phone when the mobile phone is in a horizontal state is called the upper part or upper side of the mobile phone, the side located below the mobile phone is called the lower part or lower side of the mobile phone, the side located on the left side of the mobile phone is called the left side of the mobile phone, and the side located on the right side of the mobile phone is called the right side of the mobile phone. The side where the mobile phone screen is located is called the front side, and the side opposite to the front side is the back side.

[0131] For example, the mobile phone can determine that its posture is the first posture according to the sensor data, and display the 3D model in the first display state. Taking the above initial display state as an example, the mobile phone can collect sensor data when it is placed horizontally, and determine that the current posture of the mobile phone is the initial posture according to the sensor data, and display the 3D model in the initial display state, such as Figure 4 (C) in.

[0132] For example, after the user changes the posture of the mobile phone, see Figure 7 In (A), the user turns the mobile phone to the right, and the mobile phone can obtain the sensor data at the corresponding moment after the deflection to the right, and determine that the posture of the mobile phone is the second posture according to the sensor data. In this way, in response to the posture of the mobile phone switching from the first posture to the second posture, see Figure 7 As shown in (B), the mobile phone can display the 3D model in the second display state. The second display state is the display state of the 3D model after being deflected to the right.

[0133] In some embodiments of the present application, the display state may include at least one of the posture, display position, and display size of the 3D model. After determining that the mobile phone has changed its posture, it may adjust at least one of the posture, display position, and display size of the 3D model.

[0134] In some embodiments of the present application, when the mobile phone displays the 3D model in the second display state, the rotation parameter of the mobile phone when switching from the first posture to the second posture can be determined; wherein the rotation parameter includes the rotation direction and / or the rotation angle. The mobile phone can also rotate the displayed 3D model according to the rotation parameter.

[0135] The mobile phone can determine the rotation direction of the mobile phone when switching from the first posture to the second posture based on the sensor data and the first rule, and the first rule is used to specify the mapping relationship between the sensor data and the posture of the mobile phone. In this way, the embodiment of the present application provides a specific implementation method for determining the rotation direction of the electronic device based on the sensor data and the first rule, which improves the flexibility of determining the rotation direction of the electronic device and facilitates the subsequent adjustment of the displayed 3D model according to the rotation direction. Thereby, the user's immersion and control flexibility are enhanced.

[0136] In some examples, the mobile phone can determine the rotation direction of the mobile phone based on the first sensor data and the first rule. The mobile phone can also determine the rotation direction of the 3D model based on the rotation direction of the mobile phone and the second rule. The second rule is used to specify the mapping relationship between the posture change direction of the mobile phone and the rotation direction of the 3D model.

[0137] In an achievable manner, the first rule may include establishing a spatial coordinate system with the center point of the mobile phone as the origin, and determining that the mobile phone has undergone a posture change and the rotation direction of the mobile phone is forward flipping when the positive direction of the z-axis is the orientation direction of the mobile phone screen, the three-dimensional coordinate z value in the first sensing data is greater than 0, and the change value of y is less than 0, or when the negative direction of the z-axis is the orientation direction of the mobile phone screen, the z value is less than 0, and the change value of y is greater than 0. Forward flipping means that the upper part of the mobile phone rotates toward the front of the mobile phone and the lower part rotates toward the rear of the mobile phone. For example, the mobile phone can rotate with the x-axis as the rotation axis, the upper part of the mobile phone rotates toward the negative direction of the y-axis and the positive direction of the z-axis, and the lower part of the mobile phone rotates toward the negative direction of the z-axis and the positive direction of the y-axis.

[0138] The second rule may include that the direction of the posture change of the mobile phone is the same as the rotation direction of the 3D model. In this way, the mobile phone can control the rotation direction of the 3D model to flip forward according to the rotation direction of the mobile phone being flip forward. Similarly, the flip forward of the 3D model means that the upper part of the 3D model rotates toward the front of the 3D model and the lower part rotates toward the rear of the 3D model.

[0139] For example, see Figure 8 In (A), the first coordinate system is established with the mobile phone placed horizontally. The center point of the mobile phone is the origin of the coordinate system (0, 0, -1). The screen of the mobile phone is facing vertically upward, and the positive direction of the z-axis is the screen direction. It should be noted that the mobile phone has a certain thickness, and the coordinates of the center point of the mobile phone are only illustrative examples. Of course, the coordinate system can be established with any position in the mobile phone as the origin of the coordinate system.

[0140] For example, taking the coordinates (0, 5, -1) of the top end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensor data are (0, 4, 3), and the three-dimensional coordinates collected at the historical moment (10 seconds before the current moment) are (0, 5, -1). In this way, compared with the three-dimensional coordinates at the historical moment, the three-dimensional coordinate z value (value 3) at the current moment is greater than 0 and the change value of y is less than 0 (from value 4 to value 5, the change value is -1), which can indicate that the user turned the mobile phone forward.

[0141] In this way, the 3D model displayed in the mobile phone corresponds to a second coordinate system, which is a spatial coordinate system corresponding to the 3D model, and the direction of the second coordinate system is consistent with the first coordinate system. When the mobile phone detects that the user flips the mobile phone forward, the 3D model can be controlled to flip forward.

[0142] For further example, see Figure 8 In (B), the screen of the mobile phone is facing vertically downward, and the negative direction of the z-axis is the screen direction. For example, taking the coordinates (0, -5, -1) of the top end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (0, -4, -3), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (0, -5, -1). In this way, compared with the three-dimensional coordinates at the historical moment, the three-dimensional coordinate z value (value -3) at the current moment is less than 0 and the change value of y is greater than 0 (from value -4 to value -5, the change value is 1), which can indicate that the user flipped the phone forward. Thus, the mobile phone can control the 3D model to flip forward.

[0143] The first rule may also include determining that the mobile phone has changed its posture and the rotation direction of the mobile phone is backward when the positive direction of the z-axis is the orientation direction of the mobile phone screen and the z value is greater than 0 and the y change value is greater than 0, or when the negative direction of the z-axis is the orientation direction of the mobile phone screen and the three-dimensional coordinate z value in the first sensing data is less than 0 and the y change value is less than 0. Wherein, backward means that the lower part of the mobile phone rotates toward the front of the mobile phone and the upper part rotates toward the rear of the mobile phone. For example, the mobile phone can rotate with the x-axis as the rotation axis, the lower part of the mobile phone rotates toward the positive direction of the y-axis and the positive direction of the z-axis, and the upper part of the mobile phone rotates toward the negative direction of the z-axis and the negative direction of the y-axis.

[0144] For example, see Fig. 9In (A), the screen of the mobile phone is facing vertically upward, and the positive direction of the z-axis is the screen direction. For example, taking the coordinates (0, -5, -1) of the bottom end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (0, -4, 3), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (0, -5, -1). In this way, when the three-dimensional coordinate z value of the current moment is greater than 0 and the change value of y is greater than 0 compared to the three-dimensional coordinates at the historical moment, it can indicate that the user flipped the phone over. Similarly, the mobile phone can control the 3D model to flip over.

[0145] For further example, see Fig. 9 In (B), the screen of the mobile phone is facing vertically downward, and the negative direction of the z-axis is the screen direction. For example, taking the (0, 5, -1) coordinates of the bottom end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (0, 3, -5), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (0, 5, -1). In this way, compared with the three-dimensional coordinates at the historical moment, the z value of the three-dimensional coordinates at the current moment is less than 0 and the change value of y is less than 0, which can indicate that the user flipped the phone over. Similarly, the mobile phone can control the 3D model to flip over.

[0146] The first rule may also include determining that the mobile phone has undergone a posture change and the rotation direction of the mobile phone is left flipping when the positive direction of the z axis is the orientation direction of the mobile phone screen and the z value is greater than 0 and the x change value is greater than 0, or when the negative direction of the z axis is the orientation direction of the mobile phone screen and the three-dimensional coordinate z value in the first sensing data is less than 0 and the x change value is less than 0, wherein the left flipping means that the mobile phone rotates counterclockwise on the plane where the mobile phone screen is located, for example, the mobile phone can rotate counterclockwise on the xy plane with the origin O as the center.

[0147] For example, see Fig.10 In (A), the screen of the mobile phone is facing vertically upward, and the positive direction of the z-axis is the screen direction. For example, taking the (-5, 0, -1) coordinates of the left end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (-3, 0, 3), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (-5, 0, -1). In this way, when the three-dimensional coordinate z value of the current moment is greater than 0 and the change value of x is greater than 0 compared to the three-dimensional coordinates at the historical moment, it can indicate that the user turned the phone to the left. Similarly, the mobile phone can control the 3D model to turn left.

[0148] For further example, see Fig.10In (B), the screen of the mobile phone is facing vertically downward, and the negative direction of the z-axis is the screen direction. For example, taking the (-5, 0, -1) coordinates of the left end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (-5, 0, -1), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (-3, 0, 0). In this way, when the three-dimensional coordinate z value of the current moment is less than 0 and the change value of x is less than 0 compared to the three-dimensional coordinates at the historical moment, it can indicate that the user turned the phone to the left. Similarly, the mobile phone can control 3D left flipping.

[0149] The first rule may also include determining that the mobile phone has changed its posture and the rotation direction of the mobile phone is right flipping when the positive direction of the z-axis is the orientation direction of the mobile phone screen, the z value is greater than 0 and the change value of x is less than 0, or when the negative direction of the z-axis is the orientation direction of the mobile phone screen, the three-dimensional coordinate z value in the first sensing data is less than 0 and the change value of x is greater than 0. Right flipping means that the mobile phone rotates clockwise on the plane where the mobile phone screen is located. For example, the mobile phone can rotate clockwise on the xy plane with the origin O as the center.

[0150] For example, see Fig.11 In (A), the screen of the mobile phone is facing vertically upward, and the positive direction of the z-axis is the screen direction. For example, taking the coordinates (5, 0, -1) of the right end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (3, 0, 3), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (5, 0, -1). In this way, when the three-dimensional coordinate z value of the current moment is greater than 0 and the change value of x is less than 0 compared to the three-dimensional coordinates at the historical moment, it can indicate that the user turned the phone right. Similarly, the mobile phone can control the 3D model to turn right.

[0151] For further example, see Fig.11 In (B), the screen of the mobile phone is facing vertically downward, and the negative direction of the z-axis is the screen direction. For example, taking the coordinates (5, 0, -1) of the right end face of the mobile phone as an example, the three-dimensional coordinates collected at the current moment in the first sensing data are (3, 0, -3), and the three-dimensional coordinates collected at the historical moment (10s before the current moment) are (5, 0, -1). In this way, when the three-dimensional coordinate z value of the current moment is less than 0 and the change value of x is greater than 0 compared to the three-dimensional coordinates at the historical moment, it can indicate that the user turned the phone right. Similarly, the mobile phone can control 3D right flipping.

[0152] In some embodiments of the present application, the user can change the posture of the mobile phone by shaking the mobile phone so that the mobile phone can change the display state of the 3D model according to the first sensor data and the first rule and the second rule. In the process of controlling the 3D model to rotate in a preset direction, the mobile phone can control the 3D model to rotate according to a preset rotation angle. Among them, the preset angle may include 30°, 60°, 90°, 180°, 360°, etc. Of course, the mobile phone can also control the 3D model to rotate according to a preset rotation angle and a preset number of rotations. Among them, the preset number of rotations may include 1 time, 2 times, 3 times, 3 times, 5 times, etc. The embodiments of the present application do not specifically limit the numerical values ​​corresponding to the preset rotation angle and the preset number of rotations.

[0153] In this way, users can control the rotation of the 3D model by changing the posture of the phone, which reduces user operations while ensuring the user's visual experience, lowers the user's control threshold, and improves the user's immersive experience.

[0154] In some embodiments of the present application, when controlling the 3D model, the mobile phone can not only rotate based on the above-mentioned rotation direction, but also determine the corresponding rotation angle to accurately control the rotation of the 3D model.

[0155] In one possible implementation, the mobile phone can also calculate the rotation angle of the 3D model based on the first sensor data. Specifically, the mobile phone can determine the target offset parameter based on the preset initial parameter, the preset threshold parameter and the sensor data currently collected in the sensor data. The mobile phone can also calculate the rotation angle of the 3D model based on the target offset parameter.

[0156] Specifically, the mobile phone can determine the target offset parameter according to the preset initial parameter, the preset threshold parameter, and the sensor data currently collected in the first sensor data. Then, the mobile phone can calculate the rotation angle corresponding to the 3D model according to the target offset parameter, and control the 3D model to rotate according to the rotation angle and rotation direction of the 3D model.

[0157] The preset initial parameters may include preset initial angles such as 0°, 2°, 5°, 10°, 15°, etc. The preset threshold parameters may include preset maximum angles such as 20°, 22°, 25°, 30°, 35°, etc. The embodiments of the present application do not specifically limit the values ​​corresponding to the preset initial parameters and the preset threshold parameters.

[0158] Specifically, in the process of determining the target offset parameter, the mobile phone can add the preset initial parameter and the preset threshold parameter to obtain a first value. When the currently collected sensor data is less than or equal to the first value, the currently collected sensor data is added to the product of the offset speed and the offset time in the first sensor data to obtain a second value. The mobile phone determines the minimum value of the first value and the second data as the target offset parameter. It should be noted that the currently collected sensor data can be used to characterize the data collected at the latest moment, for example, the three-dimensional coordinate value collected at the latest moment.

[0159] When the currently collected sensor data is greater than the first value, the difference between the preset threshold parameter and the preset initial parameter is calculated to obtain a third value. At the same time, the mobile phone can calculate the difference between the currently collected sensor data and the product of the offset speed and the offset time in the first sensor data to obtain a fourth value. The mobile phone determines the minimum value of the third value and the fourth value as the target offset parameter.

[0160] In some examples, when the currently collected sensor data is less than or equal to the first value, the mobile phone can determine the target offset parameter according to a first formula, where the first formula includes:

[0161] curVal1=min(curVal0+speed×time, defaultVal+maxVal); Formula (1)

[0162] Among them, curVal1 is the target offset parameter, curVal0 is the currently collected sensor data, speed is the offset speed, time is the offset time, defaultVal is the preset initial parameter, and maxVal is the preset threshold parameter.

[0163] When the currently collected sensor data is greater than the first value, the mobile phone can determine the target offset parameter according to the second formula; the second formula includes:

[0164] curVal1=min(curVal0-speed×time, defaultVal-maxVal); Formula (2)

[0165] Among them, curVal1 is the target offset parameter, curVal0 is the currently collected sensor data, speed is the offset speed, time is the offset time, defaultVal is the preset initial parameter, and maxVal is the preset threshold parameter.

[0166] Thus, the embodiment of the present application provides a specific implementation method for determining the target offset parameter according to the preset initial parameter, the preset threshold parameter and the sensor data, which improves the flexibility and accuracy of determining the rotation angle, and facilitates the subsequent adjustment of the displayed 3D model according to the rotation angle. Thus, the user's immersion and control flexibility are enhanced.

[0167] In some embodiments of the present application, the currently collected sensor data may include the data collected at the latest moment and the data collected at the historical moment in the first sensor data. Specifically, the mobile phone can calculate the average value between the data collected at the latest moment and the data collected at the historical moment to obtain the currently collected sensor data. Exemplarily, the average value between the three-dimensional coordinate value collected at the latest moment and the three-dimensional coordinate value collected at the historical moment is calculated to obtain the currently collected sensor data. Among them, the three-dimensional coordinate value collected at the historical moment may include at least one three-dimensional coordinate value collected before the latest moment. For example, the three-dimensional coordinate value collected at the historical moment may include three-dimensional coordinate values ​​sampled twice before the latest moment. It should be noted that the embodiments of the present application do not limit the specific implementation method of the currently collected sensor data in the first sensor data.

[0168] In some examples, the target offset parameter obtained by the mobile phone is the rotation angle corresponding to the rotation of the mobile phone, and the rotation angle for the mobile phone needs to be converted into the rotation angle for the 3D model. For example, the mobile phone can calculate the rotation angle for the 3D model based on the target offset parameter, and then determine the rotation transformation matrix corresponding to the 3D model based on the rotation angle of the 3D model. In this way, the mobile phone can control the rotation of the 3D model based on the rotation transformation matrix and rotation direction of the 3D model.

[0169] In this way, the embodiment of the present application can also control the 3D model to rotate according to the corresponding rotation angle. When the user changes the posture of the mobile phone (such as shaking the mobile phone), the corresponding rotation angle during the shaking process can be determined, and the rotation control of the 3D model can be achieved more accurately. Thus, the user's immersive experience can be further improved.

[0170] In some embodiments of the present application, a 3D model can be presented based on a virtual camera perspective. The virtual camera is a very important component in a virtual scene, and is mainly used to show the user a scene image of the virtual scene where the 3D model controlled by the user is located. Two important functions of a virtual camera are perspective adjustment and field of view adjustment. Among them, perspective adjustment refers to adjusting the rotation and position of the virtual camera so that the user can view the virtual scene from different perspectives and directions. Field of view adjustment refers to adjusting the focal length and field of view angle of the virtual camera. Typically, a 3D model is provided with a virtual camera, and the 3D model can be further controlled by controlling the virtual camera to perform corresponding actions.

[0171] Specifically, when the mobile phone calculates the rotation angle corresponding to the 3D model according to the target offset parameter, the mobile phone can determine the rotation angle of the virtual camera according to the target offset parameter. Then, the mobile phone can determine the control parameters of the virtual camera corresponding to the 3D model according to the rotation angle of the virtual camera. In this way, the mobile phone can directly control the rotation of the virtual camera corresponding to the 3D model according to the rotation transformation matrix and rotation direction of the virtual camera, so as to control the rotation of the 3D model.

[0172] It should be noted that the embodiment of the present application does not specifically limit the implementation form of the rotation transformation matrix of the above-mentioned 3D model and the rotation transformation matrix of the virtual camera.

[0173] In this way, in the process of controlling the 3D model, the virtual camera can be directly adjusted to perform corresponding actions to further adjust the posture of the 3D model, thereby improving the flexibility of adjusting the posture of the 3D model and making the perspective adjustment of the virtual camera more flexible and intelligent.

[0174] In some embodiments of the present application, when the mobile phone displays the 3D model in the second display state, the display state of the 3D model can also be adjusted according to the second sensing data in the sensor data. Specifically, based on the second sensing data, it is determined that the mobile phone is close to the user, and the display size of the 3D model is enlarged. Based on the second sensing data, it is determined that the mobile phone is far away from the user, and the display size of the 3D model is reduced. Wherein, approaching refers to the mobile phone moving toward the front of the mobile phone, for example, the screen of the mobile phone moves in the positive direction of the z-axis. Moving away refers to the mobile phone moving toward the back of the mobile phone, for example, the screen of the mobile phone moves in the negative direction of the z-axis.

[0175] It should be noted that the above mobile phone postures are only used for exemplary descriptions. In actual applications, the above mobile phone postures can be limited and distinguished in the mobile phone through parameters such as angular velocity and acceleration, and the embodiments of this application will not be elaborated on.

[0176] For example, see Fig.12 In (A), when the mobile phone determines that the mobile phone is close to the user according to the second sensor data, the mobile phone controls the 3D model to be enlarged, that is, the display size of the 3D model is enlarged. Then, the 3D model is switched from the first display state (such as the display state before enlargement) to the second display state (such as the display state after enlargement). Fig.12 In (B), when the mobile phone determines that the mobile phone is far away from the user according to the second sensor data, the mobile phone controls the 3D model to shrink and reduce the display size of the 3D model. Then, the 3D model is switched from the first display state (such as the display state before the shrinking) to the second display state (such as the display state after the shrinking).

[0177] In an embodiment of the present application, the second sensor data can characterize the distance between the user and the mobile phone. Usually, the user brings the mobile phone close to the face in order to see the picture displayed on the screen more clearly. In this way, when the mobile phone determines that the mobile phone is close to the user according to the second sensor data, it can control the displayed 3D model to be enlarged. On the contrary, in order to increase the playability of the user, when the user moves the mobile phone away from the face, the mobile phone determines that the mobile phone is away from the user according to the second sensor data, and the displayed 3D model can be controlled to be reduced. Thereby, the flexibility of the user to control the 3D model can be improved, and the playability and immersive experience of the user can be enhanced.

[0178] In some embodiments of the present application, in order to accurately determine whether the mobile phone is close to the user or far away from the user, the mobile phone can obtain the distance change value between the mobile phone and the user in the process of determining that the mobile phone is close to the user according to the second sensor data, such as determining the distance change value according to the second sensor data. When the distance change value is greater than the distance threshold, it is determined that the mobile phone is close to the user. When the distance change value is less than or equal to the distance threshold, it is determined that the mobile phone is far away from the user.

[0179] In some examples, the distance change value is determined according to the second sensor data, and when the distance change value is greater than the distance threshold, it is determined that the mobile phone is close to the user and the display size of the 3D model is enlarged. In the process of the mobile phone determining that the mobile phone is far away from the user according to the second sensor data, the mobile phone can determine that the mobile phone is far away from the user and reduce the display size of the 3D model when the distance change value is less than or equal to the distance threshold. The distance threshold may include 3cm, 5cm, 8cm, 10cm, 15cm, 20cm, etc. The embodiments of the present application do not limit the specific distance threshold.

[0180] For example, the distance threshold may be 5 cm. The depth data collected at the current moment in the second sensor data is 30 cm, and the depth data collected at the historical moment (10 seconds before the current moment) is 10 cm. It can be determined that the distance changes from 10 cm to 30 cm, and the distance change value is -20 cm. The distance change value is less than the distance threshold, and the display size of the 3D model is reduced.

[0181] As another example, the depth data collected at the current moment in the second sensor data is 10 cm, and the depth data collected at the historical moment (10 seconds before the current moment) is 20 cm. It can be determined that the distance changes from 20 cm to 10 cm, and the distance change value is 10 cm. The distance change value is greater than the distance threshold, and the display size of the 3D model is enlarged.

[0182] In this way, the embodiment of the present application can adjust the display posture of the 3D model by determining the change in the distance between the mobile phone and the user. The user does not need to perform tedious operations, which effectively improves the user's playability. Thus, the user can immersively control the 3D model.

[0183] In some embodiments of the present application, similarly, in the above-mentioned process of zooming in and out of the 3D model, when the distance change value is greater than the distance threshold, the viewing angle of the virtual camera corresponding to the 3D model can be directly enlarged to enlarge the display size of the 3D model. When the distance change value is less than or equal to the distance threshold, the viewing angle of the virtual camera corresponding to the 3D model is reduced to reduce the display size of the 3D model.

[0184] In this way, the embodiment of the present application can adjust the display state of the 3D model by comparing the distance change value and the distance threshold, thereby preventing the user from accidentally moving the phone and changing the display state of the 3D model, improving the accuracy of the user's adjustment of the 3D model, and thus improving the user's experience.

[0185] In some embodiments of the present application, the mobile phone can also adjust the display state of the 3D model through the received voice command of the user. In other words, the mobile phone can also adjust at least one of the posture, display position and display size of the 3D model through the voice command input by the user.

[0186] In one possible implementation, the sensor data may also include audio data. The mobile phone may obtain the audio data collected by the sound sensor, and the audio data is used to instruct the 3D model to perform a corresponding action. The mobile phone may adjust the posture of the 3D model displayed on the mobile phone according to the audio data, and display the 3D model in the third display state.

[0187] For example, the first display state may be the initial posture of the 3D model. The user inputs a voice command to the mobile phone (such as enlarging the 3D model), and the mobile phone detects the audio data corresponding to the voice command, controls the 3D model to be enlarged, and displays the 3D model in the third display state. Fig.12 As shown in (A) in FIG. 1 , the third display state may be a display state after the 3D model is enlarged.

[0188] In this way, users can also adjust the posture of the 3D model through voice interaction to adjust its display status in the mobile phone. This effectively improves the user's playability and flexibility in controlling the 3D model, and reduces the user's operation process. In turn, it enables users to immersively control the 3D model.

[0189] In some embodiments of the present application, when the mobile phone displays the 3D model in the third display state according to the audio data, the audio data can be input into the 3D language model to obtain the 3D model motion matrix corresponding to the audio data. The 3D language model is a model trained according to the language instruction samples and the 3D model motion matrix samples. The mobile phone can display the 3D model in the third display state according to the 3D model motion matrix.

[0190] Thus, the embodiment of the present application provides a specific method for adjusting the 3D model according to the audio data, thereby improving the flexibility of adjusting the 3D model, so that the user can directly input the audio data to adjust the 3D model, thereby improving the user's experience.

[0191] In some examples, the mobile phone can train the initial 3D language model according to the language instruction samples and the 3D model motion matrix samples to obtain the 3D language model. In other words, the 3D language model is a model trained according to the language instruction samples and the 3D model motion matrix samples. In this way, the mobile phone can input the audio data into the 3D language model, output the 3D model motion matrix corresponding to the audio data, and adjust the posture of the 3D model according to the 3D model motion matrix, and display the 3D model in the third display state.

[0192] Exemplarily, continuing with the above-mentioned voice command as an example, the user can input a voice command such as "enlarge the 3D model" to the mobile phone. The mobile phone receives the voice command input by the user and pre-processes the voice command. Specifically, the voice command data can be normalized in advance, and the language features in the voice command can be extracted. The mobile phone inputs the extracted language features into the 3D language model and outputs the 3D model motion matrix corresponding to the voice command. In this way, the mobile phone can control the enlargement of the 3D model according to the 3D model motion matrix corresponding to the voice command, and display the 3D model in the third display state.

[0193] It should be noted that preprocessing can include feature scaling, feature selection, feature conversion, etc. These preprocessing steps are intended to improve the training effect and prediction accuracy of the model. At the same time, during the model training process, appropriate loss functions and optimization algorithms can be selected, and parameters can be adjusted and optimized to improve the training effect and generalization ability of the model.

[0194] In some examples, when training the initial 3D language model, a large number of language instruction samples and 3D model motion matrix samples can be used as training samples for training. This allows the 3D language model to learn the ability to convert voice instructions into 3D model motion matrices. The above process of training the initial 3D language model and the subsequent process of using the 3D model can be implemented on the server side.

[0195] The mobile phone can send the acquired audio data to the server. After receiving the audio data, the server inputs the audio data into the 3D language model, outputs the 3D model motion matrix corresponding to the audio data, and sends the 3D model motion matrix to the mobile phone. The mobile phone adjusts the posture of the 3D model according to the 3D model motion matrix and displays it.

[0196] In some embodiments of the present application, the 3D language model may include convolutional neural networks (CNN), region proposal networks (RPN), regions with CNN features (RCNN), fast RCNN networks (Faster-RCNN), MobileV2 networks, and residual networks. Of course, the 3D language model may also include a combination of multiple networks to recognize the voice commands input by the user. It should be noted that the 3D language model provided in the embodiments of the present application may include one or more of the YOLO algorithm, the SSD algorithm, and the DenseBox algorithm. These algorithms all have the advantages of fast speed and high precision. The embodiments of the present application do not limit the specific form of the 3D language model.

[0197] In some embodiments of the present application, similarly, in the process of adjusting the 3D model according to the audio data, the virtual camera corresponding to the 3D model can be directly controlled according to the audio data to adjust the 3D model. Specifically, the mobile phone can obtain the motion matrix of the virtual camera corresponding to the 3D model according to the audio data, and adjust the viewing angle and / or field of view of the virtual camera according to the motion matrix of the virtual camera.

[0198] In some embodiments of the present application, the mobile phone can adjust the display state of the 3D model through at least one of the first sensor data, the second sensor data and the audio data. In other words, the mobile phone can not only adjust the 3D model through a certain sensor data alone, but also adjust the 3D model through the audio data input by the user. Of course, the 3D model can also be adjusted in combination with the first sensor data, the second sensor data and the audio data at the same time to comprehensively characterize the information of different aspects of the user, and further determine whether the 3D model needs to be adjusted and how to adjust the 3D model specifically. The flexibility and accuracy of controlling the 3D model are effectively improved, and the user experience is improved.

[0199] In one possible implementation, the mobile phone can also set the determination priorities corresponding to the first sensor data, the second sensor data, and the audio data. In this way, the mobile phone can process the first sensor data, the second sensor data, and the audio data according to different priorities. Exemplarily, the priority corresponding to the first sensor data and the second sensor data is set higher than that of the audio data. The mobile phone can first adjust the display state of the 3D model according to the first sensor data and the second sensor data, and then adjust it according to the audio data.

[0200] The present application also provides a display system. Fig.13A display system provided in an embodiment of the present application includes an electronic device 110 (e.g., a user's mobile phone) and at least one linkage device 111 (e.g., a user's tablet computer). The electronic device 110 can be linked with the at least one linkage device 111. For example, the at least one linkage device 111 can adjust the posture of a 3D model displayed in the electronic device 110.

[0201] In some embodiments, a communication connection may be established between the electronic device 110 and the linkage device 111. The communication connection may include a wired communication connection or a wireless communication connection. For example, the wireless communication connection may include a short-range wireless communication connection such as a Bluetooth communication connection, a near-field communication connection, and a wireless local area network communication connection.

[0202] In other embodiments, the display system provided by the embodiment of the present application may further include a server 112. A communication connection is established between the electronic device 110, the linkage device 111 and the server 103. The server 103 may be used to support the communication service between the electronic device 110 and the linkage device 111 to complete the communication between the devices. The server 112 may be implemented by a server cluster composed of multiple servers, or by a single server. It should be noted that the above-mentioned display system may not be provided with a server 112, and the embodiment of the present application does not specifically limit this.

[0203] Furthermore, the operating system installed in the linkage device 111 includes but is not limited to Or other operating systems. This application does not limit the specific type of linkage device 111 and the type of operating system when an operating system is installed.

[0204] The display system provided in the embodiment of the present application may also include other electronic devices in addition to the electronic device 110, the linkage device 111 and the server 112. The display system provided in the embodiment of the present application includes but is not limited to the interaction between three devices, and may also be an interaction between one device and multiple devices. Those skilled in the art can determine the type and quantity of the electronic devices according to actual needs, and these designs do not exceed the protection scope of the embodiments of the present application.

[0205] In some embodiments of the present application, continuing to take the electronic device as a mobile phone and the linkage device as a tablet computer as an example, when the mobile phone and the tablet computer establish a communication connection, if the mobile phone and the tablet computer are associated, they are mutually trusted devices, for example, the mobile phone and the tablet computer are logged in to the same account. For another example, the mobile phone and the tablet computer are located in the same communication network. For another example, the accounts logged in by the mobile phone and the tablet computer belong to the same group (such as the same family account). For another example, the mobile phone and the tablet computer have directly established a Wi-Fi P2P connection, etc. Then, the mobile phone and the tablet computer can automatically establish a communication connection in the form of a super terminal, and then interact with data. The super terminal method refers to the ability to uniformly manage, schedule and integrate the capabilities of various electronic devices according to business needs to provide services to the outside world, so that different electronic devices can achieve fast connection, mutual assistance in capabilities, and resource sharing.

[0206] In some examples, a user can control the 3D model displayed in a mobile phone by operating a tablet computer. Specifically, when a mobile phone is associated with a tablet computer, the tablet computer can collect sensor data, and the sensor data includes at least one of sensor data for characterizing the movement of a linkage device, sensor data for characterizing the distance between the linkage device and the user, and audio data. The tablet computer can determine an adjustment instruction based on the sensor data, and the adjustment instruction is used for the mobile phone to adjust the display state of the 3D model. The mobile phone receives the adjustment instruction for the 3D model sent by the tablet computer, and displays the 3D model in the fourth display state.

[0207] It should be noted that the embodiment of the present application does not specifically limit the first display state, the second display state, the third display state, and the fourth display state.

[0208] In one possible implementation, the adjustment instruction generated by the tablet computer may include adjustment parameters corresponding to the 3D model, such as a rotation direction and / or rotation angle, and for example, parameters corresponding to zooming in and / or zooming out. The adjustment instruction may also include adjustment parameters for a virtual camera corresponding to the 3D model, such as a rotation transformation matrix, etc. The embodiment of the present application does not limit the specific data included in the adjustment instruction.

[0209] In another possible implementation, the adjustment instruction generated by the tablet computer may include the collected sensor data. In this way, the tablet computer may send the sensor data directly to the mobile phone, and the mobile phone processes the sensor data to adjust the display state of the 3D model. It should be noted that the process of the mobile phone adjusting the 3D model according to the sensor data is similar to the content of the above embodiment, and will not be repeated here.

[0210] In this way, the user can also adjust the display status of the 3D model in the mobile phone by operating the linkage device, which further improves the flexibility of controlling the 3D model and increases the device interactivity between the user and the linkage device and the electronic device, thereby improving the user experience.

[0211] The present application also provides a communication device. Fig.14 The communication device includes an acquisition module 1401 and a display module 1402. The acquisition module 1401 is used to determine the posture of the communication device according to the sensor data. The display module 1402 is used to display the three-dimensional 3D model in a first display state when the posture of the communication device is a first posture. The display module 1402 is also used to display the 3D model in a second display state in response to the posture of the communication device switching from the first posture to the second posture; wherein the display state includes at least one of the posture, display position and display size of the 3D model. The communication device can perform some or all of the steps in the above-mentioned display method embodiment.

[0212] In some schemes, multiple embodiments of the present application may be combined, and the combined scheme may be implemented. Optionally, some operations in the process of each method embodiment may be optionally combined, and / or the order of some operations may be optionally changed. Furthermore, the execution order between the steps of each process is only exemplary and does not constitute a limitation on the execution order between the steps, and other execution orders may be used between the steps. It is not intended to indicate that the execution order is the only order in which these operations can be performed.

[0213] A person skilled in the art will be able to think of many ways to reorder the operations described in the embodiments of the present application. In addition, it should be noted that the process details involved in a certain embodiment of the present application are also applicable to other embodiments in a similar manner, or different embodiments can be used in combination.

[0214] In addition, some steps in the method embodiment may be equivalently replaced by other possible steps. Alternatively, some steps in the method embodiment may be optional and may be deleted in certain usage scenarios. Alternatively, other possible steps may be added to the method embodiment.

[0215] Furthermore, the various method embodiments may be implemented separately or in combination.

[0216] The present application also provides an electronic device, such as the mobile phone described above. Fig.15 As shown, the electronic device may include one or more processors 1510 , a memory 1520 , and a communication interface 1530 .

[0217] The memory 1520 and the communication interface 1530 are coupled to the processor 1510. For example, the memory 1520, the communication interface 1530 and the processor 1510 may be coupled together via a bus 1540.

[0218] The communication interface 1530 is used for data transmission with other devices. The memory 1520 stores computer program codes. The computer program codes include computer instructions. When the computer instructions are executed by the processor 1510, the electronic device executes the relevant method steps in the embodiments of the present application.

[0219] Processor 1510 may be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0220] The bus 1540 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus 1540 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0221] An embodiment of the present application also provides an electronic device, which includes a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the relevant method steps in the above method embodiment.

[0222] An embodiment of the present application also provides a communication device, which includes a memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the communication device executes the relevant method steps in the above method embodiment.

[0223] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program code is stored. When the processor executes the computer program code, the electronic device executes the relevant method steps in the method embodiment.

[0224] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, it enables the computer to execute the relevant method steps in the above method embodiment.

[0225] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0226] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0227] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0228] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0229] The electronic device, computer storage medium or computer program product provided in this application is used to execute the corresponding method provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding method provided above and will not be repeated here.

[0230] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0231] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0232] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, which may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0233] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0234] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0235] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application shall be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A display method, characterized in that: Used in electronic equipment, including: Determining the posture of the electronic device based on the sensor data; When the posture of the electronic device is a first posture, displaying a three-dimensional 3D model in a first display state; In response to the posture of the electronic device switching from the first posture to the second posture, the 3D model is displayed in a second display state; wherein the display state includes at least one of the posture, display position and display size of the 3D model.

2. The method according to claim 1, characterized in that The sensor data is collected by a first sensor and / or a second sensor, the first sensor is used to detect the movement of the electronic device, and the second sensor is used to detect the distance between the electronic device and the user.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Acquiring audio data, where the audio data is used to adjust the display state of the 3D model; The 3D model is displayed in a third display state according to the audio data.

4. The method according to claim 3, characterized in that The step of displaying the 3D model in a third display state according to the audio data comprises: Inputting the audio data into a 3D language model to obtain a 3D model motion matrix corresponding to the audio data; The 3D model is displayed in a third display state according to the 3D model motion matrix.

5. The method according to claim 2, characterized in that: The displaying of the 3D model in the second display state comprises: Determining a rotation parameter of the electronic device when switching from the first posture to the second posture; wherein the rotation parameter includes a rotation direction and / or a rotation angle; The 3D model is rotated and displayed according to the rotation parameters.

6. The method according to claim 5, characterized in that The determining a rotation parameter of the electronic device when switching from the first posture to the second posture includes: The rotation direction of the electronic device when switching from the first posture to the second posture is determined according to the sensor data and a first rule, wherein the first rule is used to define a mapping relationship between the sensor data and the posture of the electronic device.

7. The method according to claim 6, characterized in that The method further comprises: Determining a target offset parameter according to a preset initial parameter, a preset threshold parameter and the sensor data currently collected in the sensor data; Calculating the rotation angle of the 3D model according to the target offset parameter; The rotation direction of the 3D model is determined according to the rotation direction of the electronic device and a second rule; wherein the second rule is used to specify a mapping relationship between the posture change direction of the electronic device and the rotation direction of the 3D model.

8. The method according to claim 7, characterized in that The step of determining the target offset parameter according to the preset initial parameter, the preset threshold parameter and the currently collected sensor data in the sensor data includes: Adding the preset initial parameter and the preset threshold parameter to obtain a first value; When the currently collected sensor data is less than or equal to the first value, the currently collected sensor data is added to the product of the offset speed and the offset time in the sensor data to obtain a second value; determining a minimum value between the first value and the second value as the target offset parameter; When the currently collected sensor data is greater than the first value, calculating the difference between the preset threshold parameter and the preset initial parameter to obtain a third value; Calculating a difference between the currently collected sensor data and a product of a displacement speed and a displacement time in the sensor data to obtain a fourth value; A minimum value between the third value and the fourth value is determined as the target offset parameter.

9. The method according to claim 7 or 8, characterized in that: The 3D model is presented based on a virtual camera perspective, and the calculation of a rotation angle corresponding to the 3D model according to a target offset parameter includes: Determining a rotation angle of the virtual camera according to the target offset parameter; A rotation transformation matrix of the virtual camera is determined according to the rotation angle of the virtual camera.

10. The method according to claim 9, characterized in that The displaying of the 3D model according to the rotation parameter comprises: According to the rotation transformation matrix of the virtual camera, the virtual camera is controlled to perform a rotation action.

11. The method according to any one of claims 1 to 10, characterized in that: The displaying of the 3D model in the second display state comprises: Determining, based on the sensor data, that the electronic device is close to a user, and enlarging a display size of the 3D model; Based on the sensor data, it is determined that the electronic device is away from the user, and the display size of the 3D model is reduced.

12. The method according to claim 11, characterized in that The method further comprises: Obtaining a distance change value between the electronic device and the user; When the distance change value is greater than a distance threshold, determining that the electronic device is close to the user; When the distance change value is less than or equal to the distance threshold, it is determined that the electronic device is far away from the user.

13. The method according to claim 12, characterized in that The 3D model is presented based on a virtual camera perspective; The enlarging the display size of the 3D model includes: Enlarging the 3D model to correspond to the viewing angle of the virtual camera; The reducing the display size of the 3D model includes: The 3D model is reduced to correspond to the viewing angle of the virtual camera.

14. The method according to claim 1, characterized in that The method further comprises: receiving an adjustment instruction from a linkage device, wherein the adjustment instruction is used to adjust a display state of the 3D model; The 3D model is displayed in a fourth display state.

15. A communication device, characterized in that: The communication device includes an acquisition module and a display module; The acquisition module is used to determine the posture of the communication device according to the sensor data; The display module is used to display the three-dimensional 3D model in a first display state when the posture of the communication device is a first posture; The display module is further used to display the 3D model in a second display state in response to the posture of the communication device switching from the first posture to the second posture; wherein the display state includes at least one of the posture, display position and display size of the 3D model.

16. An electronic device, characterized in that: The electronic device includes a memory, one or more processors, and a display screen; the memory is coupled to the processor; wherein the display screen is used to display a 3D model, and the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the display method as described in any one of claims 1-14.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer can execute the display method according to any one of claims 1 to 14.

Images