3D object operation method, electronic device, and computer-readable storage medium

Abstract

Provided are a 3D object operation method, an electronic device, and a computer-readable storage medium. The method comprises: when a terminal stereoscopically displays screen content, determining the visual depth of the screen content as perceived by human eyes, the visual depth comprising a first depth of field located in front of a screen and a second depth of field located behind the screen (S202); detecting a first position of a user finger in front of the screen (S204); and when the first position is within a first distance range of the screen, operating a 3D object within the first depth of field, and when the first position is within a second distance range of the screen, operating a 3D object within the second depth of field (S206).

Description

3D object operation method, electronic device and computer-readable storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to a Chinese patent application filed with the China Patent Office on January 24, 2024, with application number 202410104598.2 and invention name “3D object operation method, electronic device and computer-readable storage medium”. The entire contents of the Chinese patent application are incorporated herein by reference. Technical Field

[0003] The present application relates to the field of terminal display technology, and in particular to a method for operating a 3D object, an electronic device, and a computer-readable storage medium. Background Art

[0004] With the development of stereoscopic display technology, more and more terminals can use stereoscopic display technology to stereoscopically display the content displayed on the terminal screen, so that users can produce 3D visual effects when viewing the screen content, thereby improving user experience.

[0005] The principle of a terminal using stereoscopic display technology to display screen content in stereo may be: by changing the display characteristics of the screen content, a certain parallax can be generated between the left eye and the right eye of the user when viewing the screen content, thereby giving the screen content a sense of depth and space, thereby achieving a 3D display effect. In related technologies, in order to further enhance the user experience, the user may be allowed to operate the 3D objects displayed in stereo, such as moving and rotating the 3D objects. However, in some application scenarios, when the terminal displays the screen content in stereo, the visual depth of the screen content to the user's human eye may include the front and back of the screen, that is, the 3D objects include the 3D objects located in front of the screen and the 3D objects located behind the screen. For the 3D objects in front of the screen, the user can directly operate them, but for the 3D objects behind the screen, since they are visually located behind the screen, the user cannot directly operate them.

[0006] Summary of the Invention

[0007] The present application provides a 3D object operation method, an electronic device, and a computer-readable storage medium, which are used to solve the problem that when a terminal displays screen content in 3D stereoscopic form, a user cannot operate a 3D object that is visually located behind the terminal screen.

[0008] In a first aspect, a method for operating a 3D object is provided, comprising: determining, when a terminal displays screen content in stereoscopic form, a visual depth of the screen content to a human eye, the visual depth comprising a first depth of field and a second depth of field, the first depth of field being located in front of the screen and the second depth of field being located behind the screen; detecting a first position of a user's finger in front of the screen; and operating a 3D object within the first depth of field when the first position is within a first distance range of the screen, and operating the 3D object within the second depth of field when the first position is within a second distance range of the screen.

[0009] In a second aspect, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method described in the first aspect.

[0010] According to a third aspect, a computer-readable storage medium is provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method according to the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG1 is a schematic diagram of a terminal performing stereoscopic display of screen content in the related art;

[0012] FIG2 is a flow chart of a method for operating a 3D object according to an embodiment of the present application;

[0013] FIG3 is a schematic diagram of determining a target position corresponding to a first position of a user's finger within a visual depth according to a position mapping relationship according to an embodiment of the present application;

[0014] FIG4 is a schematic diagram of an embodiment of the present application for operating a 3D object;

[0015] FIG5 is a schematic diagram of the operable range of a user's finger according to an embodiment of the present application;

[0016] FIG6 is a schematic diagram of providing visual feedback to a 3D object according to an embodiment of the present application;

[0017] FIG7 is a flowchart of a method for operating a 3D object according to another embodiment of the present application;

[0018] FIG8 is a schematic structural diagram of an electronic device according to an embodiment of the present application;

[0019] FIG9 is a schematic structural diagram of a 3D object operation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In some application scenarios, when a terminal displays screen content in stereoscopic mode, the user's visual depth of the screen content includes both the front and back of the screen. Accordingly, 3D objects also include both 3D objects located in front of the screen and 3D objects located behind the screen. As shown in Figure 1, when a terminal displays screen content in stereoscopic mode, the user's visual depth includes a visual depth BC located in front of the screen and a visual depth AB located behind the screen.

[0021] When the user operates a 3D object, the user can directly operate the 3D object in front of the screen, but the user cannot directly operate the 3D object behind the screen. In the related art, a method such as translation can be adopted to push the 3D object located behind the screen to the front of the screen, and then operate it. For example, when the user wants to click on the 3D object behind the screen, the user can first push the 3D object to the front of the screen, then perform a click operation on it, and finally push the 3D object back to the back of the screen. However, this operation method is relatively cumbersome and will also squeeze the original layout, which will cause great trouble to the user in actual use.

[0022] Embodiments of the present application provide a method for operating 3D objects, an electronic device, and a computer-readable storage medium. When a terminal displays screen content in stereoscopic form and the visual depth of the screen content to the human eye includes a first depth of field in front of the screen and a second depth of field behind the screen, the range in front of the terminal screen can be divided into a first distance range and a second distance range. When the position of the user's finger in front of the screen is within the first distance range, 3D objects within the first depth of field can be operated. When the position of the user's finger in front of the screen is within the second distance range, 3D objects within the second depth of field can be operated. In this way, by corresponding the first depth of field visually located in front of the screen and the second depth of field located behind the screen to the first distance range and the second distance range in front of the screen, respectively, operations on 3D objects in front of and behind the screen can be implemented.

[0023] In order to help those skilled in the art better understand the technical solutions of this application, the following will clearly and completely describe the technical solutions of this application in conjunction with the drawings of one or more embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0024] The terms "first," "second," and the like in this application and the claims are used to distinguish similar objects and are not used to describe a particular order or precedence. It should be understood that such terms are interchangeable where appropriate so that this application can be implemented in sequences other than those illustrated or described herein. In addition, the term "and / or" in this application and the claims refers to at least one of the connected objects, and the character " / " generally indicates that the connected objects are in an "or" relationship.

[0025] It should be noted that the technical solution provided in the embodiment of the present application can be applicable to naked-eye 3D scenes, and can also be applied to non-naked-eye 3D scenes, which is not limited here. The terminal in the embodiment of the present application can be a terminal with a stereoscopic display function. In some implementations, the terminal may include but is not limited to a 3D interface display device, a camera system device (single camera or dual camera), a touch detection system, a gyroscope sensor, a gravity acceleration sensor, a microprocessor, a baseband module, a wireless transceiver module, a power module, a terminal operating system, and a user operation instruction monitoring system. Among them, in the technical solution provided in the embodiment of the present application, the 3D interface display device can be used to present a 3D display effect. In the naked-eye 3D scene, the 3D display device can include a 3D naked-eye flat-panel display screen. The terminal operating system can be used to present a 3D interface on the 3D interface display device, switch and jump interfaces when receiving user operations, and perform data operations, etc., and can include an application layer, a framework layer, and a kernel layer. The user operation command monitoring system can be used to capture image information from a camera, analyze and detect user gestures and / or facial information, and determine the user's hand movements, finger positions, eye gaze, lip movements, facial expressions, and other expressions, thereby corresponding to preset interface operation commands. The camera device can be used to detect gesture dynamics and gesture position depth, that is, to determine the position and distance of fingers, palms, etc. from the camera. The instructions analyzed by the user operation command monitoring system can be transmitted to the terminal operating system, and the terminal performs interface operations based on the instructions.

[0026] When the terminal displays the screen content in stereoscopic form, the displayed 3D object may be a virtual image, etc., which is not limited here.

[0027] The following describes in detail the technical solutions provided by various embodiments of the present application in conjunction with the accompanying drawings.

[0028] Fig. 2 is a flow chart of a method for operating a 3D object according to an embodiment of the present application. The embodiment shown in Fig. 2 includes the following steps.

[0029] S202: When the terminal displays screen content in stereoscopic mode, determining a visual depth of the screen content for human eyes, where the visual depth includes a first depth of field and a second depth of field, where the first depth of field is located in front of the screen and the second depth of field is located behind the screen.

[0030] When the terminal receives a stereoscopic display instruction for screen content or needs to display screen content in stereo, it can use stereo display technology to display the screen content in stereo. The implementation method of the terminal using stereo display technology to display screen content in stereo can be found in the implementation of related technologies and will not be described in detail here.

[0031] After the terminal displays the screen content in stereoscopic form, it presents the screen content to the user with a 3D display effect. For the user, the screen content they see has a clear sense of depth, that is, the user's human eye will perceive the screen content with a certain visual depth. In this case, the terminal can determine the visual depth of the user's human eye to the screen content.

[0032] When determining the visual depth of the user's eyes relative to the screen content, the terminal can use stereoscopic display technology to determine the parallax between the user's left and right eyes when viewing the screen content, and determine the visual depth of the user's eyes relative to the screen content based on the parallax. Of course, other methods can also be used to determine the visual depth, and examples are not given here.

[0033] In the embodiment of the present application, the visual depth of the screen content as perceived by the user's eyes includes the visual depth in front of the terminal screen and the visual depth behind the terminal screen. For ease of distinction, the visual depth in front of the terminal screen can be represented as the first depth of field, and the visual depth behind the terminal screen can be represented as the second depth of field. After obtaining the visual depth of the screen content as perceived by the user's eyes, the terminal can further obtain the first depth of field and the second depth of field.

[0034] S204: Detecting a first position of the user's finger in front of the screen.

[0035] When the terminal displays screen content in stereo, the terminal can detect the position of the user's finger in front of the screen. For ease of distinction, the position of the user's finger in front of the screen can be represented as a first position. The first position can be the relative position of the user's finger relative to the terminal screen.

[0036] When detecting the position of the user's finger, the terminal may use the terminal's camera to capture an image of the area in front of the terminal screen, identify the user's finger based on the captured image, and after identifying the user's finger, calculate the distance between the user's finger and the terminal screen to determine the first position of the user's finger in front of the screen. Of course, other methods can also be used to determine the first position of the user's finger in front of the screen, which will not be described in detail here.

[0037] S206: operating the 3D object within the first depth of field when the first position is within a first distance range of the screen, and operating the 3D object within the second depth of field when the first position is within a second distance range of the screen.

[0038] The terminal can divide the distance range in front of the screen (the distance range perpendicular to the terminal screen) into a first distance range and a second distance range. After detecting the first position of the user's finger in front of the screen, the terminal can determine in which distance range the first position is located. If the first position is within the first distance range, the 3D object within the first depth of field can be operated. If the first position is within the second distance range, the 3D object within the second depth of field can be operated. In this way, the user can operate the 3D objects in front of and behind the screen from the front of the screen.

[0039] In some embodiments, the first distance range and the second distance range described above may satisfy at least one of the following: the first distance range and the second distance range do not overlap; the first distance range is close to the human eye, and the second distance range is close to the screen; the sum of the first distance range and the second distance range is equal to the first depth of field.

[0040] For example, in some embodiments, the first distance range is close to the human eye, and the second distance range is close to the screen. In this way, the user can operate the 3D object in front of the screen (i.e., the 3D object within the first depth of field) within the distance range close to the human eye, and can operate the 3D object behind the screen (i.e., the 3D object within the second depth of field) within the distance range close to the screen. As a result, the user can operate the 3D object in front of the terminal screen within a short distance range and operate the 3D object behind the terminal screen within a long distance range, which is in line with the user's operating habits.

[0041] For another example, in some embodiments, the first distance range and the second distance range do not overlap with each other and the sum of the first distance range and the second distance range is equal to the first depth of field. In this way, the user can operate 3D objects within different depths of field within different distance ranges. In addition, since the sum of the first distance range and the second distance range is equal to the first depth of field, the user's operable distance range can be limited to the first depth of field, which facilitates the terminal to detect the position of the user's finger.

[0042] In practical applications, the first distance range and the second distance range may be set according to actual needs so that the first distance range and the second distance range can meet one or more of the above three items, which are not limited here.

[0043] In some embodiments, if the first position is within the first distance range, operating the 3D object within the first depth of field may include the following steps: determining a second position corresponding to the first position within the first depth of field based on a first position mapping relationship between the first distance range and the first depth of field; and operating the 3D object corresponding to the second position within the first depth of field.

[0044] The first position mapping relationship may be a position mapping relationship between a space within a first distance range and a space within a first depth of field, and the first position mapping relationship may be predetermined. In some embodiments, the first position mapping relationship may include a first mapping relationship, a second mapping relationship, and a third mapping relationship. The first mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a first direction, the second mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a second direction, and the third mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a third direction. The first direction, the second direction, and the third direction are perpendicular to each other. The first direction, the second direction, or the third direction may be a direction perpendicular to the terminal screen.

[0045] The first position of the user's finger in front of the screen may be a coordinate position, which may include a first coordinate in a first direction, a second coordinate in a second direction, and a third coordinate in a third direction. Thus, when determining a second position corresponding to the first position within the first depth of field based on the first position mapping relationship, the following steps may be included: determining a fourth coordinate corresponding to the first coordinate based on the first mapping relationship; determining a fifth coordinate corresponding to the second coordinate based on the second mapping relationship; determining a sixth coordinate corresponding to the third coordinate based on the third mapping relationship; and determining the positions within the first depth of field corresponding to the fourth, fifth, and sixth coordinates as the second position corresponding to the first position.

[0046] After determining the second position, operations can be performed on the 3D object corresponding to the second position within the first depth of field. For example, if the user performs a drag operation at the first position, the terminal can perform corresponding operations on the 3D object corresponding to the second position within the first depth of field based on the user's drag operation. In this way, operations can be performed on the 3D object in front of the screen.

[0047] The 3D object corresponding to the second position may include at least one of a first object and a second object. The position of the first object within the first depth of field includes the second position, and the distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance. In other words, the 3D object corresponding to the second position may be the 3D object at the second position, or may be a 3D object at a position where the distance from the second position is less than or equal to the preset distance. The preset distance may be set according to actual needs and is not limited here.

[0048] In some embodiments, if the first position is within the second distance range, operating the 3D object within the first depth of field may include the following steps: determining a third position corresponding to the first position within the second depth of field based on a second position mapping relationship between the second distance range and the second depth of field; and operating the 3D object corresponding to the third position within the second depth of field.

[0049] The second position mapping relationship may be a position mapping relationship between a space within a second distance range and a space within a second depth of field, and the second position mapping relationship may be predetermined. In some embodiments, the second position mapping relationship may include a fourth mapping relationship, a fifth mapping relationship, and a sixth mapping relationship. The fourth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in the first direction, the fifth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in the second direction, and the sixth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in the third direction. The first direction, the second direction, and the third direction are perpendicular to each other. The first direction, the second direction, or the third direction may be a direction perpendicular to the terminal screen.

[0050] The first position of the user's finger in front of the screen can be a coordinate position, which can include a first coordinate in a first direction, a second coordinate in a second direction, and a third coordinate in a third direction. Thus, when determining a third position corresponding to the first position within the second depth of field based on the second position mapping relationship, the following steps can be included: determining a seventh coordinate corresponding to the first coordinate based on the first mapping relationship; determining an eighth coordinate corresponding to the second coordinate based on the second mapping relationship; determining a ninth coordinate corresponding to the third coordinate based on the third mapping relationship; and determining the positions within the second depth of field corresponding to the seventh, eighth, and ninth coordinates as the third position corresponding to the first position.

[0051] After determining the third position, operations can be performed on the 3D object corresponding to the third position within the second depth of field. For example, if the user performs a rotation operation at the third position, the terminal can perform corresponding operations on the 3D object corresponding to the third position within the second depth of field based on the user's rotation operation. In this way, the user can operate 3D objects behind the screen while standing in front of the screen.

[0052] The 3D object corresponding to the third position may include at least one of a third object and a fourth object. The position of the third object within the second depth of field includes the third position, and the distance between the position of the fourth object within the second depth of field and the third position is less than or equal to the preset distance. In other words, the 3D object corresponding to the third position may be the 3D object at the third position, or may be a 3D object at a position where the distance from the third position is less than or equal to the preset distance. The preset distance may be set according to actual needs and is not limited here.

[0053] To facilitate understanding of how to determine the second position according to the first position mapping relationship and how to determine the third position according to the second position mapping relationship, an implementation manner shown in FIG3 is taken as an example for description below.

[0054] In Figure 3, B1 represents the terminal screen surface, A1 represents the boundary of the 3D object visible behind the terminal screen, and C1 represents the boundary of the 3D object visible in front of the terminal screen. A1B1 represents the second depth of field, and B1C1 represents the first depth of field. In segment A1B1, the user can see 3D object D1, but because D1 is behind the screen, the user's finger cannot reach D1 when they want to operate it.

[0055] Based on the technical solution provided in the embodiment of the present application, the user's visual depth A1C1 of the 3D object can be mapped to the distance range A2C2 in front of the terminal screen (here A2C2 coincides with B1C1, and for ease of explanation, different letters are used to express them separately). Among them, the visual depth A1B1 behind the screen is mapped to the distance range A2B2 in front of the screen (i.e., the second position mapping relationship mentioned above), and the visual depth B1C1 in front of the screen is mapped to the distance range B2C2 in front of the screen (i.e., the first position mapping relationship mentioned above). In Figure 3, the coordinate axis in the horizontal direction (i.e., the coordinate axis perpendicular to the terminal screen) is the Z axis, the coordinate axis in the vertical direction is the Y axis, and the coordinate axis perpendicular to the Y axis and the Z axis is the X axis. When mapping A1B1 and A2B2, mapping can be performed in all three directions of the X axis, Y axis, and Z axis. Similarly, when mapping B1C1 and B2C2, mapping can also be performed in all three directions of the X axis, Y axis, and Z axis.

[0056] When the first position of the user's finger is detected within A2C2, the target position corresponding to the first position within A1C1 can be determined based on the aforementioned position mapping relationship between A1C1 and A2C2 (including the position mapping relationship between A1B1 and A2B2, and the position mapping relationship between B1C1 and B2C2, each of which includes mapping relationships in the X-axis, Y-axis, and Z-axis directions). To simplify the description, the following uses the Z-axis as an example to illustrate how to determine the target position corresponding to the first position of the user's finger within A1C1 based on the mapping relationship in the Z-axis direction.

[0057] Assuming that the coordinate of the first position of the user's finger on the Z axis is detected to be F2(z2), and the coordinate of the position corresponding to F2 in A1C1 on the Z axis is F1(z1), then F1(z1) can be determined by the following formula 1 or formula 2.

[0058] Formula 1: F1(z1)=f2(F2(z2))+β2, C2(z2)≥F2(z2)≥B2(z2).

[0059] Formula 2: F1(z1)=f1(F2(z2))+β1, B2(z2)≥F2(z2)≥A2(z2).

[0060] f1 and f2 are functions, and β1 and β2 are coefficients. Formula 1 shows the mapping relationship between B2C2 and B1C1 on the Z axis, and Formula 2 shows the mapping relationship between A2B2 and A1B1 on the Z axis. When the user's finger position F2 is within B2C2, F1 can be determined according to Formula 1. When the user's finger position F2 is within A2B2, F1 can be determined according to Formula 2.

[0061] For example, a PAD is a glasses-free 3D display that is 240 mm long and 180 mm wide. As shown in Figure 3, B1 of the 3D PAD is the center point of the screen and serves as the coordinate origin. The finger moves within the range of B1C1, and the mapped coordinates are A1C1. B2' is the projection of B2 onto B1C1. Among them, B1B2' = 20 mm, B2'C1 = 60 mm, B1C1 = 80 mm, and A1B1 = 80 mm. B1C1 is the positive half-axis of the z-axis, and B1A1 is the negative half-axis of the z-axis. Using the above formulas 1 and 2 to describe the relationship between the finger detection coordinate z2 and the finger's mapped coordinate z1, the above formula 1 can be expressed as:

[0062] z1=(B1B2'+B2'C1) / B2'C1*(z2-B1B2')+0=(20+60) / 60*(z2-20)+0, 80≥z2≥20.

[0063] Based on the above formula, if z2=80, then z1=(20+60) / 60*(80-20)+0=80.

[0064] If z2=60, then z1=(20+60) / 60*(60-20)+0=53.33.

[0065] If z2=40, then z1=(20+60) / 60*(40-20)+0=26.66.

[0066] If z2=20, then z1=(20+60) / 60*(20-20)+0=0.

[0067] That is, when the user's finger is 80 / 60 / 40 / 20 mm away from the screen, the mapping coordinates of the finger in the 3D interface are 80 / 53.33 / 26.660 mm, and F1 is in front of the screen at this time.

[0068] The above formula 2 can be expressed as.

[0069] z1=A1B1 / B1B2'*z2-A1B1=80 / 20*z2+(-80), 20≥z2≥0.

[0070] Based on the above formula, if z2=20, then z1=80 / 20*20-80=0.

[0071] If z2=10, then z1=80 / 20*10-80=-40.

[0072] If z2=5, then z1=80 / 20*5-80=-60.

[0073] If z2=0, then z1=80 / 20*0-80=-80.

[0074] That is, when the user's finger is 20 / 10 / 5 / 0 mm away from the screen, the mapping coordinates of the finger in the 3D interface are 0 / -40 / -60 / -80 mm, and the F1 is already behind the screen.

[0075] After determining the F1 corresponding to F2, when F1 coincides with the coordinates of a 3D object, it can be determined that the finger has touched the object. For example, the current terminal detects that the coordinate F2 of the finger is within the range of A2B2, and the finger is at position D2, defined as F2(z2_d2). The range of F1 corresponding to the finger coordinates within A1C1 is A1B1, and the position mapped to D1 within the range of A1B1 is F1(z1_d1). At this time, there is a virtual image in the interface with coordinates Img(z3). If the coordinate values ​​of F1(z1_d1) and Img(z3) are the same or their error is less than a certain range, the terminal can determine that the finger has touched the virtual image. In this way, the finger's activity range can be within the A2C2 range in front of the screen, but the range of the 3D object that the finger can operate is A1C1 (including the front and back of the screen).

[0076] The coordinate system described in Figure 3 lays the foundation for realizing the 3D human-computer interface operation shown in Figure 4. As shown in Figure 4, the user's finger moves from the starting position toward the screen. At the target position 1, the mapped coordinates of the finger match the coordinates of the object in the 3D interface, and the terminal determines that the finger touches the object in the 3D interface. Push this target toward the screen. When the finger coordinates reach the B2 position, the 3D displayed object (the coordinates mapped by the finger) has reached the B1 position. The finger enters the B2B1 interval, and the 3D displayed object reaches the B1A1 interval. When the finger continues to move forward and reaches the B1 position (screen), the 3D displayed object reaches the A1 position, and the B1A1 position is behind the screen. In this way, operations can be implemented on objects in the entire space of the 3D interface, including objects in front of and behind the screen.

[0077] It should be noted that in the embodiment shown in Figure 3, the touch range of the user's finger is the same as the position range of the 3D object presentation (i.e., A2C2 equals B1C1). In actual applications, in order to maintain a good operating experience, the presentation position of the 3D object is correlated with the size of the terminal display screen. The touch range of the user's finger (i.e., the operating range) can be at a position in front of the screen that is about 1 / 2 of the screen width from the screen. As shown in Figure 5, oa is 1 / 2 of the screen width, ob is the farthest distance the user's finger can operate the virtual image, and ob is approximately equal to oa.

[0078] The values ​​of A1B1, B1C1, and B1B2' shown in Figure 3 can be optimized and adjusted based on actual product requirements in practical applications. Table 1 provides recommended values ​​for several products, which can be adjusted appropriately in practical applications.

[0079] Table 1

[0080] In some embodiments, after determining the 3D object corresponding to the second position or the third position, visual feedback may be provided to the user regarding the 3D object. The visual feedback may include at least one of the following: highlighting the 3D object; shaking the 3D object; or changing the display color of the 3D object.

[0081] As shown in FIG6 , the terminal can highlight or shake the 3D object, thereby improving the user's experience of touching the 3D object.

[0082] It should be noted that when providing visual feedback to the user, the timing of the feedback can be after determining the 3D object corresponding to the second position and before operating the 3D object, or it can be provided during the operation of the 3D object, without limitation. If visual feedback is provided during the operation of the 3D object, it can help the user know which 3D object is currently being operated on, and the visual effect is better. If visual feedback is provided after determining the 3D object corresponding to the second position and before operating the 3D object, it can help the user accurately distinguish which 3D object is currently being operated by the finger. If the 3D object is the object the user wants to operate, the user can operate it accordingly. If the 3D object is not the object the user wants to operate, the user can adjust the position of the finger based on the position of the 3D object to find the 3D object they want to operate. For example, if the 3D object the user wants to operate is visually located in front of the currently highlighted 3D object, the user can move the finger closer to the human eye until the position of the 3D object the user wants to operate is highlighted. This can improve the accuracy of user operation and enhance the user experience.

[0083] Fig. 7 is a flowchart of a method for operating a 3D object according to another embodiment of the present application. The embodiment shown in Fig. 7 may include the following steps.

[0084] S701: The terminal is initialized.

[0085] S702: The terminal starts the 3D terminal product.

[0086] S703: The terminal displays the screen content in stereoscopic form and displays a 3D interface.

[0087] S704: The terminal detects the user's eye position and adjusts the 3D interface display.

[0088] S705: The terminal detects the position of the user's finger and determines the coordinate position of the user's finger in the 3D space.

[0089] S706: The terminal determines whether the coordinate position of the user's finger coincides with the coordinate position of the 3D object.

[0090] If yes, execute S207; if no, return to execute S705.

[0091] S707: Highlight the 3D object.

[0092] You can also shake the 3D object or change the color of the 3D object.

[0093] S708: The terminal performs corresponding operations on the 3D object according to the user operation.

[0094] S709: End.

[0095] The implementation of the above S701 to S709 can refer to the implementation of the corresponding steps in the embodiment shown in FIG2 , and will not be described in detail here.

[0096] In an embodiment of the present application, when a terminal displays screen content in stereoscopic form and the visual depth of the screen content to the human eye includes a first depth of field in front of the screen and a second depth of field behind the screen, the range in front of the terminal screen can be divided into a first distance range and a second distance range. When the position of the user's finger in front of the screen is within the first distance range, 3D objects within the first depth of field can be operated. When the position of the user's finger in front of the screen is within the second distance range, 3D objects within the second depth of field can be operated. In this way, by corresponding the first depth of field visually located in front of the screen and the second depth of field located behind the screen to the first distance range and the second distance range in front of the screen, respectively, operations on 3D objects in front of and behind the screen can be implemented.

[0097] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0098] FIG8 is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Referring to FIG8 , at the hardware level, the electronic device includes a processor, an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk storage device. Of course, the electronic device may also include hardware required for other services.

[0099] The processor, network interface, and memory can be interconnected via an internal bus, such as an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. These buses can be classified as address buses, data buses, and control buses. For ease of illustration, FIG8 shows only one bidirectional arrow, but this does not imply that there is only one bus or only one type of bus.

[0100] The memory is used to store programs. The programs may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0101] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a 3D object operation device at a logical level. The processor executes the program stored in the memory and is configured to perform the following operations: when the terminal displays the screen content in stereoscopic mode, determine the visual depth of the screen content to the human eye, wherein the visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen; detect a first position of a user's finger in front of the screen; and when the first position is within a first distance range of the screen, operate the 3D object within the first depth of field; and when the first position is within a second distance range of the screen, operate the 3D object within the second depth of field.

[0102] The method performed by the 3D object operation device disclosed in the embodiment shown in Figure 8 of the present application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor or software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps and logic block diagrams disclosed in this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0103] The electronic device can also execute the method of FIG. 2 and realize the function of the drive access device in the embodiment shown in FIG. 2 , which will not be described in detail in this application.

[0104] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0105] The present application also proposes a computer-readable storage medium, which stores one or more programs, each of which includes instructions. When the instructions are executed by a portable electronic device including multiple applications, the portable electronic device can execute the method of the embodiment shown in Figure 2 and is used to perform the following operations: when the terminal displays the screen content in stereoscopic mode, determine the visual depth of the screen content to the human eye, the visual depth including a first depth of field and a second depth of field, the first depth of field being located in front of the screen and the second depth of field being located behind the screen; detect a first position of the user's finger in front of the screen; when the first position is within a first distance range of the screen, operate a 3D object within the first depth of field, and when the first position is within a second distance range of the screen, operate the 3D object within the second depth of field.

[0106] FIG9 is a schematic diagram of a 3D object operation device 90 according to an embodiment of the present application. Referring to FIG9 , in a software implementation, the 3D object operation device 90 may include: a determination module 91, a detection module 92, and an operation module 93. The determination module 91 determines the visual depth of the screen content as viewed by the human eye, when the terminal displays the screen content in stereoscopic mode. The visual depth includes a first depth of field and a second depth of field, where the first depth of field is located in front of the screen and the second depth of field is located behind the screen. The detection module 92 detects a first position of a user's finger in front of the screen. The operation module 93 operates a 3D object within a first depth of field when the first position is within a first distance range of the screen, and operates a 3D object within a second depth of field when the first position is within a second distance range of the screen.

[0107] In some embodiments, the first distance range and the second distance range satisfy at least one of the following: the first distance range and the second distance range do not overlap with each other; the first distance range is close to the human eye, and the second distance range is close to the screen; the sum of the first distance range and the second distance range is equal to the first depth of field.

[0108] In some embodiments, the operation module 93 operates the 3D object within the first depth of field when the first position is within a first distance range of the screen, including: determining a second position corresponding to the first position within the first depth of field based on a first position mapping relationship between the first distance range and the first depth of field when the first position is within the first distance range of the screen; and operating the 3D object corresponding to the second position within the first depth of field.

[0109] In some embodiments, the operation module 93 operates the 3D object within the second depth of field when the first position is within the second distance range of the screen, including: determining a third position corresponding to the first position within the second depth of field based on a second position mapping relationship between the second distance range and the second depth of field when the first position is within the second distance range of the screen; and operating the 3D object corresponding to the third position within the second depth of field.

[0110] In some embodiments, the first position mapping relationship includes a first mapping relationship, a second mapping relationship, and a third mapping relationship; wherein, the first mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a first direction, the second mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a second direction, and the third mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a third direction, and the first direction, the second direction, and the third direction are perpendicular to each other.

[0111] In some embodiments, the first position includes a first coordinate in the first direction, a second coordinate in the second direction, and a third coordinate in the third direction; the operation module 93 determines the second position corresponding to the first position within the first depth of field, including: determining a fourth coordinate corresponding to the first coordinate according to the first mapping relationship; determining a fifth coordinate corresponding to the second coordinate according to the second mapping relationship; determining a sixth coordinate corresponding to the third coordinate according to the third mapping relationship; and determining the position corresponding to the fourth coordinate, the fifth coordinate, and the sixth coordinate within the first depth of field as the second position.

[0112] In some embodiments, the 3D object corresponding to the second position includes at least one of a first object and a second object; wherein the position of the first object within the first depth of field includes the second position; and the distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance.

[0113] In some embodiments, the operation module 93 further includes at least one of the following: highlighting the 3D object; performing a shaking operation on the 3D object; and changing a display color of the 3D object.

[0114] The 3D object operating device 90 provided in this application can also execute the method of Figure 2 and realize the functions of the 3D object operating device 90 in the embodiment shown in Figure 2, which will not be repeated in this application.

[0115] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0116] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0117] An embodiment of the present application further provides a computer program / program product, which is stored in a storage medium. The computer program / program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0118] An embodiment of the present application further provides a processing device, which is configured to execute the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described here.

[0119] In short, the above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

[0120] The systems, devices, modules, or units described in the above embodiments may be implemented by a computer chip or entity, or by a product having a certain function. A typical implementation device is a computer. The computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0121] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0123] The various embodiments in this application are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment.

Claims

1. A method for operating a 3D object, comprising: When the terminal performs stereoscopic display on the screen content, determining the visual depth of the human eye with respect to the screen content, the visual depth including a first depth of field and a second depth of field, the first depth of field being in front of the screen and the second depth of field being behind the screen; Detecting a first position of the user's finger in front of the screen; When the first position is within a first distance range from the screen, operating on the 3D object within the first depth of field, and when the first position is within a second distance range from the screen, operating on the 3D object within the second depth of field.

2. The method according to claim 1, comprising at least one of the following: The first distance range and the second distance range do not overlap; The first distance range is close to the human eye and the second distance range is close to the screen; The sum of the first distance range and the second distance range is equal to the first depth of field.

3. The method according to claim 1, wherein when the first position is within the first distance range from the screen, operating on the 3D object within the first depth of field comprises: When the first position is within the first distance range from the screen, determining a second position corresponding to the first position within the first depth of field according to a first position mapping relationship between the first distance range and the first depth of field; Operating on the 3D object within the first depth of field corresponding to the second position.

4. The method according to claim 1, wherein when the first position is within the second distance range from the screen, operating on the 3D object within the second depth of field comprises: When the first position is within the second distance range from the screen, determining a third position corresponding to the first position within the second depth of field according to a second position mapping relationship between the second distance range and the second depth of field; Operating on the 3D object within the second depth of field corresponding to the third position.

5. The method according to claim 3, wherein the first position mapping relationship includes a first mapping relationship, a second mapping relationship, and a third mapping relationship; Among them, The first mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a first direction, the second mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a second direction, and the third mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a third direction, the first direction, the second direction, and the third direction being perpendicular to each other.

6. The method according to claim 5, wherein the first position includes a first coordinate in the first direction, a second coordinate in the second direction, and a third coordinate in the third direction; The determining of the second position corresponding to the first position within the first depth of field comprises: Determining a fourth coordinate corresponding to the first coordinate according to the first mapping relationship; Determining a fifth coordinate corresponding to the second coordinate according to the second mapping relationship; Determining a sixth coordinate corresponding to the third coordinate according to the third mapping relationship; Determining the position within the first depth of field corresponding to the fourth coordinate, the fifth coordinate, and the sixth coordinate as the second position.

7. The method according to claim 3, wherein the 3D object corresponding to the second position includes at least one of a first object and a second object; Among them, the position of the first object within the first depth of field includes the second position; the distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance.

8. The method according to claim 1, the method further comprising at least one of the following: highlighting the 3D object; performing a shaking operation on the 3D object; changing the display color of the 3D object.

9. An electronic device, comprising: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enabling the electronic device to execute the method according to any one of claims 1 to 8.

Images