Methods for manipulating 3D objects, electronic devices, and computer-readable storage media

Abstract

This application provides a method for manipulating 3D objects, an electronic device, and a computer-readable storage medium. The method includes: determining the visual depth of the screen content as viewed by a human eye when the terminal displays the screen content in stereoscopic form; the visual depth includes a first depth of field and a second depth of field, the first depth of field being located in front of the screen and the second depth of field being located behind the screen; detecting a first position of a user's finger in front of the screen; and manipulating a 3D object within the first depth of field when the first position is within a first distance range of the screen, and manipulating a 3D object within the second depth of field when the first position is within a second distance range of the screen. Thus, by corresponding the first depth of field visually located in front of the screen and the second depth of field located behind the screen to the first and second distance ranges in front of the screen, respectively, manipulation of 3D objects in front of and behind the screen can be achieved.

Description

Technical Field

[0001] This application relates to the field of terminal display technology, and in particular to a method for operating a 3D object, an electronic device, and a computer-readable storage medium. Background Technology

[0002] With the development of stereoscopic display technology, more and more terminals can use stereoscopic display technology to display the content displayed on the terminal screen in stereoscopic form, so that users can have a 3D visual effect when viewing the screen content, thereby improving the user experience.

[0003] The principle behind using stereoscopic display technology to display screen content in a stereoscopic manner can be summarized as follows: by altering the display characteristics of the screen content, a certain parallax is created between the left and right eyes when the user views the content, thus giving the screen content a sense of depth and space, thereby achieving a 3D display effect. In related technologies, to further enhance the user experience, users can interact with the stereoscopically displayed 3D objects, such as moving or rotating them. However, in some application scenarios, when the terminal displays screen content in stereoscopic form, the visual depth perceived by the user's eyes may include both the area in front of and behind the screen; that is, 3D objects include those located in front of and behind the screen. Users can directly interact with 3D objects in front of the screen, but cannot directly interact with 3D objects behind the screen because they are visually located behind the screen. Summary of the Invention

[0004] This application provides a method for manipulating 3D objects, an electronic device, and a computer-readable storage medium to solve the problem that users cannot manipulate 3D objects that are visually located behind the terminal screen in scenarios where the terminal displays screen content in 3D stereoscopic form.

[0005] To solve the above-mentioned technical problems, this application is implemented as follows:

[0006] Firstly, a method for manipulating 3D objects is provided, including:

[0007] When the terminal displays the screen content in stereoscopic form, the visual depth of the human eye on the screen content is determined. The visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen.

[0008] Detect the first position of the user's finger in front of the screen;

[0009] When the first position is within a first distance range of the screen, the operation is performed on the 3D object within the first depth of field; when the first position is within a second distance range of the screen, the operation is performed on the 3D object within the second depth of field.

[0010] Secondly, an electronic device is provided, comprising:

[0011] processor;

[0012] Memory used to store the processor's executable instructions;

[0013] The processor is configured to execute the instructions to implement the method as described in the first aspect.

[0014] Thirdly, a computer-readable storage medium is provided, wherein when instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method described in the first aspect.

[0015] In this embodiment, when the terminal displays the screen content in stereoscopic form, and the visual depth of the screen content perceived by the human eye includes a first depth of field in front of the screen and a second depth of field behind the screen, the area in front of the terminal screen can be divided into a first distance range and a second distance range. When the user's finger is positioned within the first distance range in front of the screen, 3D objects within the first depth of field can be manipulated. When the user's finger is positioned within the second distance range in front of the screen, 3D objects within the second depth of field can be manipulated. Thus, by corresponding the visually perceived first depth of field in front of the screen and the second depth of field behind the screen with the first and second distance ranges in front of the screen, manipulation of 3D objects in front of and behind the screen can be achieved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of a terminal displaying screen content in a stereoscopic manner in related technologies;

[0018] Figure 2 This is a flowchart illustrating a method for operating a 3D object according to an embodiment of this application;

[0019] Figure 3This is a schematic diagram illustrating an embodiment of the present application that determines the target position corresponding to the first position of the user's finger within the visual depth based on a position mapping relationship;

[0020] Figure 4 This is a schematic diagram illustrating the manipulation of a 3D object according to an embodiment of this application;

[0021] Figure 5 This is a schematic diagram illustrating the operable range of a user's finger according to an embodiment of this application;

[0022] Figure 6 This is a schematic diagram illustrating visual feedback to a 3D object according to an embodiment of this application;

[0023] Figure 7 This is a flowchart illustrating the operation method of a 3D object according to another embodiment of this application;

[0024] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram of the structure of a 3D object operation device according to an embodiment of this application. Detailed Implementation

[0026] In some application scenarios, when a terminal displays screen content in stereoscopic form, the visual depth of the screen content perceived by the user's eyes includes both the area in front of and behind the screen. Similarly, 3D objects also include those located in front of and behind the screen. For example... Figure 1 As shown, when the terminal displays the screen content in stereoscopic form, the visual depth of the user's eyes includes the visual depth BC located in front of the screen and the visual depth AB located behind the screen.

[0027] When users interact with 3D objects, they can directly manipulate those in front of the screen, but cannot directly manipulate those behind the screen. Related technologies may employ a method such as panning to bring 3D objects from behind the screen to the front before allowing interaction. For example, to click on a 3D object behind the screen, the user could first bring the object to the front, click on it, and then bring it back to the back. However, this method is cumbersome and also disrupts the original layout, causing significant inconvenience for users in practice.

[0028] This application provides a method for manipulating 3D objects, an electronic device, and a computer-readable storage medium. When a terminal displays screen content in stereoscopic form, and the visual depth of the screen content for the human eye includes a first depth of field in front of the screen and a second depth of field behind the screen, the area in front of the terminal screen can be divided into a first distance range and a second distance range. When the user's finger is positioned within the first distance range in front of the screen, 3D objects within the first depth of field can be manipulated; when the user's finger is positioned within the second distance range in front of the screen, 3D objects within the second depth of field can be manipulated. Thus, by corresponding the visually perceived first depth of field in front of the screen and the second depth of field behind the screen with the first and second distance ranges in front of the screen, manipulation of 3D objects in front of and behind the screen can be achieved.

[0029] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in this application will be clearly and completely described below with reference to the accompanying drawings of one or more embodiments. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of this application.

[0030] The terms "first," "second," etc., used in this application and the claims are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that this application can be implemented in orders other than those illustrated or described herein. Furthermore, in this application and the claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] It should be noted that the technical solutions provided in this application embodiment are applicable to naked-eye 3D scenarios, and optionally, they are also applicable to non-naked-eye 3D scenarios; no specific limitations are made here. The terminal in this application embodiment can be a terminal with stereoscopic display functionality. Optionally, in some more specific implementations, the terminal may include, but is not limited to, a 3D interface display device, a camera system device (single or dual cameras), a touch detection system, a gyroscope sensor, a gravity acceleration sensor, a microprocessor, a baseband module, a wireless transceiver module, a power module, a terminal operating system, and a user operation command monitoring system. In the technical solutions provided in this application embodiment, the 3D interface display device can be used to present 3D display effects. Optionally, in naked-eye 3D scenarios, the 3D display device may include a 3D naked-eye flat panel display. The terminal operating system can be used to present a 3D interface on the 3D interface display device, perform interface switching and navigation when receiving user operations, and perform data calculations, and may include an application layer, a framework layer, and a kernel layer. The user operation command monitoring system can acquire image information from a camera, analyze and detect user gestures and / or facial information, determine the user's hand movements, finger positions, eye gaze positions, lip movements, facial expressions, etc., and correspond these to preset interface operation commands. The camera device can be used to detect gesture dynamics and gesture position depth, that is, to determine the position and distance of fingers, palms, etc., from the camera. The commands parsed by the user operation command monitoring system can be transmitted to the terminal operating system, and the terminal operates the interface according to the commands.

[0032] When the terminal displays the screen content in stereoscopic form, the displayed 3D object can be a virtual image or the like, without any specific limitations.

[0033] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0034] Figure 2 This is a flowchart illustrating an embodiment of the operation method of a 3D object in this application. Figure 2 The illustrated embodiment includes the following steps.

[0035] S202: When the terminal displays the screen content in stereoscopic form, determine the visual depth of the screen content for the human eye. The visual depth includes a first depth of field and a second depth of field. The first depth of field is located in front of the screen, and the second depth of field is located behind the screen.

[0036] When a terminal receives a command to display screen content in stereoscopic mode, or in a scenario where stereoscopic display of screen content is required, it can use stereoscopic display technology to display the screen content in stereoscopic mode. For details on how the terminal uses stereoscopic display technology to display screen content in stereoscopic mode, please refer to the relevant technical documentation; these details will not be elaborated upon here.

[0037] After the terminal displays the screen content in stereoscopic form, the screen content will be presented to the user with a 3D display effect. For the user, the screen content they see will have a clear sense of depth, that is, the user's eyes will have a certain visual depth of the screen content. In this case, the terminal can determine the user's visual depth of the screen content.

[0038] When determining the visual depth of a user's eyes on screen content, the terminal can optionally use stereoscopic display technology to determine the parallax between the user's left and right eyes when viewing the screen content, and then determine the visual depth of the user's eyes on the screen content based on the parallax. Of course, other methods can also be used to determine visual depth, which will not be listed here.

[0039] In this embodiment, the visual depth of the user's eyes on the screen content includes the visual depth in front of the terminal screen and the visual depth behind the terminal screen. For ease of distinction, the visual depth in front of the terminal screen can be represented as the first depth of field, and the visual depth behind the terminal screen can be represented as the second depth of field. After obtaining the visual depth of the user's eyes on the screen content, the terminal can further obtain the first depth of field and the second depth of field.

[0040] S204: Detects the first position of the user's finger in front of the screen.

[0041] When the terminal displays the screen content in stereoscopic form, it can detect the position of the user's finger in front of the screen. For ease of distinction, the position of the user's finger in front of the screen can be referred to as the first position. Optionally, the first position can be the relative position of the user's finger with respect to the terminal screen.

[0042] When detecting the position of a user's finger, the terminal can optionally use its camera to capture an image of the area in front of the terminal screen. Based on the captured image, the terminal can identify the user's finger, calculate the distance between the finger and the screen, and thus determine the initial position of the finger in front of the screen. Of course, other methods can also be used to determine the initial position of the finger in front of the screen, which will not be elaborated upon here.

[0043] S206: When the first position is within a first distance range of the screen, operate on a 3D object within a first depth of field; when the first position is within a second distance range of the screen, operate on a 3D object within a second depth of field.

[0044] The terminal can divide the distance range in front of the screen (the distance range perpendicular to the terminal screen) into a first distance range and a second distance range. After detecting that the user's finger is in the first position in front of the screen, the terminal can determine which distance range that first position is located within. If the first position is within the first distance range, then 3D objects within the first depth of field can be manipulated. If the first position is within the second distance range, then 3D objects within the second depth of field can be manipulated. Therefore, the user can manipulate 3D objects in front of and behind the screen from the position in front of the screen.

[0045] Optionally, in some embodiments, the first distance range and the second distance range described above may satisfy at least one of the following:

[0046] The first distance range and the second distance range do not overlap;

[0047] The first distance range is close to the human eye, and the second distance range is close to the screen;

[0048] The sum of the first distance range and the second distance range equals the first depth of field.

[0049] For example, in some implementations, the first distance range is closer to the human eye, and the second distance range is closer to the screen. In this way, the user can operate on the 3D objects in front of the screen (i.e., 3D objects within the first depth of field) within the distance range close to the human eye, and operate on the 3D objects behind the screen (i.e., 3D objects within the second depth of field) within the distance range close to the screen. Thus, the user can operate on the 3D objects in front of the terminal screen at a close distance and operate on the 3D objects behind the terminal screen at a far distance, which conforms to the user's operating habits.

[0050] For example, in some implementations, the first distance range and the second distance range do not overlap and the sum of the first distance range and the second distance range is equal to the first depth of field. In this way, the user can operate on 3D objects within different depths of field at different distance ranges. In addition, since the sum of the first distance range and the second distance range is equal to the first depth of field, the user's operable distance range can be limited to the first depth of field, making it convenient for the terminal to detect the position of the user's finger.

[0051] In practical applications, a first distance range and a second distance range can be set according to actual needs, so that the first distance range and the second distance range can satisfy one or more of the above three conditions, without making specific limitations here.

[0052] Optionally, in some implementations, if the first position is within a first distance range, operating on a 3D object within a first depth of field may include the following steps:

[0053] Based on the first position mapping relationship between the first distance range and the first depth of field, determine the second position corresponding to the first position within the first depth of field;

[0054] Perform operations on the 3D object corresponding to the second position within the first depth of field.

[0055] The first position mapping relationship can be a position mapping relationship between space within a first distance range and space within a first depth of field, and the first position mapping relationship can be predetermined. Optionally, in some embodiments, the first position mapping relationship may include a first mapping relationship, a second mapping relationship, and a third mapping relationship. The first mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a first direction, the second mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a second direction, and the third mapping relationship is a position mapping relationship between the first distance range and the first depth of field in a third direction. The first direction, the second direction, and the third direction are mutually perpendicular. Optionally, the first direction, the second direction, or the third direction can be a direction perpendicular to the terminal screen.

[0056] The first position of the user's finger in front of the screen can be a coordinate position, specifically including a first coordinate in a first direction, a second coordinate in a second direction, and a third coordinate in a third direction. Thus, determining the second position within a first depth of field corresponding to the first position based on the first position mapping relationship can include the following steps:

[0057] Determine the fourth coordinate corresponding to the first coordinate based on the first mapping relationship;

[0058] Determine the fifth coordinate corresponding to the second coordinate based on the second mapping relationship;

[0059] Determine the sixth coordinate corresponding to the third coordinate based on the third mapping relationship;

[0060] The position within the first depth of field corresponding to the fourth, fifth, and sixth coordinates is determined as the second position corresponding to the first position.

[0061] After determining the second position, operations can be performed on the 3D object within the first depth of field that corresponds to the second position. For example, if the user performs a drag operation at the first position, the terminal can perform corresponding operations on the 3D object within the first depth of field that corresponds to the second position based on the user's drag operation. This allows for manipulation of 3D objects in front of the screen.

[0062] The 3D object corresponding to the second position can include at least one of the first object and the second object. The position of the first object within the first depth of field includes the second position, and the distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance. That is, the 3D object corresponding to the second position can be a 3D object at the second position, or a 3D object at a position where the distance to the second position is less than or equal to the preset distance. The preset distance can be set according to actual needs and is not specifically limited here.

[0063] Optionally, in some implementations, if the first position is within the second distance range, operating on the 3D object within the first depth of field may include the following steps:

[0064] Based on the second position mapping relationship between the second distance range and the second depth of field, determine the third position corresponding to the first position within the second depth of field;

[0065] Perform operations on the 3D object corresponding to the third position within the second depth of field.

[0066] The second position mapping relationship can be a position mapping relationship between space within a second distance range and space within a second depth of field, and the second position mapping relationship can be predetermined. Optionally, in some embodiments, the second position mapping relationship may include a fourth mapping relationship, a fifth mapping relationship, and a sixth mapping relationship. The fourth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in a first direction, the fifth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in a second direction, and the sixth mapping relationship is a position mapping relationship between the second distance range and the second depth of field in a third direction. The first direction, the second direction, and the third direction are mutually perpendicular. Optionally, the first direction, the second direction, or the third direction can be a direction perpendicular to the terminal screen.

[0067] The first position of the user's finger in front of the screen can be a coordinate position, specifically including a first coordinate in a first direction, a second coordinate in a second direction, and a third coordinate in a third direction. Thus, determining the third position within the second depth of field corresponding to the first position based on the second position mapping relationship can include the following steps:

[0068] Determine the seventh coordinate corresponding to the first coordinate based on the first mapping relationship;

[0069] Determine the eighth coordinate corresponding to the second coordinate based on the second mapping relationship;

[0070] Determine the ninth coordinate corresponding to the third coordinate based on the third mapping relationship;

[0071] The position within the second depth of field corresponding to the seventh, eighth, and ninth coordinates is determined as the third position corresponding to the first position.

[0072] Once the third position is determined, operations can be performed on the 3D objects within the second depth of field that correspond to that third position. For example, if the user performs a rotation operation at the third position, the terminal can perform corresponding operations on the 3D objects within the second depth of field that correspond to the third position based on the user's rotation operation. Thus, the user can manipulate 3D objects behind the screen while in front of the screen.

[0073] The 3D object corresponding to the third position can include at least one of a third object and a fourth object. The position of the third object within the second depth of field includes the third position, and the distance between the position of the fourth object within the second depth of field and the third position is less than or equal to a preset distance. That is, the 3D object corresponding to the third position can be a 3D object at the third position, or a 3D object at a position where the distance to the third position is less than or equal to the preset distance. The preset distance can be set according to actual needs and is not specifically limited here.

[0074] To facilitate understanding how to determine the second position based on the first position mapping relationship, and how to determine the third position based on the second position mapping relationship, the following will use... Figure 3 The following is an example of a more specific implementation method.

[0075] Figure 3 In the diagram, B1 represents the surface of the terminal screen, A1 represents the boundary of the visible 3D object behind the terminal screen, and C1 represents the boundary of the 3D object visible to the user in front of the terminal screen. A1B1 represents the second depth of field, and B1C1 represents the first depth of field. Specifically, in segment A1B1, the user can see the 3D object D1; however, because D1 is located behind the screen, the user's finger cannot reach it to interact with it.

[0076] Based on the technical solution provided in this application embodiment, the user's visual depth A1C1 of a 3D object can be mapped to the distance range A2C2 in front of the terminal screen (here A2C2 coincides with B1C1, but different letters are used to express them separately for ease of explanation). Specifically, the visual depth A1B1 behind the screen is mapped to the distance range A2B2 in front of the screen (i.e., the second position mapping relationship mentioned above), and the visual depth B1C1 in front of the screen is mapped to the distance range B2C2 in front of the screen (i.e., the first position mapping relationship mentioned above). Figure 3In this diagram, the horizontal coordinate axis (i.e., the coordinate axis perpendicular to the terminal screen) is the Z-axis, the vertical coordinate axis is the Y-axis, and the coordinate axis perpendicular to both the Y and Z axes is the X-axis. When mapping A1B1 to A2B2, mapping can be performed in all three directions: X, Y, and Z. Similarly, when mapping B1C1 to B2C2, mapping can also be performed in all three directions: X, Y, and Z.

[0077] When the first position of the user's finger is detected to be within A2C2, the target position within A1C1 corresponding to the first position can be determined based on the positional mapping relationship between A1C1 and A2C2 (including the positional mapping relationship between A1B1 and A2B2, and the positional mapping relationship between B1C1 and B2C2, each including mapping relationships in the X-axis, Y-axis, and Z-axis directions). For simplicity, the following will use the Z-axis as an example to illustrate how to determine the target position within A1C1 corresponding to the first position of the user's finger based on the mapping relationship in the Z-axis direction.

[0078] Assuming the first detected position of the user's finger on the Z-axis is F2(z2), and the corresponding position within A1C1 on the Z-axis is F1(z1), then F1(z1) can be determined using either Formula 1 or Formula 2:

[0079] Formula 1: F1(z1)=f2(F2(z2))+β2, C2(z2)≥F2(z2)≥B2(z2);

[0080] Formula 2: F1(z1)=f1(F2(z2))+β1, B2(z2)≥F2(z2)≥A2(z2).

[0081] In the above equations, f1 and f2 are functions, and β1 and β2 are coefficients. Formula 1 represents the mapping relationship between B2C2 and B1C1 on the Z-axis, and Formula 2 represents the mapping relationship between A2B2 and A1B1 on the Z-axis. When the user's finger position F2 is within B2C2, F1 can be determined according to Formula 1; when the user's finger position F2 is within A2B2, F1 can be determined according to Formula 2.

[0082] For example, a certain tablet is a glasses-free 3D display, 240mm long and 180mm wide. Figure 3As shown, B1 of the 3D PAD is the center point of the screen, serving as the origin of the coordinate system. The finger moves within the range of B1C1, and the mapped coordinates are A1C1. B2' is the projection of B2 onto B1C1. Where B1B2' = 20mm, B2'C1 = 60mm, B1C1 = 80mm, and A1B1 = 80mm. B1C1 is the positive half-axis of the z-axis, and B1A1 is the negative half-axis of the z-axis. Using formulas 1 and 2 above to describe the relationship between the finger detection coordinate z2 and the mapped coordinate z1 of the finger, formula 1 can be expressed as:

[0083] z1=(B1B2'+B2'C1) / B2'C1*(z2-B1B2')+0=(20+60) / 60*(z2-20)+0, 80≥z2≥20.

[0084] Based on the above formula, if z2 = 80, then z1 = (20 + 60) / 60 * (80 - 20) + 0 = 80;

[0085] If z2 = 60, then z1 = (20 + 60) / 60 * (60 - 20) + 0 = 53.33;

[0086] If z2 = 40, then z1 = (20 + 60) / 60 * (40 - 20) + 0 = 26.66;

[0087] If z2 = 20, then z1 = (20 + 60) / 60 * (20 - 20) + 0 = 0.

[0088] That is, when the user's finger is 80 / 60 / 40 / 20mm away from the screen, the mapped coordinates of the finger in the 3D interface are 80 / 53.33 / 26.660mm, and at this time F1 is in front of the screen.

[0089] Formula 2 above can be expressed as:

[0090] z1=A1B1 / B1B2'*z2-A1B1=80 / 20*z2+(-80), 20≥z2≥0.

[0091] Based on the above formula, if z2 = 20, then z1 = 80 / 20*20 - 80 = 0;

[0092] If z2 = 10, then z1 = 80 / 20*10 - 80 = -40;

[0093] If z2 = 5, then z1 = 80 / 20 * 5 - 80 = -60;

[0094] If z2 = 0, then z1 = 80 / 20*0 - 80 = -80.

[0095] That is, when the user's finger is 20 / 10 / 5 / 0mm away from the screen, the mapped coordinates of the finger in the 3D interface are 0 / -40 / -60 / -80mm, and at this time F1 is behind the screen.

[0096] After determining F1 corresponding to F2, when F1 coincides with the coordinates of a 3D object, it can be determined that the finger has touched that object. For example, if the terminal detects that the finger's coordinates F2 are within the range A2B2, and the finger is located at position D2, defined as F2(z2_d2), the range of F1 corresponding to the finger's coordinates within A1C1 is A1B1, and the position of D2 within the range A1B1 is D1, with coordinates F1(z1_d1). If there is a virtual image on the interface with coordinates Img(z3), and if the coordinate values ​​of F1(z1_d1) and Img(z3) are the same or their error is less than a certain range, the terminal can determine that the finger has touched the virtual image. In this way, the finger's range of motion is limited to the A2C2 range in front of the screen, but the range of 3D objects that the finger can manipulate is A1C1 (including the area in front of and behind the screen).

[0097] like Figure 3 The described coordinate system is for achieving Figure 4 This lays the foundation for the 3D human-computer interface operation shown. For example... Figure 4 As shown, the user's finger moves towards the screen from the starting position. At target position 1, the finger's mapped coordinates match the object's coordinates in the 3D interface, and the terminal determines that the finger is touching an object in the 3D interface. This target is then pushed towards the screen. When the finger's coordinates reach position B2, the 3D displayed object (the coordinates mapped by the finger) has reached position B1. The finger enters the B2B1 interval, and the 3D displayed object reaches the B1A1 interval. When the finger continues to move forward to position B1 (the screen), the 3D displayed object reaches position A1, and position B1A1 is located behind the screen. This allows for manipulation of objects throughout the entire 3D interface space, including objects in front of and behind the screen.

[0098] It should be noted that, in Figure 3 In the illustrated embodiment, the user's finger's touch range is the same as the area where the 3D object is presented (i.e., A2C2 equals B1C1). In practical applications, to maintain a good user experience, the presentation position of the 3D object is related to the size of the terminal display screen. Preferably, the user's finger's touch range (i.e., the operating range) can be located approximately half the screen width away from the screen at its furthest point in front of the screen. Figure 5 As shown, oa is half the screen width, ob is the farthest distance the user's finger can reach when interacting with the virtual image, and ob is approximately equal to oa.

[0099] against Figure 3The specific values ​​of A1B1, B1C1, and B1B2' shown can be optimized and adjusted according to actual product requirements in practical applications. Table 1 provides recommended values ​​for several products, which can be adjusted appropriately in practical applications.

[0100] Table 1

[0101] 3D screen for mobile phones 6 inches, 132.8mm x 74.7mm 10mm 40mm 40mm 3D tablet screen 12 inches, 243.8mm x 182.9mm 20mm 80mm 80mm Computer 3D screen 27 inches, 617.5mm x 371.2mm 40mm 160mm 160mm

[0102] Optionally, in some implementations, after determining the 3D object corresponding to the second or third position, visual feedback can be provided to the user for that 3D object. When providing visual feedback, optionally, at least one of the following may be included:

[0103] Highlight 3D objects;

[0104] Perform a shaking operation on a 3D object;

[0105] Change the display color of 3D objects.

[0106] like Figure 6 As shown, the terminal can highlight or shake 3D objects, thereby improving the user's experience of touching 3D objects.

[0107] It should be noted that when providing visual feedback to users, the timing can be either after identifying the 3D object at the second position and before interacting with it, or during interaction with the 3D object; there is no specific limitation. Providing visual feedback during interaction with the 3D object makes it easier for users to identify which 3D object they are currently interacting with, resulting in a better visual experience. Providing visual feedback after identifying the 3D object at the second position and before interacting with it allows users to accurately identify which 3D object their finger is currently interacting with. If the 3D object is the one the user wants to interact with, the user can perform the corresponding operation. If the 3D object is not the one the user wants to interact with, the user can adjust the position of their finger based on the position of the 3D object to find the desired object. For example, if the 3D object the user wants to interact with is visually in front of the currently highlighted 3D object, the user can move their finger closer to their eyes until the desired 3D object is highlighted. This improves the accuracy of user operations and enhances the user experience.

[0108] Figure 7 This is a flowchart illustrating the operation method of a 3D object according to another embodiment of this application. Figure 7 The illustrated embodiments may include the following steps.

[0109] S701: The terminal is initialized.

[0110] S702: Terminal launches 3D terminal products.

[0111] S703: The terminal displays the screen content in stereoscopic form and displays a 3D interface.

[0112] S704: The terminal detects the user's eye position and adjusts the 3D interface display accordingly.

[0113] S705: The terminal detects the position of the user's finger and determines the coordinate position of the user's finger in 3D space.

[0114] S706: The terminal determines whether the coordinate position of the user's finger coincides with the coordinate position of the 3D object.

[0115] If yes, then execute S207; otherwise, return to execute S705.

[0116] S707: Highlight 3D objects.

[0117] Alternatively, you can shake the 3D object or change its color.

[0118] S708: The terminal performs corresponding operations on the 3D object based on the user's operation.

[0119] S709: End.

[0120] For specific implementation details of S701 to S709 above, please refer to [link / reference]. Figure 2 The specific implementation of the corresponding steps in the illustrated embodiments will not be described in detail here.

[0121] In this embodiment, when the terminal displays the screen content in stereoscopic form, and the visual depth of the screen content perceived by the human eye includes a first depth of field in front of the screen and a second depth of field behind the screen, the area in front of the terminal screen can be divided into a first distance range and a second distance range. When the user's finger is positioned within the first distance range in front of the screen, 3D objects within the first depth of field can be manipulated. When the user's finger is positioned within the second distance range in front of the screen, 3D objects within the second depth of field can be manipulated. Thus, by corresponding the visually perceived first depth of field in front of the screen and the second depth of field behind the screen with the first and second distance ranges in front of the screen, manipulation of 3D objects in front of and behind the screen can be achieved.

[0122] The foregoing has described specific embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0123] Figure 8 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Please refer to it. Figure 8 At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and memory. The memory may include main memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for other business operations.

[0124] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The symbol is represented by a single double-headed arrow, but this does not mean that there is only one bus or one type of bus.

[0125] Memory is used to store programs. Specifically, programs may include program code, which includes computer operation instructions. Memory may include main memory and non-volatile memory, and provides instructions and data to the processor.

[0126] The processor reads the corresponding computer program from non-volatile memory into main memory and then executes it, forming a 3D object at the logical level. The processor executes the program stored in memory and specifically performs the following operations:

[0127] When the terminal displays the screen content in stereoscopic form, the visual depth of the human eye on the screen content is determined. The visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen.

[0128] Detect the first position of the user's finger in front of the screen;

[0129] When the first position is within a first distance range of the screen, the operation is performed on the 3D object within the first depth of field; when the first position is within a second distance range of the screen, the operation is performed on the 3D object within the second depth of field.

[0130] The above is as stated in this application. Figure 8 The method executed by the 3D object manipulation device disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can reside in a mature storage medium in the field, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0131] The electronic device can also perform Figure 2 The method, and implement the drive access device in Figure 2 The functions described in the illustrated embodiments will not be repeated here.

[0132] Of course, in addition to software implementation, the electronic device of this application does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0133] This application also proposes a computer-readable storage medium that stores one or more programs, the programs including instructions that, when executed by a portable electronic device including multiple applications, enable the portable electronic device to perform... Figure 2 The method of the illustrated embodiment is specifically used to perform the following operations:

[0134] When the terminal displays the screen content in stereoscopic form, the visual depth of the human eye on the screen content is determined. The visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen.

[0135] Detect the first position of the user's finger in front of the screen;

[0136] When the first position is within a first distance range of the screen, the operation is performed on the 3D object within the first depth of field; when the first position is within a second distance range of the screen, the operation is performed on the 3D object within the second depth of field.

[0137] Figure 9 This is a schematic diagram of the structure of a 3D object manipulation device 90 according to an embodiment of this application. Please refer to... Figure 9 In one software implementation, the manipulation device 90 for the 3D object may include: a determining module 91, a detecting module 92, and an operating module 93, wherein:

[0138] The determination module 91 determines the visual depth of the screen content to the human eye when the terminal displays the screen content in stereoscopic form. The visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen.

[0139] Detection module 92 detects the first position of the user's finger in front of the screen;

[0140] The operation module 93 operates on 3D objects within the first depth of field when the first position is within a first distance range of the screen, and operates on 3D objects within the second depth of field when the first position is within a second distance range of the screen.

[0141] Optionally, in some embodiments, the first distance range and the second distance range satisfy at least one of the following:

[0142] The first distance range and the second distance range do not overlap;

[0143] The first distance range is closer to the human eye, and the second distance range is closer to the screen;

[0144] The sum of the first distance range and the second distance range equals the first depth of field.

[0145] Optionally, in some embodiments, the operation module 93, when the first position is within a first distance range of the screen, operates on the 3D object within the first depth of field, including:

[0146] When the first position is within a first distance range of the screen, a second position corresponding to the first position within the first depth of field is determined based on a first position mapping relationship between the first distance range and the first depth of field.

[0147] Perform operations on the 3D object within the first depth of field that corresponds to the second position.

[0148] Optionally, in some embodiments, the operation module 93, when the first position is within a second distance range of the screen, operates on the 3D object within the second depth of field, including:

[0149] When the first position is within a second distance range of the screen, a third position corresponding to the first position within the second depth of field is determined based on the second position mapping relationship between the second distance range and the second depth of field.

[0150] Perform operations on the 3D object within the second depth of field that corresponds to the third position.

[0151] Optionally, in some embodiments, the first position mapping relationship includes a first mapping relationship, a second mapping relationship, and a third mapping relationship;

[0152] Wherein, the first mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the first direction, the second mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the second direction, and the third mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the third direction, wherein the first direction, the second direction and the third direction are mutually perpendicular.

[0153] Optionally, in some embodiments, the first position includes a first coordinate in the first direction, a second coordinate in the second direction, and a third coordinate in the third direction; the operation module 93 determines a second position corresponding to the first position within the first depth of field, including:

[0154] Determine the fourth coordinate corresponding to the first coordinate based on the first mapping relationship;

[0155] Determine the fifth coordinate corresponding to the second coordinate based on the second mapping relationship;

[0156] Determine the sixth coordinate corresponding to the third coordinate based on the third mapping relationship;

[0157] The position within the first depth of field that corresponds to the fourth, fifth, and sixth coordinates is determined as the second position.

[0158] Optionally, in some embodiments, the 3D object corresponding to the second position includes at least one of a first object and a second object;

[0159] Wherein, the position of the first object within the first depth of field includes the second position;

[0160] The distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance.

[0161] Optionally, in some embodiments, the operation module 93 further includes at least one of the following:

[0162] Highlight the 3D object;

[0163] Perform a shaking operation on the 3D object;

[0164] Change the display color of the 3D object.

[0165] The 3D object manipulation device 90 provided in this application can also perform... Figure 2 The method, and the manipulation device 90 for 3D objects in Figure 2 The functions of the embodiments shown will not be described again in this application.

[0166] In summary, the above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0167] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0168] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0169] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0170] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

Claims

1. A method for manipulating a 3D object, comprising: When the terminal displays the screen content in stereoscopic form, the visual depth of the human eye on the screen content is determined. The visual depth includes a first depth of field and a second depth of field, wherein the first depth of field is located in front of the screen and the second depth of field is located behind the screen. Detect the first position of the user's finger in front of the screen; When the first position is within a first distance range of the screen, the operation is performed on the 3D object within the first depth of field; when the first position is within a second distance range of the screen, the operation is performed on the 3D object within the second depth of field.

2. The method of claim 1, comprising at least one of the following: The first distance range and the second distance range do not overlap; The first distance range is closer to the human eye, and the second distance range is closer to the screen; The sum of the first distance range and the second distance range equals the first depth of field.

3. The method of claim 1, wherein operating on a 3D object within the first depth of field when the first position is within a first distance range of the screen comprises: When the first position is within a first distance range of the screen, a second position corresponding to the first position within the first depth of field is determined based on a first position mapping relationship between the first distance range and the first depth of field. Perform operations on the 3D object within the first depth of field that corresponds to the second position.

4. The method of claim 1, wherein operating on a 3D object within the second depth of field when the first position is within a second distance range of the screen comprises: When the first position is within a second distance range of the screen, a third position corresponding to the first position within the second depth of field is determined based on the second position mapping relationship between the second distance range and the second depth of field. Perform operations on the 3D object within the second depth of field that corresponds to the third position.

5. The method as described in claim 3, wherein the first position mapping relationship includes a first mapping relationship, a second mapping relationship, and a third mapping relationship; in, The first mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the first direction; the second mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the second direction; and the third mapping relationship is the positional mapping relationship between the first distance range and the first depth of field in the third direction. The first direction, the second direction, and the third direction are perpendicular to each other.

6. The method of claim 5, wherein the first position includes a first coordinate in the first direction, a second coordinate in the second direction, and a third coordinate in the third direction; Determining the second position within the first depth of field corresponding to the first position includes: Determine the fourth coordinate corresponding to the first coordinate based on the first mapping relationship; Determine the fifth coordinate corresponding to the second coordinate based on the second mapping relationship; Determine the sixth coordinate corresponding to the third coordinate based on the third mapping relationship; The position within the first depth of field that corresponds to the fourth, fifth, and sixth coordinates is determined as the second position.

7. The method of claim 3, wherein the 3D object corresponding to the second position includes at least one of a first object and a second object; in, The position of the first object within the first depth of field includes the second position; The distance between the position of the second object within the first depth of field and the second position is less than or equal to a preset distance.

8. The method of claim 1, further comprising at least one of the following: Highlight the 3D object; Perform a shaking operation on the 3D object; Change the display color of the 3D object.

9. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the method as claimed in any one of claims 1 to 8.

Images