3D interface with improved object selection

Abstract

This invention relates to a field user interface. Specifically, it relates to an interface that displays 3D objects, receives user selections of objects, and performs the selection using a comparison between object size and a threshold, so that the selected object aligns with the user's intent.

Description

Technical Field

[0001] This invention relates to the field of user interfaces. More specifically, this invention relates to the selection of 3D objects in a 3D user interface. Background Technology

[0002] 3D interfaces are now used in a wide range of applications. In essence, 3D interfaces allow users to interact with 3D objects from different perspectives. 3D interfaces are adaptable to a wide range of contexts. For example, 3D modeling can be performed locally on the computer the user interacts with or remotely on a server, with the user interacting with images sent by the server. In the latter case, the 3D interface and corresponding tools can be provided by SaaS (Software as a Service).

[0003] One application permitted by 3D interfaces is CAD (Computer-Aided Design), which involves using computers to design 3D products. CAD applications allow the design of virtually any product. Many CAD applications exist. For example, the CATIA editor allows users to model equipment using multiple 3D elements. CAD applications such as CATIA not only allow designing the shape of a product but also allow performing multiple tests and simulations. They also allow generating multiple different product views (e.g., generating 3D documents).

[0004] In CAD applications, as in other applications that rely on a 3D interface, multiple 3D objects are displayed to the user, who can select one or more objects they wish to interact with. In CAD applications, this allows users to move objects / parts to organize a 3D scene, for example. To do this, the user must use standard selection tools to select one or more parts, such as single selection (including selecting a single object, e.g., using a click or tap), multi-selection (including selecting multiple objects at the same time, e.g., using multiple clicks or taps), or "selection traps" (including defining a shape (i.e., rectangle, polygon, freeform, etc.) under which all objects are selected). Once one or more objects are selected, the user can modify these objects using any transformation tools provided by the 3D interface.

[0005] In existing technology systems, object selection is directly mapped to the location where the selection is performed. For example, if a user clicks, the 3D object displayed at the click location is selected. However, the subsequent selection of a 3D object does not necessarily correspond to the user's intention. This is particularly true for complex scenes with many 3D objects of different sizes: when a user clicks on a scene, the selected object could be a very small object, precisely placed at the click location, yet invisible to the user. No existing technology system currently ensures that the selected 3D object corresponds to the user's intention. Figure 1 To.

[0006] Therefore, a 3D interface is needed that allows users to view and select 3D objects while ensuring that the selection is consistent with the selection the user may intend to make. Summary of the Invention

[0007] To this end, the present invention discloses a computer-implemented method comprising the following steps: displaying a set of 3D objects on at least one display; receiving from at least one input interface a user's selection of at least one first 3D object belonging to the set; calculating the 3D projection of the selection on the display; obtaining a result of comparing at least one size of the 3D projection with at least one threshold; if the result is negative, adding at least one second 3D object to the selection or removing it from the selection; and modifying the appearance parameters of the selection.

[0008] Advantageously, the at least one second 3D object is determined based on the at least one threshold.

[0009] Advantageously, while the result of the comparison is negative, the computer-implemented method further includes multiple iterations of the following steps: adding at least one second 3D object to the selection or removing it from the selection; calculating the 3D projection of the selection on the display; and obtaining the result of the comparison of the at least one size of the 3D projection with the at least one threshold.

[0010] Advantageously, adding at least one second 3D object to the selection or removing it from the selection includes: adding at least one second 3D object near the 3D projection to the selection, or removing at least one second 3D object at the edge of the 3D projection from the selection.

[0011] Advantageously, the set of 3D objects is organized as a tree, and: the selection is defined as a node of the tree and includes the child nodes of that node; adding at least one second 3D object to the selection includes defining the parent node of that node as the selection; removing at least one second 3D object from the selection includes defining the child node of the node that includes the first 3D object as the selection.

[0012] Advantageously, the at least one dimension is the maximum distance between the pixels of the 3D projection.

[0013] Advantageously, wherein the at least one dimension is the surface of the 3D projection.

[0014] Advantageously, the threshold is defined from the user's perspective.

[0015] Advantageously, the threshold is defined based on the precision of the user's selection.

[0016] The present invention also discloses a computer program product stored on a non-volatile computer-readable data storage medium, the computer program product comprising computer-executable instructions for causing a computer system to perform a computer-implemented method according to an embodiment of the present invention.

[0017] The present invention also discloses a non-transitory computer-readable data storage medium comprising computer-executable instructions for causing a computer system to perform methods according to embodiments of the present invention.

[0018] The present invention also discloses a computer system configured to implement a method according to an embodiment of the present invention, the computer system comprising: at least one display; at least one input interface; and at least one computing device configured to perform the method according to an embodiment of the present invention. Attached Figure Description

[0019] The invention will be better understood from the following description and accompanying drawings, which are provided for illustrative purposes only, and various features and advantages of the invention will become apparent, as illustrated in the drawings:

[0020] - Figure 1 Examples of 3D scenes viewed at two different zoom levels in the prior art, as well as the selection of 3D objects, are shown;

[0021] - Figure 2 Examples of the functional architecture of a device in several embodiments of the present invention are shown;

[0022] - Figure 3a and Figure 3b Two examples of computer-implemented methods in various embodiments of the present invention are shown;

[0023] - Figure 4 Three examples are shown of selecting 3D objects at three different scaling ratios in the same 3D scene;

[0024] - Figure 5An example of the selection of 3D parts in a gear is shown, which is modeled as a 3D object organized in a hierarchical manner. Detailed Implementation

[0025] Figure 1 Examples of 3D scenes viewed at two different zoom levels in the prior art, as well as the selection of 3D objects, are shown.

[0026] Image 100 illustrates a 3D scene representing a car. The 3D scene can be edited by a CAD application such as the CATIA editor or by any other 3D application in which the user can interact with 3D objects. The scene consists of multiple 3D objects, each representing a part of the car. For example, each wheel is represented by a 3D object. The user can click on different parts of the car to select the corresponding 3D object. The scene can be associated with one or more editable viewpoints.

[0027] Image 110 illustrates the same scenario, where the user has zoomed out to obtain a "bird's-eye view" of the car. At this zoom level, the user can still click on car parts to select the corresponding 3D objects. For example, when the user clicks on wheel 111, the wheel is selected and highlighted. However, this is actually meaningless because the user cannot truly distinguish individual parts at this zoom level. At this zoom level, the user is more likely to try to select the car as a whole, rather than just wheel 111.

[0028] However, existing systems use the same selection rules regardless of scaling and the apparent size of the 3D object. Therefore, existing systems cannot perform selections that might satisfy the user's intent.

[0029] Figure 2 Examples of the functional architecture of the device in several embodiments of the present invention are shown.

[0030] Device 200 can be any kind of device with computing capabilities. For example, device 200 can be a computer, smartphone or tablet, IoT device, server, etc.

[0031] The device includes access to at least one display 210. Display 210 can be any kind of display that allows a user to view images from the device 200. For example, display 210 can be any screen (e.g., glasses, a touchscreen, a watch, or a video projector). Display 210 can be embedded within the same housing as the device 200. This is the case, for example, if display 210 is a smartphone or tablet screen (whether touch-sensitive or not). The display can also be embedded in a housing separate from the device 200, and the device 200 is connected to display 210. This is the case, for example, if the display is a separate screen or a video projector. The device 200 can be connected via any kind of connection (e.g., via wired or wireless connection (e.g., Bluetooth)). TM Wi-Fi TM Or any kind of wireless connection that allows device 200 to send images to display 210 can be associated with display 210. For simplicity, in the remainder of this disclosure, at least one display 210 will be described as "the display". However, according to various embodiments of the invention, more than one display may be used. For example, a computing device with multiple screens may be used, and a collection of objects may be displayed on multiple screens.

[0032] A monitor can also be a combination of basic displays. For example, a monitor can include two screens side by side, and the user can use both screens seamlessly.

[0033] The device 200 also includes at least one input interface 220 to receive commands from a user to select one or more 3D objects. The at least one input interface 220 can be any kind of interface that allows the user to input commands, such as a mouse, button, key, keyboard, eye contact, etc.

[0034] although Figure 1 At least one input interface 220 is shown as a connection to a mouse and keyboard in a separate housing; however, according to various embodiments of the invention, at least one input interface may be a connection to an external device, a part of the device, or a connection to a display 210. For example, at least one input interface may include a wired or wireless connection to a mouse, a wired or wireless connection to a keyboard, a connection to an external haptic screen (which may also serve as the display 210 in this case), a touchpad or keyboard embedded in the housing of the device 200, a microphone for receiving voice commands, or any other suitable type of input interface.

[0035] Device 200 further includes access to one or more memories 230 storing a collection of 3D objects. The one or more memories 230 can be any kind of internal, external, volatile, or non-volatile memory. For example, the one or more memories 230 can be formed from: internal or external hard disk drives, memory in the cloud accessible by device 200, memory shared among multiple devices, memory on a server, CD-ROMs or combinations thereof, or more generally, any suitable memory or combination thereof.

[0036] One or more memories 230 store a collection of 3D objects. The collection of 3D objects may, for example, represent parts of a product designed in a CAD application. This invention is generally applicable to any collection of 3D objects in any 3D application, wherein the 3D objects are user-selectable.

[0037] According to various embodiments of the invention, for example, if the computing device is a workstation, and the display 210 and input interface 220 are the workstation's display and input interface, then the display 210, input interface 220, and computing device 200 may be located in the same location. Alternatively, if the invention is implemented within a Software as a Service (SaaS) application, the display 210, input interface 220, and computing device 200 may be located in different locations. In this case, the computing device 200 may be a server connected to a user device that displays and receives input via a remote client such as a web browser.

[0038] As will be described in more detail below, device 200 allows users to select one or more 3D objects from a set, to which further processing can be applied.

[0039] As already discussed, one object of the present invention is to ensure that the selection aligns with the user's intentions. Figure 1 To.

[0040] For this purpose, device 200 includes processing logic 240, which is configured to perform steps of a computer-implemented method according to an embodiment of the present invention, as defined below.

[0041] According to various embodiments of the present invention, the processing logic may be a processor that operates according to software instructions, a hardware configuration of the processor, or a combination thereof. It should be understood that any or all of the functions discussed herein may be implemented in pure hardware and / or by a processor operating according to software instructions and / or a configuration of a machine learning engine or neural network. The processing logic may also be a multi-core processor, a series of processors, or a combination thereof that performs operations in parallel. It should also be understood that any or all software instructions may be stored in a non-transitory computer-readable medium. The term "configuration of processing logic" refers to any unit (e.g., hardware configuration, software instructions, machine learning, training or neural network, or any other adaptation unit or combination thereof) suitable for the processing logic to perform operations.

[0042] Figure 3a and Figure 3b Two examples of computer-implemented methods in various embodiments of the present invention are shown.

[0043] Figure 3a A first example of a computer-implemented method according to several embodiments of the present invention is shown.

[0044] The computer implementation method 300a is feasible on a large number of computing devices. For simplicity, it will be combined with... Figure 2 The device 200 shown in the diagram explains the method. Method 300a is intended to allow a user to select one or more 3D objects, and to make the selection obvious to the user.

[0045] To this end, method 300a includes a first step 310: displaying a collection of 3D objects on display 210. As already discussed, the collection of 3D objects can represent all or part of a 3D scene and can be objects from any kind of 3D application (e.g., 3D products in a CAD application). This allows the user to view the collection of 3D objects.

[0046] Method 300a further includes a second step 320: receiving from at least one input interface 220 a user's selection of at least one first 3D object belonging to the set.

[0047] This selection can be performed using any known method. For example, a user can click on one of the objects, or draw a rectangular shape containing multiple objects. In the latter case, all objects within a bounding box located within the rectangular shape can be selected.

[0048] Method 300a further includes a third step 330: calculating the selected 3D projection on display 210.

[0049] Calculating a 3D projection involves determining what the 2D shape of the 3D object would be if projected onto the display 210 in 2D form. The 3D projection can depend at least on the object's size, shape, and position, as well as the scaling factor. It can also depend on the resolution and size of the display 210. Any known 3D projection method can be used to achieve the 3D projection.

[0050] The output of a 3D projection is typically a 2D shape, which can be defined as a set of pixels on which a selected 3D object will be projected.

[0051] According to various embodiments of the present invention, at least one dimension of a 3D projection can be calculated.

[0052] The size of a 3D projection can be a distance, for example, expressed in pixels, inches, or centimeters. This distance can be, for example, the height, width, diameter, maximum extension, or maximum distance between pixels belonging to the 3D projection.

[0053] Using the maximum distance between pixels belonging to the 3D projection as the size provides the advantage of well defining the 1D extension of an object, and thus provides good insight into what will be the importance of an object on the screen.

[0054] The size of a 3D projection can also be expressed as, for example, the surface of a 3D projection in pixels, square centimeters, or square inches.

[0055] Using its surface as the size for 3D projection allows for the acquisition of information about the amount of data, which the user can view and customize.

[0056] According to various embodiments of the present invention, single or multiple dimensions can be used. Dimensions can also be combined within a composite dimension. For example, a composite dimension can be obtained by combining the height and width of a 3D projection, or by combining the height, width, and surface of a 3D projection.

[0057] Once at least one dimension of the 3D projection is obtained, it can be compared with at least one threshold, and method 300a includes a fourth step 340 to obtain the result of the comparison.

[0058] Different comparisons can be used. For example, a comparison can include verifying whether a size is less than, equal to, or greater than a threshold. A comparison can also be formed by multiple comparisons. For example, if the surface of the 3D projection is greater than or equal to the minimum surface and less than or equal to the maximum surface, the result of the comparison can be positive. A comparison may also involve multiple different dimensions. For example, if the height of the 3D projection is greater than or equal to the minimum height and the width of the 3D projection is greater than or equal to the minimum width, the comparison can be positive.

[0059] Therefore, according to various embodiments of the invention, a comparison allows determining whether at least one size of the selected 3D projection matches a desired value, and thus corresponds to the selection the user intends to perform. In other words, this allows determining whether the selected object is large enough and / or small enough to correspond to the desired selection. Of course, at least one threshold can be adapted to the user's needs.

[0060] According to various embodiments of the present invention, at least one threshold can be defined in many different ways.

[0061] In several embodiments of the present invention, at least one threshold is defined based on the user's viewpoint.

[0062] This allows us to ensure that users benefit from their choices, regardless of the monitor's size, and that they have a relevant viewing angle for that choice. The viewing angle can indeed be mapped to the monitor to obtain thresholds, for example, in pixels, centimeters, or inches. It can be determined, for example, based on the monitor's resolution and the distance between the user and the monitor.

[0063] In several embodiments of the present invention, at least one threshold is defined based on the precision of the user's selection.

[0064] This allows us to ensure that the final selected size matches the size that the user is actually able to select.

[0065] The chosen size can therefore be adapted to, for example, a pointing method: if the input interface is a tactile screen, a lower threshold can be used for precise selection if a precise selection method such as pointing with a stiletto is used, while a higher threshold can be used for imprecise selection if an imprecise selection method such as pressing a finger on a tactile screen is used.

[0066] The comparison of 3D projection with at least one size thus allows ensuring that the selection corresponds to the user's possible intent, such as the appropriate set of objects to see and / or the set of objects that matches the precision of the user's selection.

[0067] At step 350, method 300a includes determining whether the result is positive or negative.

[0068] If the result is negative, method 300a includes an additional step 360: adding at least one second 3D object to the selection or removing it from the selection.

[0069] This step may include: adding at least one second 3D object to the selection if the projection is too small, for example, if at least one size is less than at least one threshold, and / or removing the 3D object from the selection if the projection is too large, for example, if at least one size is greater than at least one threshold.

[0070] This allows us to ensure that the selection aligns with the user's intended intent, regardless of whether it is modified or not.

[0071] In several embodiments of the invention, at least one second 3D object is added to or removed from the selection based on the at least one threshold.

[0072] This allows for the selection of at least one second 3D object specifically to match the threshold, and thus ensures that the final selection contains only what is needed to match the threshold.

[0073] This can be done in different ways. For example, a circle whose radius corresponds to a threshold of surface or distance can be calculated near the center of the user's selection, and all 3D objects whose 3D projection falls on a circle that does not belong to the selection can be added to the selection.

[0074] Method 300a includes the final step 370 of modifying the selected appearance parameters.

[0075] This allows users to view the objects that have been selected to ensure that the selection is correct, as well as to modify the selection and / or know the objects on which they will perform further actions.

[0076] Modifications to the selected appearance parameters can include any changes to the appearance parameters that allow the user to distinguish the selected object from other objects: highlighting, color modification, etc.

[0077] Therefore, method 300a allows ensuring that the final selection presented to the user (whether or not it is modified) will correspond to the selection that may be intended to be performed by the user.

[0078] Figure 3b A second example of a computer-implemented method according to several embodiments of the present invention is shown.

[0079] Method 300b is very similar to method 300a, and further includes steps 310, 320, 330, 340, 350, 360 and 370.

[0080] Furthermore, at the output of step 370, new iterations of steps 330, 340, and 350 are performed. Therefore, until the comparison result is positive at steps 340 and 350, i.e., until at least one dimension of the 3D projection matches at least one threshold, an object is removed or added at step 360, the 3D projection is recalculated at step 330, and at least one dimension of the projection is compared with at least one threshold at step 340. As many iterations as needed can be performed until the selected 3D projection allows a positive result at steps 340 and 350.

[0081] This iterative operation of adding or removing objects from a selection allows for the addition or removal of only what is needed to achieve the selection that the user might intend to make.

[0082] Adding or removing objects from a selection can be done in different ways. For example, a reference can be applied. Figure 3a The embodiments discussed.

[0083] In several embodiments of the present invention, adding at least one second object to the selection includes adding at least one second 3D object near the 3D projection to the selection, and removing at least one second 3D object from the selection includes removing at least one second 3D object at the edge of the 3D projection from the selection.

[0084] Therefore, if a 3D object needs to be added (e.g., if the size of the 3D projection is less than the minimum threshold), objects near the selection are added; conversely, if a 3D object needs to be removed (e.g., if the size of the 3D projection is greater than the maximum threshold), objects at the edges of the selection are removed. Thus, the selection is gradually expanded or reduced until the desired size threshold is met.

[0085] The selection of objects to add or remove can also depend on the size being considered. For example, if the size is the height of the 3D projection, objects can be added vertically to or removed vertically from the selection, while if the size is the width of the 3D projection, the same operation can be performed horizontally.

[0086] In several embodiments of the invention, 3D objects are organized as a tree. For example, in CAD applications, 3D objects are often grouped hierarchically, from lower-level parts of a product to higher-level parts. In this case, the natural hierarchical relationship can be used for efficient management of selection if the following conditions are met:

[0087] - The step of adding at least one second 3D object to the selection includes defining the parent node of the node as the selection;

[0088] - The step of removing at least one second 3D object from the selection includes defining the child nodes of the node that includes the first 3D object as the selection.

[0089] Therefore, if you need to add an object (i.e., if the 3D projection is too small based on the selected size and threshold) or remove an object (i.e., if the projection is too large based on the selected size or threshold), the selection can be expanded or shrunk to a higher / lower level accordingly.

[0090] This allows selection of logical groups corresponding to 3D objects that are more likely to be selected by the user.

[0091] In some cases, such as when a child node is too small and its parent node is too large, no node corresponds to the selection of the matching criterion. In this case, iteration can be stopped to select the hierarchical level corresponding to the size with the smallest difference from the threshold. Alternatively, other rules can be used to distinguish the most relevant level in this situation. For example, the selection of a child node may be preferred if the 3D projection of the parent node cannot be fully seen on the display.

[0092] Figure 4 Three examples are shown where 3D objects are selected at three different scaling ratios in the same 3D scene.

[0093] exist Figure 4 In the image, images 410, 420, and 430 represent a 3D scene from a CAD application, which includes a Formula 1 race car assembled as 3D parts.

[0094] The only difference between images 410, 420, and 430 is the scaling used: in 410, the scaling is very low to show a “bird’s-eye view” of the car; in 420, the scaling is average, which allows the car and its major components to be identified; and in 430, high scaling is used, which reveals many details of the car’s appearance.

[0095] In each case, the user clicks the same location 411, 421, 431 on the car's wheels. However, due to the present invention, the selection is different:

[0096] - In 410, select and highlight the entire car 412;

[0097] - In 420, select the two front wheels 422, and the mechanical components between them;

[0098] - In 430, only the inner part of the wheel 432 that the user clicked was selected.

[0099] Therefore, in both cases, the selection corresponds to the logical choice made by the user based on the scaling ratio and the apparent size of the 3D object: under the user's click, it makes no sense to select only a part of the car in 410, and it also makes no sense to select the entire car in 430.

[0100] This improved option enhances the user's ability to interact with virtually any 3D application.

[0101] In this example, the 3D object representing the car is organized hierarchically in a tree of parts. In 420, this allows selection of the correct level of abstraction corresponding to logical grouping 422.

[0102] Figure 5 This shows an example of the selection of 3D components in a gear, which is modeled as a 3D object organized in a hierarchical manner.

[0103] Images 510 and 520 both represent a 3D scene from a 3D CAD application, where gears are modeled. The gears are modeled as a hierarchical tree of 3D parts.

[0104] In step 510, the user clicks on component 511, which is then selected and highlighted. In this example, component 511 corresponds to a node in the tree, and its parent node represents the entire gear. Selection of the entire gear 521 is not performed here because, at the current zoom level, the gear is too large, and its 3D projection is not even suitable for the display area.

[0105] The examples above demonstrate the ability of this invention to provide users with efficient and intuitive selection of 3D objects.

[0106] However, the examples described above are given as non-limiting embodiments of the invention. They do not limit the scope of the invention as defined by the appended claims in any way.

Claims

1. A computer-implemented method (300a, 300b), comprising the following steps: Display (310) a collection of 3D objects on at least one display (210); Receive (320) a user selection of at least one first 3D object belonging to the set from at least one input interface (220); Calculate (330) the selected 3D projection on the display; Obtain the result of a comparison (340) between at least one dimension of the 3D projection and at least one threshold; If the result is negative (350), then at least one second 3D object is added to the selection or removed from the selection (360), wherein the result is negative if the at least one size is less than a minimum threshold or greater than a maximum threshold; Modify the selected appearance parameter (370).

2. The computer-implemented method (300a) according to claim 1, wherein, The at least one second 3D object is determined based on the at least one threshold.

3. The computer-implemented method (300b) according to claim 1, wherein, while the comparison result is negative, the method further comprises multiple iterations of the following steps: Add at least one second 3D object to the selection or remove it from the selection (360); Calculate (330) the 3D projection selected on the display; The result of the comparison (340) between the at least one size of the 3D projection and the at least one threshold is obtained.

4. The computer-implemented method according to claim 3, wherein, Adding at least one second 3D object to the selection or removing it from the selection (360) includes: adding at least one second 3D object near the 3D projection to the selection, or removing at least one second 3D object at the edge of the 3D projection from the selection.

5. The computer-implemented method according to claim 3, wherein, The collection of 3D objects is organized as a tree, and wherein: The selection is defined as a node of the tree, and includes the child nodes of the node; Adding at least one second 3D object to the selection (360) includes defining the parent node of the node as the selection; Removing at least one second 3D object from the selection (360) includes defining the child nodes of the node that includes the first 3D object as the selection.

6. The computer-implemented method according to any one of claims 1 to 5, wherein, The at least one dimension is the maximum distance between the pixels of the 3D projection.

7. The computer-implemented method according to any one of claims 1 to 5, wherein, The at least one dimension is the surface of the 3D projection.

8. The computer-implemented method according to any one of claims 1 to 5, wherein, The threshold is defined from the user's perspective.

9. The computer-implemented method according to any one of claims 1 to 5, wherein, The threshold is defined based on the precision of the user's selection.

10. A computer program product stored on a non-volatile computer-readable data storage medium, comprising computer-executable instructions for causing a computer system to perform a computer-implemented method according to any one of the preceding claims.

11. A non-transitory computer-readable data storage medium comprising computer-executable instructions for causing a computer system to perform the method according to any one of claims 1 to 9.

12. A computer system configured to implement the method according to any one of claims 1 to 9, comprising: At least one display (210); At least one input interface (220); At least one computing device (200) is configured to perform the method according to any one of claims 1 to 9.

Images