Manipulating a selection of objects of a digital industrial scene

Abstract

Systems and a method for displaying a digital scene comprising a plurality of objects. Data of a first selection of a first set of objects are received. Data of a selection of a scaling rule are received. A scaling regulator enabling a user to manipulate the current scaling index within a scaling range is displayed. Via the regulator, it is received data of a manipulation of the current scaling index to obtain a second selection of a second set of objects. The second selection of objects is applied by applying the scaling rule to the received index manipulation. A subset of the determined second set of objects is displayed within the digital scene.

Description

MANIPULATING A SELECTION OF OBJECTS OF A DIGITAL INDUSTRIAL SCENETECHNICAL FIELD

[0001] The present disclosure is directed, in general, to computer-aided design, visualization, and manufacturing (“CAD”) systems, product lifecycle management (“PLM”) systems, product data management (“PDM”) systems, production environment simulation, and similar systems, that manage data for products and other items (collectively, “Product Data Management” systems or PDM systems).BACKGROUND OF THE DISCLOSURE

[0002] In industrial engineering, software applications with 3D representations of industrial scenes with industrial objects are used to optimize and manage the design, engineering and manufacturing processes. Examples of usages of industrial software application include, but are not limited by, manufacturing process design, manufacturing process simulation, production operational control, manufacturing process analysis, equipment collision checks, inspections, reviews and virtual commissioning. As used herein the term object may denote any resource (like e.g. device, machinery, equipment, tool, part, conveyor, wall, fence, table, human, room), any zone, any surface, any path, any physical element and / or non physical element - e.g. a location or an operation - of a digital industrial environment.

[0003] For the sake of explanation, the technical problem faced by industrial engineers is now explained for robotic simulation platforms although the skilled persons easily appreciate that similar considerations apply for other types of industrial software applications whereby industrial objects are represented in 3D during the entire life cycle of an industrial product.

[0004] Examples of robotic simulation platforms and systems include, but are not limited to, Computer Assisted Robotic (“CAR”) tools, Process Simulate (a product of theSiemens Group), robotic software simulations tools, software applications for industrial robotic simulation and other systems and virtual stations for industrial robotic simulation.

[0005] Robotic simulation platforms enable simulation engineers to simulate multiple robotic operations performed by multiple industrial robots on simulated scenes of shopfloors.

[0006] Figure 3A and 3B schematically illustrate examples of GUI viewers of a robotic simulation platform.

[0007] Often, on desktop applications, the Graphical User Interface (“GUI”) screen 310 of the simulated industrial scene displays several industrial objects 331 whereby dozens may be virtual robots operating simultaneously. In order to inspect and perform actions on a different groups of robots, simulation engineers make us of the filtering functionalities included within the tree navigation panels 322, 323 of the GUIs.

[0008] In fact, typically, to inspect and simulate the 3D kinematics behaviors of multiple robots, the simulation engineers need to select a certain number of robots. In order to do this, they often make use of desktop applications having GUIs comprising a plurality of viewer panels / windows e.g. a 3D digital scene viewer 321 with the industrial objects 331 and lateral viewers 322, 323, with their navigation trees 332, 333 as shown in Figure 3 A.

[0009] The tree navigation viewers 322, 323 enable the users to select industrial objects according to various filtering options such as for example hierarchies e.g. objects, operations etc. Therefore, in industrial desktop applications, users typically make use of graphical viewers with lateral panels with navigation trees 332, 333 which show the data in various filtering and hierarchy possibilities to select a desired group of objects whereby, for example, by selecting an object in the tree, the children in the hierarchy may advantageously also be selected.

[0010] Figure 3B illustrates in greater detail an example of object viewer with navigation tree 342 of an exemplary spotweld station of an industrial scene. For example,the tree includes items of parts and of resources, whereby the resources of the spotweld station includes a turntable and a robot including a gripper and robot’s entities such as e.g. base, links, tool-frame, Tool Center Point Frame (“TCPF”). For example, the user chooses the component level command from a graphical pick level menu (not shown) and then they pick a specific robot in the hierarchy of the tree e.g. robot robl is selected 351 and correspondingly in the scene viewer 321 the robot robl with its entities is selected by highlighting it in a specific color (not shown). In general, the component / entity level is mostly used for the graphic viewer when the user wants to select a specific level of the object. In general, the selection in the tree is more discrete - what the user selects is what is highlighted.

[0011] Typically, in a robotic simulation platform, to perform a given industrial operation on a certain number of robots, the user needs to select and display such group of robots in the digital scene viewer. To do this, she picks in the tree 332, 342 of the lateral viewer the object entity “robot” and then she drags a box 361 in the digital scene 321 so that all the robots included in the box 361 are marked or highlighted with a specific color (here for explanation purposes the marking is illustrated by marking the robots via dots or dashes) so that the industrial operation on the selected / highlighted / marked robots can be performed. In case the user would like to manipulate the selection - i.e. to select more or less robots, the user has then to draw a larger box or a smaller box.

[0012] In summary, in desktop applications, the user can select all robots within a dragged bounding box 361 by using the filtering option capabilities on the panels with navigation trees 332, 342. The selected robots are highlighted with a predefined color in the digital scene viewer 321. In such a manner, the user can then easily perform the given industrial operation on the highlighted robots.

[0013] Examples of industrial operations / actions to be performed on multiple objects and at various levels include, but are not limited by, inspection, playing a simulation at one or more levels, optimization, line operations controlling, virtual commissioning, objects placing; moving a group of objects (not one by one), jogging the robots;blanking / coloring / wir eframing a group of objects, testing, training, tutoring; setting notes, and other operations that need to be performed on multiple industrial objects and at different logical levels.

[0014] Without such multi-object selection manipulation capabilities, performing actions on industrial desktop applications would be a tedious task and such applications would be unusable for the industrial engineers.

[0015] However, it is noted that the current techniques for manipulating a selection of multiple objects of a digital scene of an industrial environment e.g. via browsing and via selections in navigation tree panels and / or by drawing a box with a mouse in the digital scene have been conceived for large screens and for mouse usage.

[0016] Unfortunately, such techniques may become cumbersome in modern industrial settings where users work more and more with tablets, touch screens and Extended Reality (“XR”) environments (e.g. VR, AR, Metaverse Reality etc.) and where they expect efficient capabilities such as flexible and intuitive functionalities and interactions.

[0017] Improved techniques are therefore desirable.SUMMARY OF THE DISCLOSURE

[0018] Various disclosed embodiments include methods, systems, and computer readable mediums manipulating a selection of one or more objects of a digital scene of an industrial environment. A method includes displaying a digital scene comprising a plurality of objects. The method further includes receiving data of a first selection of a first set of objects. The method further includes receiving data of a selection of a scaling rule; the rule defining the effect of a scaling index manipulation on a resulting selection of a resulting set of objects. The method further includes displaying a scaling regulator enabling a user to manipulate the current scaling index within a scaling range. The method further includes receiving, via the regulator, data of a manipulation of the current scaling index to obtain a second selection of a second set of objects. The method further includes determining the second selection of objects by applying the scaling rule to thereceived index manipulation. The method further includes displaying, within the digital scene, a subset of the determined second set of objects.

[0019] The foregoing has outlined rather broadly the features and technical advantages of the present disclosure so that those skilled in the art may better understand the detailed description that follows. Additional features and advantages of the disclosure will be described hereinafter that form the subject of the claims. Those skilled in the art will appreciate that they may readily use the conception and the specific embodiment disclosed as a basis for modifying or designing other structures for carrying out the same purposes of the present disclosure. Those skilled in the art will also realize that such equivalent constructions do not depart from the spirit and scope of the disclosure in its broadest form.

[0020] Before undertaking the DETAILED DESCRIPTION below, it may be advantageous to set forth definitions of certain words or phrases used throughout this patent document: the terms “include” and “comprise,” as well as derivatives thereof, mean inclusion without limitation; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “associated therewith,” as well as derivatives thereof, may mean to include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, be bound to or with, have, have a property of, or the like; and the term “controller” means any device, system or part thereof that controls at least one operation, whether such a device is implemented in hardware, firmware, software or some combination of at least two of the same. It should be noted that the functionality associated with any particular controller may be centralized or distributed, whether locally or remotely. Definitions for certain words and phrases are provided throughout this patent document, and those of ordinary skill in the art will understand that such definitions apply in many, if not most, instances to prior as well as future uses of such defined words and phrases. While some terms may include a wide variety of embodiments, the appended claims may expressly limit these terms to specific embodiments.BRIEF DESCRIPTION OF THE DRAWINGS

[0021] For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the descriptions taken in conjunction with the accompanying drawings, wherein like numbers designate like objects, and in which:

[0022] Figure 1 illustrates a block diagram of a data processing system in which an embodiment can be implemented.

[0023] Figure 2 schematically illustrates a flowchart for manipulating a selection of objects of a digital industrial scene in accordance with disclosed embodiments.

[0024] Figure 3 A and 3B schematically illustrate examples of GUI viewers of a robotic simulation platform.

[0025] Figure 4 schematically illustrates a flowchart of a multi-object selection algorithm in accordance with disclosed embodiments.

[0026] Figure 5 schematically illustrates a flowchart for object selection scaling with automatic prediction in accordance with disclosed embodiments.

[0027] Figure 6 illustrates an industrial scene in the VR in accordance with disclosed embodiments.

[0028] Figure 7 schematically illustrates scaling index manipulations and their effects on the selections of objects in accordance with disclosed embodiments.

[0029] Figure 8 schematically illustrates examples of scaling regulators and indicators in accordance with disclosed embodiments.

[0030] Figure 9 schematically illustrates a VR control in accordance with disclosed embodiment.

[0031] Figure 10 schematically illustrates graphical interfaces on a touchscreen in accordance with disclosed embodiment.DETAILED DESCRIPTION

[0032] FIGURES 1 through 10, discussed herein, and the various embodiments used to describe the principles of the present disclosure in this patent document are by way of illustration only and should not be construed in any way to limit the scope of the disclosure. Those skilled in the art will understand that the principles of the present disclosure may be implemented in any suitably arranged device. The numerous innovative teachings of the present application will be described with reference to exemplary non-limiting embodiments.

[0033] Previous techniques do not enable to manipulate a selection of industrial objects on tablets and XR environments in an efficient and intuitive manner.

[0034] Embodiments provide an efficient mechanism for up / down scaling the selection of objects in industrial scenes.

[0035] Embodiments provide users with the capability of changing a selection of objects without re-selecting.

[0036] Embodiments provide users with a user-friendly multi-object selection mechanism.

[0037] Embodiments enable users to intuitively select groups of objects in the VR and / or with touch screens.

[0038] Embodiments enable, in the Virtual Reality, a multi-object selection whose scope is adjustable and is not a single predefined selection scope.

[0039] Embodiments enable users to configure the selection scope in the Virtual Reality. Advantageously, in such embodiments, the object selection scope is not univocally determined by the action type and by the first object selection of the users.

[0040] Embodiments enable users to manipulate and control the selection of objects without the need of opening trees.

[0041] Embodiments enable users to control the selection of objects without the need of changing commands.

[0042] Embodiments enable users to control the selection of objects with natural gestures e.g. via one or two handsets in the VR and / or via finger gestures on touchscreens.

[0043] Embodiments provide users with a capability of selecting multiple objects with a specific context.

[0044] Embodiment enable to have a selection scope which is flexible and configurable e.g. context sensitive depending on selected object, type of industrial action, other contextual data, any combination thereof.

[0045] In embodiments, the scaling rule may be context based, predefined, determined by the system and / or selected by the user.

[0046] Embodiment provides a multi-object selection technique configured to respond to the context of the users’ actions with the capability to scale up from a single object selection while considering the objects’ context.

[0047] Embodiments enable users to perform, in the field of VR engineering capabilities, multi-object selections according to hierarchies and other considerations in efficient and user-friendly manners.

[0048] Embodiments provides the ability to widen the VR capabilities to perform engineering-oriented tasks additionally to the known traditional VR capability which include, but are not limited by, walking around facilities, seeing simulations, doing guided tours, dealing with work instructions and issuing notes regarding problems in the plant. Conveniently, embodiments enable the VR industrial space to become a complementary tool of the engineering application by achieving real modelingcapabilities, by enabling to perform industrial changes from a different perspective than the industrial changes performed in front of a 2D screen of a desktop application without the immersion capabilities of the Virtual Reality.

[0049] By extending the type of VR industrial capabilities, embodiments render the VR space more attractive for a larger number of industrial customers and for a broader range of industrial fields.

[0050] Embodiments enable users to select one or more objects and to scale such selection. Embodiments can be used in the VR space.

[0051] Embodiments enable multi-object scale selection with a “panel-free” approach.

[0052] Embodiments enable the user to choose the type of scaling rule.

[0053] Embodiments enable to automatically predict the type of scaling rule.

[0054] In embodiments, the type of scaling rule is context based.

[0055] Embodiments enable the user to efficiently select, visualize objects and perform a desired industrial action on them also in an embedded environment like the Virtual Reality or other XR environments.

[0056] Figure 1 illustrates a block diagram of a data processing system 100 in which an embodiment can be implemented, for example as a PDM system particularly configured by software or otherwise to perform the processes as described herein, and in particular as each one of a plurality of interconnected and communicating systems as described herein. The data processing system 100 illustrated can include a processor 102 connected to a level two cache / bridge 104, which is connected in turn to a local system bus 106. Local system bus 106 may be, for example, a peripheral component interconnect (PCI) architecture bus. Also connected to local system bus in the illustrated example are a main memory 108 and a graphics adapter 110. The graphics adapter 110 may be connected to display 111.

[0057] Other peripherals, such as local area network (LAN) / Wide Area Network / Wireless (e.g. WiFi) adapter 112, may also be connected to local system bus 106. Expansion bus interface 114 connects local system bus 106 to input / output (I / O) bus 116. I / O bus 116 is connected to keyboard / mouse adapter 118, disk controller 120, and I / O adapter 122. Disk controller 120 can be connected to a storage 126, which can be any suitable machine usable or machine readable storage medium, including but are not limited to nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), magnetic tape storage, and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs), and other known optical, electrical, or magnetic storage devices.

[0058] Also connected to I / O bus 116 in the example shown is audio adapter 124, to which speakers (not shown) may be connected for playing sounds. Keyboard / mouse adapter 118 provides a connection for a pointing device (not shown), such as a mouse, trackball, trackpointer, touchscreen, etc.

[0059] Those of ordinary skill in the art will appreciate that the hardware illustrated in Figure 1 may vary for particular implementations. For example, other peripheral devices, such as an optical disk drive and the like, also may be used in addition or in place of the hardware illustrated. The illustrated example is provided for the purpose of explanation only and is not meant to imply architectural limitations with respect to the present disclosure.

[0060] A data processing system in accordance with an embodiment of the present disclosure can include an operating system employing a graphical user interface. The operating system permits multiple display windows to be presented in the graphical user interface simultaneously, with each display window providing an interface to a different application or to a different instance of the same application. A cursor in the graphical user interface may be manipulated by a user through the pointing device. The position of the cursor may be changed and / or an event, such as clicking a mouse button, generated to actuate a desired response.

[0061] One of various commercial operating systems, such as a version of Microsoft Windows™, a product of Microsoft Corporation located in Redmond, Wash. May be employed if suitably modified. The operating system is modified or created in accordance with the present disclosure as described.

[0062] LAN / WAN / Wireless adapter 112 can be connected to a network 130 (not a part of data processing system 100), which can be any public or private data processing system network or combination of networks, as known to those of skill in the art, including the Internet. Data processing system 100 can communicate over network 130 with server system 140, which is also not part of data processing system 100, but can be implemented, for example, as a separate data processing system 100.

[0063] Figure 2 illustrates a flowchart for manipulating a selection of one or more objects of a digital scene of an industrial environment.

[0064] At act 205, the digital scene comprising a plurality of objects is displayed.

[0065] At act 210, the system receives data of a first selection of a first set of objects. The first selection of objects is displayed via a certain marking.

[0066] At act 215, the system receives data of a selection of a scaling rule; said rule defining the effect of a scaling index manipulation on a resulting selection of a resulting set of objects by departing from the first object set.

[0067] At act 220, the system displays a scaling regulator enabling a user to manipulate the current scaling index within a scaling range.

[0068] At act 225, the system receives via the regulator data of a manipulation or adjustment of the current scaling index to obtain a second selection of a second set of objects. For example, the user adjusts the first selection of objects by manipulating the regulator index within the scaling range.

[0069] At act 230, the second selection of objects is determined by applying the scaling rule to the received index manipulation.

[0070] At act 235, the system displays, within the digital scene, a subset of the determined second set of objects.

[0071] In embodiments, the system may display a “scaling indicator” indicating the current value of the scaling index via a separate entity than the entity of the “scaling regulator”.

[0072] In embodiments, the scaling rule may be selected from the group consisting of: scaling rules of logical type; scaling rules of hierarchical type; scaling rules of shape type; scaling rules of path type; scaling rules of material type; scaling rule with modifiable scope; scaling rules with fixed scope; other rules of different type; and, any combination of the above rules.

[0073] In embodiments, the first set of objects is computed based on a context of a picking rule.

[0074] In embodiments, acts 215 precedes act 210.

[0075] In embodiments, the environment of the scaling regulator and / or of the indicator is selected from the group consisting of: windows, mouse and / or keyboard environment; touchscreen environment; Virtual Reality environment; Augmented Reality environment; another XR environment; and / or any combination of the above environments.

[0076] In embodiments, an industrial action is performed on the second set of objects.

[0077] In embodiments, the scaling rule or the industrial action is determined via a predictor based on any of the criteria selected from the group consisting of history; previously performed action; system preferences; predefined templates; type of the selected objects; user defined criteria; context of the application; Artificial Intelligence algorithms; other criteria and techniques; any combination of the above.

[0078] In embodiments, the terms “received / receive / receiving”, as used herein, can include retrieving from storage, receiving from another device or process, receiving via an interaction with a user or otherwise.

[0079] Algorithms of exemplary embodimentsIn exemplary embodiments, the main algorithm phases and steps for manipulating a selection of objects of a digital industrial scene are illustrated below with the help of Figure 4 and Figure 5.

[0080] Figure 4 schematically illustrates data inputs and outputs for controlling a view access of an asset in accordance with disclosed embodiments.

[0081] The main phases of the multi-object selection algorithm comprise one or more of the following blocks:- scaling rule 411, first selection 412, scale 413, second selection 414, action 415;- regulator 421, manipulate 422, value 423, indicator 424,- picker 431, pick 432, picking rule 433, compute 434.

[0082] The multi-object selection algorithm comprises in its core the scaling rule algorithm 410 comprising the four blocks 411-415.

[0083] The scaling rule defines the logic with which the user can scale the selection of objects by manipulating a scaling index within a scaling range. The scaling rule defines how the objects are to be collected, the specific logic for scaling the object-selection scope e.g. by resizing / modifying the selection scope (e.g. by selecting more / less objects within a group of objects) or by moving a fixed selection scope (e.g. by selecting each time one object within a group of objects). In other words, the scaling rule defines the effect of scaling an initial selection of a set of objects by manipulating a scaling index to obtain a resulting set of objects.

[0084] In embodiments, via the scaling rule, the user can scale the object selection by making the selection of objects smaller or bigger, more detailed or less detailed, make it follow a trajectory or path (to a different object by scaling down to the previous connected object or by scaling up the next connected object; or to more objects in the same chain or path), change the same object by selecting a different section of it and other object selection types and combination thereof.

[0085] Exemplary embodiments of object selection scope include, but are not limited by, following chain by increasing the range of the chain e.g. in a weld operation from the initial selected location selecting additional locations e.g. increasing in one or both directions.

[0086] Exemplary embodiments of moving a fixed object selection scope include, but are not limited by, following chain one object each time e.g. in a weld operation going from one location to the other, back and forth in the path.

[0087] In embodiments, the scaling index within the scaling range may indicate a selection of less or more objects (scaling the size of the scope, where e.g. the minimum index points to less objects and where e.g. the maximum index points to more objects) or a selection of a different object within a path (changing the object within a chain of objects; where e,g, the minimum index is pointing to the initial object of the chain and the maximum index is pointing to the last object of the chain).

[0088] In embodiments, the scaling range and the index depend on the data of industrial environment e.g. when loading a station the depth / scale of the data might be different than when loading a line which may include several stations. In embodiments, the index - its depth l..n - may be determined from the selected object via an algorithm that computes the scaling range of the index.

[0089] In algorithm embodiments, the system receives the user’s selected index within a scaling rule range and the initial selected objects in the world to be scaled and it returns the actual selected objects according to the scaling index.

[0090] Examples of types of scaling rules include “hierarchical” ones (similar to hierarchy as illustrated in the navigation trees 322, 323), “shape / typed” ones (selecting a pyramid and continuing to scale will select more and more pyramids), “path” ones (selecting object(s) in a path, where scaling up / down increase / decrease the selection within the path or change / move an object within the path or hierarchy of objects) and other rule types and combination thereof. In short, the scaling rule type denotes some logic that defines the effect of manipulating the index within the range to increase / decrease the selection of the objects or to change a selection of an object of a specific type. It might also be a combination of several methods such as select all robots or all locations in the given path. More details and examples of scaling rule types will be provided at a later stage.

[0091] As shown in the flowchart of Figure 4, the system receives data on a scaling rule 411 and a first selection 412, whereby for example a user selects a first set of objects from the industrial scene. The user then scales 413 the first selection 412 by manipulating 422 a regulator 421 to obtain a value with one or more iterations 425. In embodiments, the manipulated index value may be displayed in an indicator 424 which may be a separate entity than the entity of the regulator 421. In embodiments, the manipulated index value is displayed on the regulator entity which acts also as indicator entity. Based on the scaling rule 411, a picker 431 is provided. The user holds the picker 431 and picks 432 an object in the industrial scene and the first selection of objects 412 is computed 434 via the picking rule 433. In embodiments, the picker 431 is used by the user to do an initial selection in the digital scene. Examples of pickers include, but are not limited to, a ray in the Virtual Reality, touch gestures on a screen, eye gestures and vocal commands and others. In embodiments, the picking rule 433 computes 434 the scope of the first selection 412 according to a specific context. In fact, each object / entity or a group of objects in the digital scene may have several properties in parallel such as e.g. type, graphical entity, weight, kinematics etc. whereby some of these may exist only at a certain level and not at higher / lower levels, and the first selection applies to a specific context. In embodiments, given a first selection picking rule, a user performs a pick 432 via picker 431 that indicates where the pick has been performed (e.g. x,y in 2D; x,y,z in 3D: 2D bounding box x,y,length.x, length.y; etc) and by using the picking rule, the scope of the first selection is computed. In other words, the picker indicates where, and according to where, the picking rule computes the first object selection.

[0092] Receiving data on the scaling rule 411 may mean that the scope is chosen by the user, it is predefined and / or it received by a process automatically determining the scaling rule 411 e.g. based on the context of the first selection 412, via Al, history of theuser etc. Receiving data of a first selection 411 may mean a user selection or a predefined stored selection etc.

[0093] The order of blocks 411, 412 is switchable, i.e. the scaling rule 411 may be received after or before the first selection 412 as shown for example in Figure 5 or in the below described two exemplary embodiments of flow sequence.

[0094] According to the first exemplary embodiment of flow sequence, the user may pre-choose his desired industrial action and whenever he performs his first object selection, the system receives the scaling rule and then the user can scale the selection of objects via this rule to perform his desired action. In summary, according to this first example, the flow sequence is <select action, first selection, scaling rule, scale, second selection, perform action>.

[0095] According to the second exemplary embodiment of flow sequence, the user may pre-choose his desired scaling rule and, after he scales his first object selection, he is provided with the available actions he can perform on the selected objects. In summary, according to this first example, the flow sequence is <scaling rule, first selection, scale, second selection, select action, perform action>.

[0096] An example embodiment of the first example of flow sequence is now illustrated:- a user selects a desired action to perform; the user picks an object in the VR e.g. via a ray mechanism or by virtually touching the object; the user opens a regulator for a received scaling rule and by <increasing / decreasing the intensity>, <more / less objects are selected> or, alternatively, <the previous / next object of a chain or of a path is selected>;- the newly determined object selection is displayed to the user via different techniques;- the action is performed.

[0097] Embodiments, the scaling rule may be received based on the context - e.g. the behavior of the scaling rule may be sensitive to the type of object of the first selection; itmay be sensitive to the desired action; it may be sensitive to other contexts or combinations of these and other ways to define the scaling rule behavior.

[0098] In embodiments, the selected objects are displayed in the digital scene via different marking techniques. Examples of objects marking and / or highlighting techniques include, but are not limited by, coloring, transparencing, highlighting, outlining, enlarging / decr easing the selection (making it bigger or smaller), making it shine (e.g. like a star), darkening / hiding the non-selected surroundings, animating the selection (jumping up and down), textual, via a screen showing the object selection, other techniques, including a combination thereof.

[0099] An example embodiment of the second example of flow sequence is now illustrated:- a user selects the scaling rule from a pool of rule types;- the users picks an object in the VR;- the user opens a regulator for the selected scaling rule and by <increasing / decreasing the intensity>, <more / less objects are selected> or, alternatively, <the previous / next object of a chain or of a path is selected>;- the newly determined object selection is displayed to the user via different techniques;- the user is provided with a choice of available actions to perform based on the scaling rule and the object selection and selects a desired action;- the action is performed.

[0100] In embodiments, the action is performed automatically on the second selection of objects.

[0101] In embodiments, exemplary algorithm embodiments include receiving a first selection 412 whereby a user selects a set of objects, receiving a scaling rule 411 defining how the user can scale the object selection with a context, and receiving a scaling index manipulation 413 whereby the user is changing the object selection by manipulating the regulator scaling index dynamically to reduce / increase / change the selection of objects.

[0102] The value of the scaling index of the regulator is included within a selection scope range, for example assume 0 to 100, or assume another range for example the actual steps like e.g. the depth 0 to 7 of the hierarchy of the object tree 342 of Figure 3.

[0103] In embodiments, the regulator is an interface configured to be manipulated by a user to receive as input the current adjusted index value representing the desired object selection scope. The system returns as output a resulting set of objects by applying the current adjusted index value to the initial object selection based on the logic of the scaling rule.

[0104] The following examples describe possible types of scaling rules. The scaling rules can be of various different types, including, but not limited to, the following examples: logical, hierarchy, kinematics, shape, material, other scaling rule types, any combination thereof. The scaling rule types define how the selection of objects is scaled, departing from the first selection to arrive to the resulting selection of objects.

[0105] The “logical” type may denote a scaling rule type of any possible logic, for example hierarchy (e.g. object hierarchy or operation hierarchy), kinematics, shape, material, other types, including combination of several types.

[0106] The “hierarchy” type may denote a scaling rule type with “operationhierarchy” or “object-hierarchy” scope and the choice between the two depend on the context of the objects selected by the user. For example, in embodiments, it is not necessary for the users to view graphical panels with object trees and / or operation trees, the system is able to determine the type of hierarchy (operation or object), by receiving the user selection, for example, if the user select a location or a path - which are visible in the industrial scene - then the system automatically determines that the user wishes an operation hierarchy or, if the user selects a robot or a part, the system automatically routes to the object hierarchy type.

[0107] The “kinematics” type may denote a scaling rule type where the scope is the kinematics tree of a robot or of another type of kinematic device. For example, the links are the physical link entities of the robot Linkl, Link2, Link3, etc and then there are thejoints which typically connect two links, jl, j2, j3, ... , e.g. one link can connect two other links via two joints, there may be also loops. For example, assume there is robot rA having six links Linkl, Link2, Link3, Link4, Link5, Link6 where these six links are mutually connected via five joints jl, 2, j3, j4, j5 as follows: Linkl ->jl -> Link2 ->j2 -> Link3 ->j3 -> Link4 ->j4 -> Link5 ->j5 ->Link6. Assume that a user wants to perform an action of joint manipulation, he first select one single link Link5 and manipulates its joint j5 and then he may wish to scaling up (increasing) his first selection to manipulate a larger chain which additionally includes the adjacent joints j3, j4 (second selection obtained by scaling), and then he may wish to further scale up to additionally include the other adjacent joins j2, j4 within the selected chain (third selection of objects). In embodiments, on equipment, there is a hierarchy of component with kinematics and each level can have a link containing the kinematics below, by selecting a link at a certain point, one can select the parent component link and manipulate their joints.

[0108] The “shape” or “typed” type may denote a scaling rule type where scope can be enlarged or reduced to include more and less objects of the same shape or object of the same type. For example, the user selects one robot, and by scaling up or down, more or less robots in the proximity of this selected robot are selected.

[0109] The “material” type may denote a scaling rule type where scope can be enlarged or reduced to include more and less objects of the same material. For example, the user selects a piece of wood, and by scaling up or down, more or less pieces of wood around the selected wood piece are selected.

[0110] The following examples describe possible actions or industrial operations that can be performed on the scaled selection of objects.

[0111] Assume that a virtual industrial scene is created for engineering purposes and it includes a production cell, a production line or a full production plant including robots, other industrial objects and all aspects of the simulation, collision, kinematics and more. Assume exemplary use cases where the user is navigating this industrial scene in the VR and they wish, for engineering purposes, to perform actions like playing the simulation, jogging the robot, jogging a joint, placing or moving a group of industrial objects,blanking / coloring / wireframing selected objects and / or setting a note on selected objects by manipulating the objects selection via scaling in accordance with disclosed embodiments.

[0112] As used herein the term play may indicate playing the simulation and / or playing any desired industrial action. For example, playing may include simulating, blanking, creating a new seam on a selected path after having selected the objects and their surfaces to paint, e.g. creating a mesh on that selected objects or creating a paint path matching the objects surfaces etc.

[0113] According to a use case example, the user’s action consists in playing the simulation. A scale indicator reflecting the regulator’s manipulations is opened in the VR; wherein tuning down selects a seam (it is a gluing or laser continuous operation) this after the location of that operation and wherein scaling up selects the compound operation containing that operation, thein its containing operation and so on until the top operation. When the user plays the simulation, the simulation plays the operation, the seam, the seam until the location, the compound operation, its compound operation and so on respectively.

[0114] According to another use case example, the user’s action consists in jogging a joint. A scale indicator regulable via a regulator is opened in the VR. In case the user selects a robot, scaling down selects a specific link of the robot; scaling up selects additionally the external axis of the robot; when the user activates the jog action, a manipulator opens and when the user moves the manipulator, the entire selection of one or more objects is kinetically moved. In case the user selects an equipment, scaling down selects a child equipment or device till the level of the leaf’s kinematics; scaling up additionally selects the external axis of the robot; when the user activates the jog action, a manipulator opens and when, the user moves the manipulator the selected level moves accordingly.

[0115] According to another use case example, the user’s action consists in performing a placement. - i.e. to move an object or a group of objects to a new location,the user may first select the type of selection and then they are enabled to increase / decrease the scale via a regulator and to move all selected objects accordingly.

[0116] There are numerous embodiments of industrial actions which can conveniently be performed on the selected objects and they include but are not limited to the following examples.

[0117] Assume the user selects multiple simulative objects with similar kinematics (e.g. two robots in a station or all robots of the station) and via the manipulator she dances them together - the objects align and do the same movement as ordered by the manipulator.

[0118] Assume the user’s industrial action may consist in moving a robot from one station to another - to enable load balancing then she needs to select the robot or its entire ecosystem (controller, tools, cables whereby the flexible cables may be simulated for collision / stress check).

[0119] Assume the user may change the position of an entire station or of an entire line.

[0120] Assume the user mirrors operations of a station / line e.g. when a part is symmetrical and same work is done for both sides or to duplicate the station / line. Examples of mirroring or duplication applications include, but are not limited by, a new car variant, to speed up a slow process in a station / line by creating parallel stations / lines that reduce workflow time. For example, when mirroring, the user can select a location and just mirror it for the robot new robot -> user can mirror an entire operation for the same robot, as a new robot -> for the same station, for a new station, or for an existing station -> line ... etc. Assume the industrial action consists in automatic weld distribution (with balancing) on one robot, on all station robots or on the entire line. Assume the industrial action consists in running a simulation on a robot -> a station -> a line.

[0121] In exemplary embodiments, if the user selects a scaling rule of “hierarchy” type, she can move the selected hierarchy, she can color the selection, she can blank ordisplay it. However, in embodiments, it might be that in order to perform a specific industrial action 415, scaling rules of different types are preferably applied, for example the user may wish to move her selection, but she wants to scale by proximity, by hierarchy, by object type e.g. move all object in her proximity, move all robot in her proximity (combination), select a robot and go up the hierarchy and its tooling then the robots controller then the station then the line - she may want to move all object of the robot connected to the controller, or the station or the line. Additionally, in embodiments, scaling rules for the objects and industrial actions on the objects may be often coupled together to define an industrial scaling technique or “scaling method” which includes both the scaling rule and the industrial action to be performed on the selected objects.

[0122] Figure 5 schematically illustrates a flowchart for object selection scaling with automatic prediction in accordance with disclosed embodiments. The flowchart of Figure 5 differs from the one in Figure 4 in that the “first selection” 511 precedes the “scaling rule” in the flow 510 and in that there is an automatic predictor 535 after picking 432.

[0123] As shown in Figure 4, there the user makes a first object(s) selection and then she can scale it whereby, although the system can automatically determine the context, the scaling rule and the action are still decided by the user without the automatic prediction of Figure 5.

[0124] As shown in Figure 5, the main phases of the algorithm including automatic prediction comprises the following blocks:- first selection 511, scaling rule 512, scale 413, second selection 414, action 415;- regulator 421, manipulate 422, value 423, indicator 424,- picker 431, pick 432, automatic predict 535, picking rule 433, compute 434.

[0125] The system receives data on a first object selection 511 and a predicted scaling rule 512. The user then scales 413 the first selection 412 by manipulating 422 a regulator 421 to obtain a value with one or more iterations 425. In embodiments, the manipulated value may be displayed in an indicator 424 which may be a separate entity than the entity of the regulator 421. In embodiments, the manipulated value is displayed on the regulatorentity which acts also as indicator entity. After the user picks 432 with a picker 431, an automatic predictor 535 automatically determines the picking rule, the scaling rule and / or the action. The user holds the picker 431 and picks 432 an object in the industrial scene and the first selection of objects 412 is computed 434 via the picking rule 433. The automatic prediction may be performed based on the history, on the previously performed action, on the system preferences, on predefined templates, on object type, on user rules, on the context of the application, on Artificial Intelligence algorithms and other criteria and techniques. In embodiments, the predictor automatically predicts the scaling rule, the picking rule and the action.

[0126] An exemplary algorithm embodiment with the predictor 535 is now illustrated. The user selects one or more objects for example in an industrial VR environment. The system predicts the picking rule, the scaling method and / or the industrial action of the user. For example, if the user selects a robot, the system predicts that the user wants to increase or decrease robots selection or if the user selects a route, the system predicts that user wants to increase or decrease the visible route and in the max value of the index will be the route destination.

[0127] The first selection of objects is computed and displayed. The user can scale the first object selection with a context by means of the regulator of the predicted scaling rule. The user can dynamically scale the multi-object selection by manipulating the regulator index value within a value range by increasing or decreasing the intensity which will select less object or more objects or the previous / next object of a chain or path. For example, in robot kinematics, the user may want to select first the joint and then they may want to hierarchically scale down / up to the entities of the joint until they chose the second object set. The second set of objects is displayed e.g. via different coloring, via highlight, via text, with a screen showing the selection, via outline etc, including a combination of techniques. The user performs the predicted industrial action on the second set of objects.

[0128] Embodiments advantageously enable an automatic multi selection of objects based on a first selection and on a scaling index manipulation.

[0129] In embodiments, according to the first selected object, the application may either let the user chose one or more scaling rules available for that object or it can automatically predict it by knowing what is the current user’s workflow.

[0130] Figure 6 illustrates an industrial scene in the VR in accordance with disclosed embodiments.

[0131] A user (not shown) holds a HTC VR control 601 in the hand (not shown) to select a TCPF frame 603 via a picker in form of a beam 602. For example, the user can select the object encountered by the beam by pushing the trigger 605 of the control 601. The control 601 is a handset with a touchpad 604 acting as scaling regulator and indicator. It is noted that, in the initial selection, although the user selects via the beam only the TCPF frame, the full robot is selected as highlighted with the different color of the robot. Then, if the user manipulates downwards the index of the scaling regulator, the actual frame of the robot is selected and instead, if the user manipulates upwards, the whole study is selected, as below explained in the description of the exemplary embodiment of Figure 7. It is noted than the highlighting with a different color of the robot in a user’s GUI is hereby shown via the dashed or dotted lines of the robot given that the drawing of Figure 6 is black and white.

[0132] In embodiments, the picking rule defines that the initial selection is at a certain logic level (e.g. component level) and the scaling up / down may keep or it may change this. In embodiments, the system may remember the previous selection level and reapply it to have an initial selection logic.

[0133] By manipulating the regulator on the touchpad, the user is enabled to select more or less objects. The user can manipulate an index pointer of the regulator until a desired multi object selection scaling is reached. With embodiments, the user can advantageously scale the object selection in a flexible and intuitive manner in the VR without the need to visualize side panels and browse tree hierarchies.

[0134] For example, in embodiments, assume the user selects the lower level (TCPF) as shown with ray, the higher level - mid level (e.g. the robot) is shown. Departing fromthis initial selection, via the scaling regulator, if the users scales down the entity that was selected will be displayed (the TCPF) and not the entire component (the robot), but if I the user scales up, the collection of this robot can be selected e.g. a robot package which for example comprises the gun of the robot, some tooling of the robot cables - depending on the desired granularity.

[0135] Advantageously, the user can manipulate the object selection with one hand via the scaling regulator and perform the industrial action on the second object set with the other hand. The second object set is the resulting scaled selection of object(s) computed with the logic defined in the scaling rule. In embodiments, the logic of the scaling rule is flexible and configurable e.g. it may be context sensitive depending on selected object, type of action, other contextual data, any combination thereof.

[0136] In embodiments, in case of objects having a small size or having no actual physical size - such as e.g. frame, operations having graphical representation for viewing purposes, in order to relax the motor skills of the user, vicinity to the object may be taken into account e.g. by locking the ray until it moves far enough from the object, by jumping, by different highlighting.

[0137] Figure 7 schematically illustrates scaling index manipulations and their effects on the selections of objects in accordance with disclosed embodiments.

[0138] Figure 7 shows a sequence of index value manipulations [Levi, Lev2, Lev5, Lev7, Lev9] in a regulator 711-715.

[0139] The user manipulates the intensity levels Levi, ... , Lev9 of the scaling regulator 711-715 to scale the object selection of the industrial scene windows 721 - 725. A scaling regulator can be manipulated by the user for example to obtain a sequence of intensity values Levi, Lev2, Lev5, Lev7, Lev9 within the corresponding sequence of regulators 711-715. Assume the first selection of objects performed by a user (not shown) is the full robot 733 as highlighted in the middle window 723 wherein the robot is highlighted via dashed line. The windows 721-725 illustrate portions of an industrial scene as seen by the user e.g. through VR glasses of or through a viewer of a screen of adesktop application. Assume the chosen scaling rule is of hierarchical type and accordingly, by manipulating the index of the scaling regulator 711-715, the user is enabled to select one entity of the robot - one or two objects - by reducing the intensity level or to select the fully study with its multiple objects by increasing the intensity level.

[0140] For example, the intensity level Levi of the regulator 711 corresponds to the selection of one object - the TCPF 731 - as shown withing the industrial scene portion of window 721. The intensity level Lev2 of the regulator 712 corresponds to the selection of two objects 732 - The TCPF and its link: including the link graphics and frame - as shown within the industrial scene portion of window 722. The intensity level Lev5 of the regulator 713 corresponds to the selection of the collection of objects of the robot 733 as shown withing the industrial scene portion of window 723. The intensity level Lev7 of the regulator 714 corresponds to the selection of the robots and of the controller 734 as shown withing the industrial scene portion of window 724. The intensity level Lev9 of the regulator 715 corresponds to the selection of the full study with all its objects as shown withing the industrial scene portion of window 725.

[0141] It is noted than the highlighting with a different color of the objects in a user’s GUI is hereby shown via the dashed or dotted lines of the objects given that the drawing of Figure 7 is black and white.

[0142] In summary, by regulating the intensity level via manipulation of the current scaling index value, the user can select less or more objects or other objects of the industrial scene.

[0143] Although the above illustrated example of regulator and indicator comprises nine discrete levels with lines, such lines may not represent the real amount of depth levels, it might be that in reality there are more / less levels (e.g. twenty or three), therefore the regulator may be visualized with a gradient and in case it is visualized integer jumps this may not imply a direct correspondence with the real scaling depth.

[0144] In embodiments, the regulator may be a graphical overlay on a VR control of any type, it may be a graphical overlay on a touchscreen, or it may be another type of entity of an XR environment.

[0145] Figure 8 schematically illustrates examples of scaling regulators and indicators in accordance with disclosed embodiments.

[0146] The scaling regulator enable the user to manipulate the current scaling index within a scaling range and the scaling indicator shows the current intensity level of the user’s choice.

[0147] In the first shown example 810, the user rotates the virtual knob - i.e. the scaling regulator - and the highlighted points - i.e. the indicator - indicate the reached intensity level within the range. In the second shown example 820, the commands (-) (\) (+) are part of the regulator and the highlighted histograms represent the indicator. In the third shown example 830, the index level can be regulated via touch gestures and the intensity levels represent the indicator. In the fourth shown example 840, a virtual disk can be rotated (the regulator) and the cursor position indicates the intensity level (the indicator). In the fifth shown example 850, in a slider bar there is a ball-shaped cursor which can be moved by a user to manipulate the current scaling index. In this example, the slider bar with the cursor may act both as regulator and indicator.

[0148] As shown in the above exemplary embodiments, the regulator can be manipulated via a large variety of scrolling mechanism e.g. the scaling can be done using buttons, or via other graphical interfaces like e.g. a scale, a knob, a slider etc. some of which remind the shape of speaker-volume controls. Although the above illustrated examples of regulators and indicators are graphical entities, the skilled persons will understand that the regulators may be manipulated not only via direct hand manipulation gestures but also via captured vocal commands or other types of capturable hand and facial gestures.

[0149] In embodiments, the scaling indicator may be the same graphical entity of the scaling regulator or it may consist of a different entity. For example, the indicator maybe a led bar or funnel that shows the scaling increase / decrease through a change of illumination intensity in a portion of industrial scene. Other examples of indicators include, but are not limited by, progress bar, modifying the background color of the screen, dimming the surrounding, illuminating the objects.

[0150] Although the above illustrated examples of regulators and indicators are graphical entities with one dimensional intensity to regulate, the skilled persons will understand that the regulators may include two-dimensional intensities whereby each dimension corresponds to a different attribute for example by selecting the coordinates X,Y of a point in a plane the user can manipulate the intensity of two scaling attributes. In summary, in embodiments, the index value may be discrete, continuous, vector, matrix.

[0151] An exemplary embodiment of a use case of an index with a continuous ID value may involve the selection of the welding path of a welding tool whereby the regulator 850 of Figure 8 may include a second knob (not shown) so that the first knob defines the welding-path start and the second knob defines the welding-path end; the increased / decreased selection would then be the path in between the start and the end.

[0152] An exemplary embodiment of a use case of a 2D / 3D index with continuous values may involve the selection of a surface area to paint in where some part surfaces are to be selected. In this case, the regulator (not shown) may for example remind a “blanket-surface” that can be elongated / shortened by user’s manipulation to increase / decrease the selection of the surface area to be painted.

[0153] In embodiments, a scaling rule may be used for several categories at once e.g mirroring more or less operations / locations, robotic tools, replace or duplicate (binary) etc.

[0154] Figure 9 schematically illustrates a VR control in accordance with disclosed embodiment.

[0155] As shown in Figure 9, a user in a VR environment (not shown) can use a VR control (e.g. the VR control from HTC) to select 910 a scaling rule from a menu 911 of options (A - E) on the control touchpad. They can pick 920 objects using the ray 921 and the control trigger 922. They can scale 930 the object(s) selection by manipulating the regulator 931 on the control touchpad and by using the trigger 932 when their scaling action is completed. After the scaling action is complete, an engineering action may conveniently be performed e.g. moving the selection, simulating via play / pause / stop buttons etc.

[0156] Figure 10 schematically illustrates graphical interfaces on a touchscreen in accordance with disclosed embodiment.

[0157] As shown in Figure 10, a user’s hand can use the tablet touchscreen to select 1010 a scaling rule from a menu 1011 of options (A - E). They can pick 1020 an object by touching it 1021 on the touchscreen. They can scale 1030 the object selection (not shown) by manipulating the regulator 1031 on the touchscreen. They can perform the industrial action 1040 by playing 1041 the simulation.

[0158] Of course, those of skill in the art will recognize that, unless specifically indicated or required by the sequence of operations, certain steps in the processes described above may be omitted, performed concurrently or sequentially, or performed in a different order.

[0159] Those skilled in the art will recognize that, for simplicity and clarity, the full structure and operation of all data processing systems suitable for use with the present disclosure is not being illustrated or described herein. Instead, only so much of a data processing system as is unique to the present disclosure or necessary for an understanding of the present disclosure is illustrated and described. The remainder of the construction and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0160] It is important to note that while the disclosure includes a description in the context of a fully functional system, those skilled in the art will appreciate that at leastportions of the present disclosure are capable of being distributed in the form of instructions contained within a machine-usable, computer-usable, or computer-readable medium in any of a variety of forms, and that the present disclosure applies equally regardless of the particular type of instruction or signal bearing medium or storage medium utilized to actually carry out the distribution. Examples of machine usable / readable or computer usable / readable mediums include: nonvolatile, hard-coded type mediums such as read only memories (ROMs) or erasable, electrically programmable read only memories (EEPROMs), and user-recordable type mediums such as floppy disks, hard disk drives and compact disk read only memories (CD-ROMs) or digital versatile disks (DVDs).

[0161] Although an exemplary embodiment of the present disclosure has been described in detail, those skilled in the art will understand that various changes, substitutions, variations, and improvements disclosed herein may be made without departing from the spirit and scope of the disclosure in its broadest form.

[0162] None of the description in the present application should be read as implying that any particular element, step, or function is an essential element which must be included in the claim scope: the scope of patented subject matter is defined only by the allowed claims.

Claims

WHAT IS CLAIMED IS:

1. A method for manipulating, by a data processing system, a selection of one or more objects of a digital scene of an industrial environment, comprising the following steps: a) displaying a digital scene comprising a plurality of objects; b) receiving data of a first selection of a first set of objects; c) receiving data of a selection of a scaling rule; said rule defining the effect of a scaling index manipulation on a resulting selection of a resulting set of objects; d) displaying a scaling regulator enabling a user to manipulate the current scaling index within a scaling range; e) via the regulator, receiving data of a manipulation of the current scaling index to obtain a second selection of a second set of objects; f) determining the second selection of objects by applying the scaling rule to the received index manipulation; g) displaying, within the digital scene, a subset of the determined second set of objects.

2. The method of claim 1 further including the step of displaying a “scaling indicator” indicating the current value of the scaling index via a separate entity than the entity of the “scaling regulator”.

3. The method of claim 1 wherein the scaling rule is selected from the group consisting of:- scaling rules of logical type;- scaling rules of hierarchical type;- scaling rules of shape type;- scaling rules of path type;- scaling rules of material type;- scaling rule with modifiable scope;- scaling rules with fixed scope;- other rules of different type;- any combination of the above rules.

4. The method of claim 1, wherein the first set of objects is computed based on a context of a picking rule.

5. The method of claim 1, wherein step c) precedes step b).

6. The method of claim 1 or 2, the environment of the scaling regulator and / or of the indicator is selected from the group consisting of:- windows, mouse and / or keyboard environment;- touchscreen environment;- Virtual Reality environment;- Augmented Reality environment;- another XR environment;- any combination of the above environments.

7. The method of claim 1 further including performing an industrial action on the second set of objects.

8. The method of claim 1 wherein the scaling rule or the industrial action is determined via a predictor based on any of the criteria selected from the group consisting of:- history;- a previously performed action;- system preferences;- predefined templates;- type of the selected objects;- user defined criteria;- context of the application;- Artificial Intelligence algorithms:- other criteria and techniques;- any combination of the above.

9. A data processing system comprising: a processor; and an accessible memory, the data processing system particularly configured to: a) display a digital scene comprising a plurality of objects; b) receive data of a first selection of a first set of objects; c) receive data of a selection of a scaling rule; said rule defining the effect of a scaling index manipulation on a resulting selection of a resulting set of objects; d) display a scaling regulator enabling a user to manipulate the current scaling index within a scaling range; e) via the regulator, receive data of a manipulation of the current scaling index to obtain a second selection of a second set of objects; f) determine the second selection of objects by applying the scaling rule to the received index manipulation; g) display, within the digital scene, a subset of the determined second set of objects.

10. The data processing system of claim 9, further configured to display a “scaling indicator” indicating the current value of the scaling index via a separate entity than the entity of the “scaling regulator”.

11. The data processing system of claim 9, wherein the scaling rule is selected from the group consisting of:- scaling rules of logical type;- scaling rules of hierarchical type;- scaling rules of shape type;- scaling rules of path type;- scaling rules of material type;- scaling rule with modifiable scope;- scaling rules with fixed scope;- other rules of different type;- any combination of the above rules.

12. The data processing system of claim 9, wherein the environment of the scaling regulator and / or of the indicator is selected from the group consisting of:- windows, mouse and / or keyboard environment;- touchscreen environment;- Virtual Reality environment;- Augmented Reality environment;- another XR environment;- any combination of the above environments.

13. The data processing system of claim 9, further configured to perform an industrial action on the second set of objects.

14. The data processing system of claim 9, wherein the scaling rule or the industrial action is determined via a predictor based on any of the criteria selected from the group consisting of:- history;- a previously performed action;- system preferences;- predefined templates;- type of the selected objects;- user defined criteria;- context of the application;- Artificial Intelligence algorithms:- other criteria and techniques;- any combination of the above.

15. A non-transitory computer- readable medium encoded with executable instructions that, when executed, cause one or more data processing system to: a) display a digital scene comprising a plurality of objects; b) receive data of a first selection of a first set of objects; c) receive data of a selection of a scaling rule; said rule defining the effect of a scaling index manipulation on a resulting selection of a resulting set of objects; d) display a scaling regulator enabling a user to manipulate the current scaling index within a scaling range; e) via the regulator, receive data of a manipulation of the current scaling index to obtain a second selection of a second set of objects; f) determine the second selection of objects by applying the scaling rule to the received index manipulation; g) display, within the digital scene, a subset of the determined second set of objects.

16. The non-transitory computer- readable medium of claim 15, further including executable instructions that, when executed, cause one or more data processing system to display a “scaling indicator” indicating the current value of the scaling index via a separate entity than the entity of the “scaling regulator”.

17. The non-transitory computer-readable medium of claim 15, wherein the scaling rule is selected from the group consisting of:- scaling rules of logical type;- scaling rules of hierarchical type;- scaling rules of shape type;- scaling rules of path type;- scaling rules of material type;- scaling rule with modifiable scope;- scaling rules with fixed scope;- other rules of different type;- any combination of the above rules.

18. The non-transitory computer-readable medium of claim 15, wherein the environment of the scaling regulator and / or of the indicator is selected from the group consisting of:- windows, mouse and / or keyboard environment; - touchscreen environment;- Virtual Reality environment;- Augmented Reality environment;- another XR environment;- any combination of the above environments.

19. The non-transitory computer- readable medium of claim 15, further including executable instructions that, when executed, cause one or more data processing system to perform an industrial action on the second set of objects.

Images