Selection of objects manipulating digital industrial scene

Abstract

Systems and methods for displaying a digital scene including a plurality of objects. Data for a first selection of a first set of objects is received. Selected data of a scaling rule is received. A zoom regulator is displayed that enables the user to manipulate the current zoom index within the zoom range. Data for manipulation of the current zoom index is received via the regulator to obtain a second selection of a second set of objects. A second selection of the object is applied by applying a scaling rule to the received indexing manipulation. The determined subset of the second set of objects is displayed within the digital scene.

Description

Technical Field

[0001] This disclosure generally relates to computer-aided design, visualization and manufacturing (“CAD”) systems, product lifecycle management (“PLM”) systems, product data management (“PDM”) systems, production environment simulation, and similar systems for managing data on products and other items (collectively, “Product Data Management” systems or PDM systems). Background Technology

[0002] In industrial engineering, software applications with 3D representations of industrial scenes containing industrial objects are used to optimize and manage design, engineering, and manufacturing processes. Examples of use for industrial software applications include, but are not limited to, manufacturing process design, manufacturing process simulation, production operation control, manufacturing process analysis, equipment collision detection, inspection, review, and virtual commissioning. As used herein, the term "object" can refer to any resource in a digital industrial environment (e.g., equipment, machines, gear, tools, parts, conveyors, walls, fences, tables, people, rooms), any area, any surface, any path, and any physical and / or non-physical element—e.g., location or operation.

[0003] For ease of explanation, we will now focus on the technical challenges faced by industrial engineers in the context of robot simulation platforms. However, this is easily understood by technical personnel, and similar considerations apply to other types of industrial software applications, thereby representing industrial objects in 3D throughout the entire lifecycle of industrial products.

[0004] Examples of robot simulation platforms and systems include, but are not limited to, computer-aided robot (“CAR”) tools, process simulations (Siemens Group products), robot software simulation tools, software applications for industrial robot simulation, and other systems and virtual stations for industrial robot simulation.

[0005] Robot simulation platforms enable simulation engineers to simulate multiple robot operations performed by multiple industrial robots in a simulated workshop setting.

[0006] Figure 3A and Figure 3B An example of a GUI viewer for a robot simulation platform is shown schematically.

[0007] Typically, in desktop applications, a graphical user interface (“GUI”) screen 310 simulating an industrial scene displays several industrial objects 331, thereby allowing dozens of virtual robots to operate simultaneously. To inspect and execute actions on different robot groups, simulation engineers utilize filtering functions included in the GUI's tree-like navigation panels 322, 323.

[0008] In fact, simulation engineers typically need to select a certain number of robots to examine and simulate the 3D kinematic behavior of multiple robots. To do this, they often utilize desktop applications with a GUI, which includes multiple viewer panels / windows, such as a 3D digital scene viewer 321 with industrial objects 331 and side viewers 322, 323 with their navigation trees 332, 333, as... Figure 3A As shown.

[0009] Tree-viewing navigation devices 322 and 323 enable users to select industrial objects based on various filtering options such as hierarchy, object, and operation. Therefore, in industrial desktop applications, users typically utilize a graphical viewer with a side panel featuring navigation trees 332 and 333 (which display data with various filtering and hierarchical possibilities) to select a desired group of objects, thereby advantageously selecting sub-objects within a hierarchy, for example, by selecting objects within the tree.

[0010] Figure 3B An example of an object viewer with a navigation tree 342 for an exemplary spot welding station in an industrial setting is shown in more detail. For example, the tree includes part items and resource items, whereby the resources of the spot welding station include a turntable and a robot, which includes grippers and robot entities such as, for example, the base, links, tool frames, and tool center point frames (“TCPF”). For example, a user selects a part-level command from a graphical selection level menu (not shown), and then they select a specific robot in the tree hierarchy, such as robot rob1 (351), and correspondingly, in scene viewer 321, robot rob1 and its entities are selected by being highlighted in a specific color (not shown). Typically, part / entity levels are primarily used in graphical viewers when a user wants to select a specific level of an object. Generally, selections in the tree are more discrete—what the user selects is what is highlighted.

[0011] Typically, in a robot simulation platform, to perform a given industrial operation on a certain number of robots, the user needs to select and display such a group of robots in the digital scene viewer. To do this, she selects the object entity "robot" in tree 332, 342 of the side viewer, and then drags box 361 in digital scene 321 so that all robots included in box 361 are marked or highlighted with a specific color (here, for illustrative purposes, the marking is shown by marking robots with dots or dashed lines), thus allowing the industrial operation to be performed on the selected / highlighted / marked robots. If the user wants to manipulate the selection—that is, select more or fewer robots—then the user must draw a larger or smaller box.

[0012] In summary, in the desktop application, users can select all robots within a draggable bounding box 361 using the filtering options on the panel with navigation trees 332, 342. The selected robots are highlighted in a predefined color in the digital scene viewer 321. In this way, users can then easily perform given industrial operations on the highlighted robots.

[0013] Examples of industrial operations / actions that need to be performed on multiple objects and at various levels include, but are not limited to, inspection, playing simulations at one or more levels, optimization, line operation control, virtual debugging, object placement; moving groups of objects (not one after another), jogging robots; hiding / shading / wireframed groups of objects, testing, training, coaching; setting annotations, and other operations that need to be performed on multiple industrial objects and at different logical levels.

[0014] Without such multi-object selection and manipulation capabilities, performing actions on industrial desktop applications would be a tedious task, and such applications would be unusable for industrial engineers.

[0015] However, note that current techniques for selecting multiple objects in a digital scene used to manipulate an industrial environment (e.g., via browsing and selection in a navigation tree panel and / or by drawing boxes in the digital scene with a mouse) are conceived for large screens and mouse use.

[0016] Unfortunately, such technology can become cumbersome in modern industrial settings where users increasingly work with tablets, touchscreens, and extended reality (“XR”) environments (e.g., VR, AR, metaverse reality, etc.) and expect efficient capabilities such as flexible and intuitive functions and interactions.

[0017] Therefore, the technology that is expected to be improved. Summary of the Invention

[0018] Various disclosed embodiments include methods, systems, and computer-readable media for manipulating the selection of one or more objects in a digital scene of an industrial environment. One method includes displaying a digital scene comprising multiple objects. The method further includes receiving data of a first selection of a first set of objects. The method further includes receiving data of a selected scaling rule that defines the effect of scaling index manipulation on the resulting selection of a set of objects. The method further includes displaying a scaling adjuster that allows a user to manipulate the current scaling index within a scaling range. The method further includes receiving data of manipulation of the current scaling index via the adjuster to obtain a second selection of a second set of objects. The method further includes determining the second selection of objects by applying a scaling rule to the received index manipulation. The method further includes displaying a subset of the determined second set of objects within the digital scene.

[0019] The foregoing has already provided a fairly broad overview of the features and technical advantages of this disclosure, enabling those skilled in the art to better understand the detailed description that follows. Further features and advantages of the subject matter forming the claims of this disclosure will be described below. Those skilled in the art will understand that they can readily use the disclosed concepts and specific embodiments as a basis for modifying or designing other structures to achieve the same purpose of this disclosure. Those skilled in the art will also recognize that such equivalent constructions do not depart from the spirit and scope of this disclosure in its broadest form.

[0020] Before proceeding with the detailed description below, it may be advantageous to define certain words or phrases used throughout this patent document: the terms “comprising” and “including” and their derivatives mean including but not limited to; the term “or” is inclusive, meaning and / or; the phrases “associated with” and “related to” and their derivatives may mean including, being included therein, interconnected with, containing, contained in, connected to or connected with, coupled to or coupled with, communicable to, cooperating with, interleaved, juxtaposed, proximate, bound to or bound with, having, having attributes, etc.; and the term “controller” means any device, system, or part thereof that controls at least one operation, whether such device is implemented in hardware, firmware, software, or some combination of at least two of them. It should be noted that the functionality associated with any particular controller can be centralized or distributed, local or remote. Definitions of certain words and phrases are provided throughout this patent document, and those skilled in the art will understand that such definitions apply in many (if not most) cases to the prior and future use of such defined words and phrases. While some terms may encompass a wide variety of implementations, the appended claims may expressly limit these terms to specific implementations. Attached Figure Description

[0021] To more fully understand the contents of this disclosure and its advantages, reference will now be made to the description taken in conjunction with the accompanying drawings, wherein similar reference numerals denote similar objects, and in the drawings:

[0022] Figure 1 A block diagram of a data processing system in which implementation methods can be carried out is shown.

[0023] Figure 2 A flowchart illustrating the selection of objects for manipulating a digital industrial scene according to a disclosed embodiment is shown schematically.

[0024] Figure 3A and Figure 3B An example of a GUI viewer for a robot simulation platform is shown schematically.

[0025] Figure 4 A flowchart illustrating a multi-object selection algorithm according to a disclosed implementation is shown schematically.

[0026] Figure 5 A flowchart illustrating automatically predictive object selection scaling according to a disclosed embodiment is shown schematically.

[0027] Figure 6 An industrial scene in VR is shown according to a disclosed implementation.

[0028] Figure 7 The scaling index manipulation according to the disclosed implementation and its effect on object selection are illustrated schematically.

[0029] Figure 8 Examples of zoom adjusters and indicators according to the disclosed embodiments are illustrated schematically.

[0030] Figure 9 A VR control according to a disclosed implementation is illustrated schematically.

[0031] Figure 10 A graphical interface on a touchscreen according to a disclosed embodiment is illustrated schematically. Detailed Implementation

[0032] This article discusses Figures 1 to 10 The various embodiments described in this patent document to illustrate the principles of this disclosure are for illustrative purposes only and should not be construed as limiting the scope of this disclosure in any way. Those skilled in the art will understand that the principles of this disclosure can be implemented in any suitably arranged device. Many of the innovative teachings of this application will be described with reference to exemplary, non-limiting embodiments.

[0033] Previous technologies could not enable the selection of industrial objects on tablets and XR environments in an efficient and intuitive way.

[0034] The implementation provides an efficient mechanism for selecting objects to zoom in / out in industrial settings.

[0035] The implementation provides users with the ability to change the selection of objects without having to reselect them.

[0036] The implementation provides users with a user-friendly multi-object selection mechanism.

[0037] The implementation allows users to intuitively select groups of objects in VR and / or using a touchscreen.

[0038] The implementation method enables multi-object selection in virtual reality, the range of which is adjustable and not a single predefined selection range.

[0039] This implementation allows users to configure the selection range in virtual reality. Advantageously, in such an implementation, the object selection range is not explicitly determined by the action type and the user's initial object selection.

[0040] The implementation allows users to manipulate and control the selection of objects without opening the tree.

[0041] The implementation allows users to control the selection of objects without changing the commands.

[0042] The implementation allows users to control object selection using natural gestures, such as via one or two controllers in VR and / or via finger gestures on a touchscreen.

[0043] The implementation provides users with the ability to select multiple objects using a specific context.

[0044] The implementation enables flexible and configurable selection ranges, such as context-sensitive selection ranges that depend on the selected object, the type of industrial action, other contextual data, or any combination thereof.

[0045] In implementation, scaling rules can be context-based, predefined, system-determined, and / or user-selected.

[0046] The implementation provides a multi-object selection technique configured to respond to the context of a user action and have the ability to zoom in on selection from a single object while taking into account the context of the object.

[0047] The implementation method enables users to perform multi-object selection in an efficient and user-friendly manner within the realm of VR engineering capabilities, based on hierarchy and other considerations.

[0048] In addition to known traditional VR capabilities, the implementation also provides the ability to broaden VR capabilities to perform engineering-oriented tasks, including but not limited to walking around a facility, viewing simulations, conducting guided tours, processing work instructions, and issuing annotations on issues in the plant. Conveniently, the implementation enables VR industrial spaces to complement engineering applications by enabling realistic modeling capabilities, allowing industrial changes to be performed from a different perspective than those performed in front of a 2D screen on a desktop application without the immersive capabilities of virtual reality.

[0049] By expanding the types of VR industrial capabilities, the implementation methods make VR spaces more attractive to a larger number of industrial customers and to a wider range of industrial sectors.

[0050] The implementation allows users to select one or more objects and zoom in or out on that selection. This implementation can be used in VR space.

[0051] The implementation method uses a "panelless" approach to achieve multi-object scaling selection.

[0052] The implementation method allows users to select the type of scaling rule.

[0053] The implementation method enables the automatic prediction of the type of scaling rule.

[0054] In the implementation, the type of scaling rule is context-based.

[0055] The implementation allows users to efficiently select and visualize objects in embedded environments such as virtual reality or other XR environments, and perform desired industrial actions on them.

[0056] Figure 1 A block diagram of a data processing system 100 is shown, in which implementations may be, for example, implemented as a PDM system, specifically configured by software or otherwise to perform the processes described herein, and particularly implemented as each of the plurality of interconnect and communication systems described herein. The illustrated data processing system 100 may include a processor 102 connected to a secondary cache / bridge 104, which in turn is connected to a local system bus 106. The local system bus 106 may be, for example, a Peripheral Component Interconnect (PCI) architecture bus. In the illustrated example, main memory 108 and a graphics adapter 110 are also connected to the local system bus. The graphics adapter 110 may be connected to a display 111.

[0057] Other peripheral devices, such as LAN / WAN / wireless (e.g., WiFi) adapter 112, can also be connected to the local system bus 106. An expansion bus interface 114 connects the local system bus 106 to the input / output (I / O) bus 116. The I / O bus 116 connects to a keyboard / mouse adapter 118, a disk controller 120, and an I / O adapter 122. The disk controller 120 can be connected to a storage device 126, which can be any suitable machine-usable or machine-readable storage medium, including but not limited to non-volatile hard-coded media such as read-only memory (ROM) or electrically erasable programmable read-only memory (EEPROM), magnetic tape storage devices, and user-recordable media such as floppy disks, hard disk drives, and compact disc read-only memory (CD-ROM) or digital versatile discs (DVDs), as well as other known optical, electrical, or magnetic storage devices.

[0058] In the example shown, an audio adapter 124 is also connected to the I / O bus 116, and a speaker (not shown) can be connected to the audio adapter 124 to play sound. A keyboard / mouse adapter 118 provides connectivity for pointing devices (not shown), such as a mouse, trackball, pointing stick, touchscreen, etc.

[0059] Those skilled in the art will understand that Figure 1 The hardware shown can vary for a particular implementation. For example, other peripheral devices such as optical disc drives can be used in addition to or in lieu of the hardware shown. The examples shown are provided for illustrative purposes only and are not intended to imply any architectural limitations with respect to this disclosure.

[0060] Data processing systems according to embodiments of this disclosure may include an operating system employing a graphical user interface (GUI). The operating system allows multiple display windows to be presented simultaneously in the GUI, each providing an interface to a different application or different instances of the same application. Users can manipulate a cursor in the GUI by instructing the device. The cursor's position can be changed and / or events such as clicking a mouse button can be generated to actuate a desired response.

[0061] With appropriate modifications, one of various commercial operating systems can be used, such as a version of Microsoft Windows™ from Microsoft Corporation, located in Redmond, Washington. An operating system can be modified or created based on this disclosure as described.

[0062] LAN / WAN / wireless adapter 112 can connect to network 130 (not part of data processing system 100), which can be any public or private data processing system network or combination of networks as known to those skilled in the art, including the Internet. Data processing system 100 can communicate with server system 140 via network 130, which is also not part of data processing system 100, but can be implemented as a separate data processing system 100, for example.

[0063] Figure 2 A flowchart is shown for selecting one or more objects in a digital scene used to manipulate an industrial environment.

[0064] At action 205, a digital scene including multiple objects is displayed.

[0065] At action 210, the system receives data on the first selection of the first group of objects. The first selection of objects is displayed via a specific marker.

[0066] At action 215, the system receives data on the selection of scaling rules; the rules limit the effect of scaling index manipulation on the resulting selection of a set of objects by starting from a first set of objects.

[0067] At action 220, the system displays a zoom adjuster that allows the user to manipulate the current zoom index within the zoom range.

[0068] At action 225, the system receives data on manipulation or adjustment of the current scaling index via the regulator to obtain a second selection of objects in the second set. For example, the user adjusts the first selection of objects by manipulating the regulator index within the scaling range.

[0069] At action 230, a second choice of object is determined by applying scaling rules to the received index manipulation.

[0070] At action 235, the system displays a subset of the identified second set of objects within the digital scene.

[0071] In one implementation, the system may display a "scaling indicator" that indicates the current value of the scaling index via an entity different from the entity of the "scaling adjuster".

[0072] In implementation, scaling rules can be selected from the following group: scaling rules of logical type; scaling rules of hierarchical type; scaling rules of shape type; scaling rules of path type; scaling rules of material type; scaling rules with a modifiable range; scaling rules with a fixed range; other rules of different types; and any combination of the above rules.

[0073] In this implementation, the first group of objects is calculated based on the context of the selection rules.

[0074] In the implementation, action 215 precedes action 210.

[0075] In implementation, the environment of the zoom adjuster and / or indicator is selected from the group including: window, mouse and / or keyboard environment; touch screen environment; virtual reality environment; augmented reality environment; another XR environment; and / or any combination of the above environments.

[0076] In the implementation method, industrial actions are performed on the second group of objects.

[0077] In implementation, scaling rules or industrial actions are determined by a predictor based on any criteria selected from the group including: history; previously performed actions; system preferences; predefined templates; the type of the selected object; user-defined criteria; the context of the application; artificial intelligence algorithms; other criteria and techniques; and any combination of the above.

[0078] In implementations, as used herein, the term “received / received / currently receiving” can include retrieving from a storage device, receiving from another device or process, receiving via interaction with a user, or otherwise receiving.

[0079] Algorithm of exemplary implementation

[0080] In an exemplary embodiment, the main algorithmic stages and steps for selecting objects to manipulate digital industrial scenarios are described below using... Figure 4 and Figure 5 As shown.

[0081] Figure 4 The diagram schematically illustrates data inputs and outputs for controlling view access to assets according to the disclosed implementation.

[0082] The main stages of a multi-object selection algorithm include one or more of the following blocks:

[0083] - Scaling rule 411, first choice 412, scaling 413, second choice 414, action 415;

[0084] - Regulator 421, Operator 422, Value 423, Indicator 424,

[0085] - Picker 431, Pick 432, Picking Rule 433, Calculate 434.

[0086] The multi-object selection algorithm includes a scaling rule algorithm 410 at its core, which comprises four blocks 411 to 415.

[0087] Scaling rules define the logic by which users can scale a selection of objects by manipulating the scaling index within the scaling range. Scaling rules define the specific logic for collecting objects and scaling the object selection range, such as by adjusting the size / modifying the selection range (e.g., by selecting more / fewer objects within a group of objects) or by moving a fixed selection range (e.g., by selecting one object at a time within a group of objects). In other words, scaling rules define how to scale an initial selection of a group of objects by manipulating the scaling index to achieve the desired effect of a final selection of objects.

[0088] In the implementation, via scaling rules, users can scale object selection by making the selection smaller or larger, more detailed or less detailed, to follow a trajectory or path (by zooming out to the previous connected object or zooming in to the next connected object, to different objects; or to more objects in the same chain or path), and to change the object by selecting different parts of the same object and other object selection types and their combinations.

[0089] Exemplary implementations of object selection range include, but are not limited to, following a chain by increasing the range of the chain, such as selecting additional locations from an initial selection location in a welding operation, for example, increasing in one or two directions.

[0090] Exemplary implementations of moving a fixed object selection range include, but are not limited to, moving one object at a time following a chain, such as moving from one position to another in a welding operation, moving back and forth in a path.

[0091] In an implementation, the scaling index within the scaling range can indicate the selection of fewer or more objects (the size of the scaling range, where, for example, the minimum index points to fewer objects and the maximum index points to more objects) or the selection of different objects within a path (changing objects within an object chain; where, for example, the minimum index points to the initial object of the chain and the maximum index points to the last object of the chain).

[0092] In this implementation, the scaling range and index depend on the data of the industrial environment; for example, the depth / scaling of the data when loading a station may differ from the depth / scaling of the data when loading a line that may include several stations. In this implementation, the index—its depth l..n—can be determined based on the selected object via an algorithm that calculates the scaling range of the index.

[0093] In the algorithm implementation, the system receives the index selected by the user within the scaling rules and the initially selected object to be scaled in the world, and returns the actual selected object based on the scaling index.

[0094] Examples of scaling rule types include "hierarchy" rules (similar to the hierarchies shown in navigation trees 322 and 323), "shape / type" rules (selecting a pyramid and continuing to zoom will select more and more pyramids), "path" rules (selecting objects in a path, where zooming in / out increases / decreases the selection within the path or changes / moves objects within the path or object hierarchy), and other rule types and combinations thereof. In short, scaling rule types represent some logic that limits the manipulation of indices within a range to increase / decrease the selection of objects or change the selection of specific types of objects. It can also be a combination of several methods, such as selecting all robots or all locations in a given path. More details and examples of scaling rule types will be provided later.

[0095] like Figure 4As shown in the flowchart, the system receives data about scaling rule 411 and first selection 412, thereby allowing the user to select a first group of objects from an industrial scene, for example. The user then scales 413 the first selection 412 by manipulating regulator 421 422 to obtain a value through one or more iterations 425. In one embodiment, the manipulated index value may be displayed in an indicator 424, which may be an entity different from the regulator 421. In another embodiment, the manipulated index value is displayed on the regulator entity, which also acts as an indicator entity. A picker 431 is set based on scaling rule 411. The user holds the picker 431 and selects 432 objects in the industrial scene, and calculates 434 the first selection 412 of objects via selection rule 433. In one embodiment, the user uses the picker 431 to make an initial selection in a digital scene. Examples of pickers include, but are not limited to, rays in virtual reality, touch gestures on a screen, eye-tracking gestures, and voice commands. In another embodiment, selection rule 433 calculates 434 the range of the first selection 412 based on a specific context. In fact, each object / entity or group of objects in a digital scene can have several attributes in parallel, such as type, graphical entity, weight, kinematic properties, etc. Thus, some of these may exist only at a certain level, not at higher / lower levels, and the first selection applies to a specific context. In the implementation, given a first selection rule, the user performs a selection 432 via a selector 431, which indicates the location where a selection has been performed (e.g., x, y in 2D; x, y, z in 3D; x, y, z of a 2D bounding box; etc.), and calculates the range of the first selection using the selection rule. In other words, the selector indicates where, and based on that, the selection rule calculates the first object selection.

[0096] Receiving data about scaling rule 411 could mean that the range is selected by the user, is predefined, or is received by a process that automatically determines scaling rule 411, for example, based on the context of first selection 412, via AI, user history, etc. Receiving data about first selection 411 could mean that the range is selected by the user or is a predefined storage selection, etc.

[0097] The order of blocks 411 and 412 is switchable, meaning that scaling rule 411 can be received after or before the first selection 412, for example... Figure 5 Or as shown in two exemplary embodiments of the process sequence described below.

[0098] According to a first exemplary implementation of the process sequence, a user can pre-select the industrial action he / she desires, and each time he / she performs his / her first object selection, the system receives a scaling rule, and the user can then scale the object selection via the rule to perform the desired action. In summary, according to this first example, the process sequence is <select action, first selection, scaling rule, scaling, second selection, execution action>.

[0099] According to a second exemplary implementation of the process sequence, the user can pre-select his desired scaling rule, and after he scales his first object selection, he is provided with available actions he can perform on the selected object. In summary, according to this first example, the process sequence is <scaling rule, first selection, scaling, second selection, selection action, execution action>.

[0100] An example implementation of the first example of the process sequence is now shown:

[0101] - The user selects the desired action to be performed;

[0102] - Users can select objects in VR, for example, via a ray-based mechanism or by touching a virtual object;

[0103] - The user opens the adjuster for the received scaling rules and can adjust the scaling by <increasing / decreasing intensity>, <selecting more / fewer objects>, or alternatively <selecting the previous / next object in a chain or path>.

[0104] - Displaying newly determined object selections to the user via various technologies;

[0105] - Perform the action.

[0106] In this implementation, scaling rules can be received based on context. For example, the behavior of the scaling rule can be sensitive to the type of the first selected object; it can be sensitive to the expected action; it can be sensitive to other contexts or combinations of these and other methods to define the behavior of the scaling rule.

[0107] In implementation, selected objects are displayed in a digital scene using various marking techniques. Examples of object marking and / or highlighting techniques include, but are not limited to: coloring, transparency, highlighting, outlining, zooming in / out of the selection (making it larger or smaller), making it glow (e.g., like a star), darkening / hiding the surrounding environment of unselected objects, animate the selection (jumping up and down), texturing, displaying the screen of the selected object, and other techniques, including combinations thereof.

[0108] Now, an example implementation of the second example of the process sequence is shown:

[0109] - Users select scaling rules from the rule type pool;

[0110] - Users select objects in VR;

[0111] - Users open the adjuster for the selected scaling rule and can adjust the scaling by <increasing / decreasing intensity>, <selecting more / fewer objects>, or alternatively <selecting the previous / next object in a chain or path>.

[0112] - Displaying newly determined object selections to the user via various technologies;

[0113] - Based on scaling rules and object selection, the system provides the user with a selection of available actions to perform, and the user selects the desired action;

[0114] - Perform the action.

[0115] In this implementation, the second selection of the object is performed automatically.

[0116] In one implementation, an exemplary algorithm implementation includes: receiving a first selection 412, whereby the user selects a set of objects; receiving a scaling rule 411 that defines how the user can scale the object selection using context; and receiving a scaling index manipulation 413, whereby the user changes the object selection by dynamically manipulating a scaling index adjuster to reduce / increase / change the selection of objects.

[0117] The scaling index value of the regulator is included in the selection range, for example, assuming 0 to 100, or assuming another range, such as the actual step size, such as the depth of the hierarchy of object tree 342 in Figure 3, from 0 to 7.

[0118] In this implementation, the regulator is an interface configured to be manipulated by the user to receive a currently adjusted index value representing the desired range of object selections as input. The system applies the currently adjusted index value to the initial object selection using logic based on scaling rules to return a set of objects as output.

[0119] The following examples illustrate possible scaling rule types. Scaling rules can be of various types, including but not limited to the following examples: logical, hierarchical, kinematic, shape, material, other scaling rule types, and any combination thereof. A scaling rule type defines how the selection of objects is scaled, starting from the first selection and proceeding to the final selection of objects.

[0120] The "Logic" type can represent any possible logic scaling rule type, such as hierarchy (e.g., object hierarchy or operation hierarchy), kinematics, shape, material, and other types, including combinations of several types.

[0121] The "hierarchy" type can represent a scaling rule type with either an "operation hierarchy" or an "object hierarchy" range, and the choice between the two depends on the context of the object selected by the user. For example, in one implementation, the user does not need to view a graphical panel with an object tree and / or operation tree; the system can determine the type of hierarchy (operation or object) by receiving the user's selection. For instance, if the user selects a location or path (visible in an industrial setting), the system automatically determines the desired operation hierarchy, or if the user selects a robot or part, the system automatically routes to the object hierarchy type.

[0122] The "Kinematics" type can represent a scaling rule type, where the scope is the kinematic tree of a robot or another type of kinematic device. For example, a link is a physical link entity of the robot, Link1, Link2, Link3, etc., and then there are joints j1, j2, j3, ... that typically connect two links. For example, a link can connect two other links via two joints, and loops can also exist. For example, suppose there is a robot rA with six links Link1, Link2, Link3, Link4, Link5, Link6, where these six links are interconnected via five joints j1, 2, j3, j4, j5, as follows: Link1->j1->Link2->j2->Link3->j3->Link4->j4->Link5->j5->Link6. Suppose a user wants to perform a joint manipulation action. First, they select a single link (Link5) and manipulate its joint j5. Then, they might want to zoom in (increase) their first selection to manipulate a larger chain that includes adjacent joints j3 and j4 (a second selection obtained through scaling). They might also want to zoom in further to include other adjacent joints j2 and j4 within the selected chain (a third selection of the object). In this implementation, the equipment has a hierarchy of kinematic components, and each level can have links containing the following kinematics. By selecting a link at a certain point, a parent component link can be selected, and their joints can be manipulated.

[0123] The "Shape" or "Type" type can represent a scaling rule type, where the range can be zoomed in or out to include more or fewer objects of the same shape or type. For example, a user selects a robot, and by zooming in or out, selects more or fewer robots near the selected robot.

[0124] The "Material" type can represent a scaling rule type, where the range can be zoomed in or out to include more or fewer objects of the same material. For example, a user selects wood, and by zooming in or out, selects more or fewer pieces of wood around the selected wood.

[0125] The following examples illustrate possible actions or industrial operations that can be performed on the selection of scaled objects.

[0126] Suppose a virtual industrial scene is created for engineering purposes, and it includes production units, production lines, or complete production plants, including robots, other industrial objects, and all aspects such as simulation, collision, and kinematics. Assume an exemplary use case where, according to the disclosed implementation, a user is navigating this industrial scene in VR, and for engineering purposes, they wish to perform actions by manipulating object selection via scaling, such as playing a simulation, jogging a robot, jogging joints, placing or moving groups of industrial objects, hiding / shading / wireframed selected objects, and / or adding annotations to selected objects.

[0127] As used herein, the term play can refer to playing a simulation and / or playing any desired industrial action. For example, play can include simulating, stenciling, creating new seams on a selected path after the object to be painted and its surface have been selected, such as creating a mesh on the selected object or creating a painting path that matches the object surface.

[0128] According to the use case example, the user's action is to play a simulation. A zoom indicator reflecting the regulator's manipulation is turned on in the VR; where down tuning selects the seam (which is a glued or laser-driven continuous operation), which is after the position of that operation, and where zooming in selects the compound operation containing that operation, then its contained operations, etc., up to the top operation. When the user plays the simulation, the simulation plays the operation, the seam, the seam up to the position, the compound operation, its compound operation, etc., respectively.

[0129] According to another use case example, the user's action involves jogging a joint. A zoom indicator, adjustable via a regulator, is activated in VR. When the user selects a robot, the selected robot's specific link is zoomed out; the external axis of another selected robot is zoomed in; when the user activates a jogging action, the manipulator is activated, and when the user moves the manipulator, the entire selection of one or more objects is moved kinetically. When the user selects a rig, the selected sub-rig or device is zoomed out down to the kinematic level of a leaf; the external axis of another selected robot is zoomed in; when the user activates a jogging action, the manipulator is activated, and when the user moves the manipulator, the selected level moves accordingly.

[0130] In another use case example, the user's action is to perform placement. That is, in order to move an object or group of objects to a new location, the user can first select the type of selection, and then allow them to increase / decrease the zoom via adjusters, and move all the selected objects accordingly.

[0131] There are many implementations of industrial actions that can be conveniently performed on the selected object, and they include, but are not limited to, the following examples.

[0132] Suppose a user selects multiple simulated objects with similar kinematics (e.g., two robots in a station or all the robots in the station), and via a manipulator, she makes them dance together—the objects align and move in the same way according to the manipulator's commands.

[0133] Suppose a user's industrial action could involve moving a robot from one station to another—in order to achieve load balancing, she would then need to select the robot or its entire ecosystem (controllers, tools, cables, which can be used to simulate flexible cables for collision / stress checks).

[0134] Suppose that users can change the location of the entire station or the entire line.

[0135] Suppose a user mirrors or duplicates a station / line operation, for example, when the part is symmetrical and the same work is performed on both sides. Examples of mirroring or duplication applications include, but are not limited to, new automotive variants to accelerate slow processes in a station / line by creating parallel stations / lines that reduce workflow time. For example, when mirroring, the user can select a location and mirror it only to a robot: new robot -> the user can mirror the entire operation to the same robot as new robot -> for the same station, for a new station, or for an existing station -> line… etc. Suppose the industrial motion involves automated welding assignment (with balancing) on ​​one robot, all station robots, or the entire line. Suppose the industrial motion involves running a simulation on the robot -> station -> line.

[0136] In an exemplary implementation, if a user selects a scaling rule of the "hierarchy" type, the selected hierarchy can be moved; she can color the selection, hide it, or show it. However, in an implementation, to perform a specific industrial action 415, it may be preferable to apply different types of scaling rules. For example, a user may want to move her selection, but she wants to scale by proximity, by hierarchy, by object type—for example, moving all objects in her proximity, moving all robots (combinations) in her proximity, selecting a robot and moving its tool up the hierarchy, then the robot controller, then the station, then the line—she may want to move all objects, stations, or lines connected to the controller, robots, or robots. Additionally, in an implementation, the scaling rules for objects and the industrial actions on the objects can often be coupled together to define an industrial scaling technique or "scaling method," which includes both the scaling rules and the industrial actions to be performed on the selected objects.

[0137] Figure 5 A flowchart illustrating automatically predictive object selection scaling according to a disclosed embodiment is shown schematically. Figure 5 Flowcharts and Figure 4The difference in the flowchart is that in process 510, "first selection" 511 precedes "scaling rules", and there is an automatic predictor 535 after selection 432.

[0138] like Figure 4 As shown, the user makes the first object selection, and then she can zoom in and out. Thus, although the system can automatically determine the context, the zoom rules and actions are still determined by the user, without... Figure 5 Automatic prediction.

[0139] like Figure 5 As shown, the main stages of the algorithm, including automatic prediction, include the following blocks:

[0140] - First choice 511, scaling rule 512, scaling 413, second choice 414, action 415;

[0141] - Regulator 421, Operator 422, Value 423, Indicator 424,

[0142] - Picker 431, Pick 432, Automatic Prediction 535, Picking Rule 433, Computer 434.

[0143] The system receives data regarding a first object selection 511 and a predicted scaling rule 512. The user then scales the first selection 412 413 by manipulating regulator 421 422 to obtain a value through one or more iterations 425. In one embodiment, the manipulated value may be displayed in an indicator 424, which may be an entity different from the entity of regulator 421. In another embodiment, the manipulated value is displayed on a regulator entity that also acts as an indicator entity. After the user selects 432 using picker 431, an automatic predictor 535 automatically determines the selection rule, scaling rule, and / or action. The user holds picker 431 and selects 432 objects in an industrial scene, and calculates 434 the first selection 412 of objects via selection rule 433. Automatic prediction can be performed based on history, previously executed actions, system preferences, predefined templates, object types, user rules, application context, artificial intelligence algorithms, and other standards and techniques. In one embodiment, the predictor automatically predicts scaling rules, selection rules, and actions.

[0144] An exemplary algorithmic implementation with predictor 535 is now shown. A user selects one or more objects, for example, in an industrial VR environment. The system predicts the user's selection rules, zoom method, and / or industrial actions. For example, if the user selects a robot, the system predicts that the user wants to increase or decrease robot selections, or if the user selects a route, the system predicts that the user wants to increase or decrease visible routes, and the route destination will be the maximum value in the index.

[0145] The first selection of objects is calculated and displayed. Users can utilize the context to scale the first object selection using a regulator based on predicted scaling rules. Users can dynamically scale multi-object selection by manipulating the regulator index value within a range of values ​​by increasing or decreasing the intensity. This will select fewer or more objects, or the previous / next object in a chain or path. For example, in robot kinematics, a user might want to select joints first, and then they might want to hierarchically zoom in / out on the entities of the joints until they select a second set of objects. The second set of objects can be displayed, for example, through different coloring, through highlighting, through text, by utilizing the screen for display selection, through outlines, etc., including combinations of techniques. The user then performs predicted industrial actions on the second set of objects.

[0146] The implementation method advantageously enables automatic multi-selection of objects based on a first selection and scaling index manipulation.

[0147] In one implementation, depending on the first selected object, the application can allow the user to select one or more scaling rules available for that object, or it can automatically predict it by understanding the current user's workflow.

[0148] Figure 6 An industrial scene in VR is shown according to a disclosed implementation.

[0149] The user (not shown) holds the HTC VR control 601 in their hand (not shown) to select the TCPF frame 603 via a picker in the form of a beam 602. For example, the user can select the object encountered by the beam by pressing the trigger 605 of the control 601. The control 601 is a handle with a touchpad 604, which acts as a zoom adjuster and indicator. Note that in the initial selection, although the user selects only the TCPF frame via the beam, the entire robot is selected and highlighted with a different color for the robot. Then, if the user manipulates the index of the zoom adjuster downwards, the actual frame of the robot is selected, and conversely, if the user manipulates it upwards, the entire study is selected, as shown below. Figure 7 As described in the exemplary embodiments. Note that it is assumed that... Figure 6 If the image is black and white, then the robot's different colors are highlighted in the user's GUI by the robot's dashed or dotted lines.

[0150] In one implementation, the selection rule defines the initial selection at a certain logical level (e.g., component level), and zooming in / out can either maintain or change this. In another implementation, the system can remember the previous selection level and reapply it with the initial selection logic.

[0151] By manipulating the adjusters on the touchpad, users can select more or fewer objects. Users can manipulate the index pointer of the adjusters until the desired multi-object selection zoom is achieved. This implementation allows users to advantageously zoom object selection in VR in a flexible and intuitive way, without needing to visualize side panels and navigate tree hierarchies.

[0152] For example, in one implementation, suppose the user selects a lower level (TCPF), as shown by the rays; the higher level—the intermediate level (e.g., robot)—is then displayed. Starting from this initial selection, via the zoom adjuster, if the user zooms out, the selected entity (TCPF) will be displayed instead of the entire part (robot); however, if the user zooms in, a collection of robots, such as a robot package, may be selected, which includes, for example, the robot's gun, robot cables, and some tools, depending on the desired granularity.

[0153] Advantageously, a user can manipulate object selection via a zoom adjuster with one hand and perform industrial actions on a second set of objects with the other. The second set of objects is a selection of scaled objects calculated using logic defined in the scaling rules. In this implementation, the logic of the scaling rules is flexible and configurable; for example, it can be context-sensitive, depending on the selected objects, action type, other contextual data, or any combination thereof.

[0154] In implementations where objects are small or have no actual physical size (such as frames, operations with graphical representations for viewing purposes), proximity of the object can be considered to alleviate the user's motor skills—for example, by locking a ray until it moves far enough away from the object, by jumping, or by different highlighting.

[0155] Figure 7 The scaling index manipulation according to the disclosed implementation and its effect on object selection are illustrated schematically.

[0156] Figure 7 The sequence of index value manipulation in regulators 711 to 715 is shown [Lev1, Lev2, Lev5, Lev7, Lev9].

[0157] The user manipulates the intensity levels Lev1, ..., Lev9 of zoom adjusters 711 to 715 to zoom the object selection in the industrial scene windows 721 to 725. For example, the user can manipulate the zoom adjusters to obtain a sequence of intensity values ​​lev1, Lev2, Lev5, Lev7, Lev9 within the corresponding adjusters 711 to 715. Assume the first selection of objects performed by the user (not shown) is a complete robot 733 highlighted in the intermediate window 723, with the robot highlighted via a dashed line. Windows 721 to 725 show portions of the industrial scene as seen by the user, for example, through VR glasses or a viewer on a desktop application screen. Assume the selected zoom rule is hierarchical, and accordingly, by manipulating the indices of zoom adjusters 711 to 715, the user can select one entity of the robot (one or two objects) by decreasing the intensity level, or select the entire study with multiple objects by increasing the intensity level.

[0158] For example, the intensity level Lev1 of regulator 711 corresponds to the selection of one object—TCPF 731—as shown in the industrial scene section of window 721. The intensity level Lev2 of regulator 712 corresponds to the selection of two objects 732—TCPF and its links: including link graphics and frames—as shown in the industrial scene section of window 722. The intensity level Lev5 of regulator 713 corresponds to the selection of the object set of robot 733, as shown in the industrial scene section of window 723. The intensity level Lev7 of regulator 714 corresponds to the selection of robot and controller 734, as shown in the industrial scene section of window 724. The intensity level Lev9 of regulator 715 corresponds to the selection of a complete study with all objects, as shown in the industrial scene section of window 725.

[0159] Note, assuming Figure 7 If the image is black and white, then the object is highlighted in the user's GUI using different colors, indicated by dashed or dotted lines.

[0160] In summary, by adjusting the intensity level through manipulating the current scaling index value, users can select fewer or more objects or other objects in an industrial scene.

[0161] Although the examples of regulators and indicators shown above include nine discrete levels with lines, such lines may not represent the true number of depth levels, and there may actually be more / fewer levels (e.g., twenty or three). Therefore, regulators can be visualized as gradients, and in the case of visualization with integer jumps, this may not imply a direct correspondence to the true scaling depth.

[0162] In implementation, the regulator can be a graphical overlay on any type of VR control; it can be a graphical overlay on a touchscreen, or it can be another type of entity in an XR environment.

[0163] Figure 8 Examples of zoom adjusters and indicators according to the disclosed embodiments are illustrated schematically.

[0164] The zoom adjuster allows users to manipulate the current zoom index within the zoom range, and the zoom indicator shows the current intensity level selected by the user.

[0165] In the first example 810, the user rotates a virtual knob—the zoom adjuster—and a highlighted point—the indicator—indicates the intensity level reached within that range. In the second example 820, commands (-), (\), and (+) are part of the adjuster, and a highlighted histogram represents the indicator. In the third example 830, the index level can be adjusted via touch gestures, and the intensity level is indicated by the indicator. In the fourth example 840, the virtual disk can be rotated (the adjuster), and the cursor position indicates the intensity level (the indicator). In the fifth example 850, a spherical cursor exists within a slider, which the user can move to manipulate the current zoom index. In this example, the slider with the cursor can function as both an adjuster and an indicator.

[0166] As illustrated in the exemplary embodiments above, the regulator can be manipulated via various scrolling mechanisms, such as using buttons or other graphical interfaces like rulers, knobs, sliders, etc. (some of which resemble the shape of speaker volume controls) for scaling. Although the examples of regulators and indicators shown above are graphical entities, those skilled in the art will understand that regulators can be manipulated not only via direct hand gestures but also via captured voice commands or other types of captured hand and facial gestures.

[0167] In implementations, the zoom indicator can be the same graphical entity as the zoom adjuster, or it can comprise different entities. For example, the indicator can be an LED bar or funnel that indicates zoom increase / decrease through changes in lighting intensity within a portion of an industrial scene. Other examples of indicators include, but are not limited to, progress bars, modifying the background color of a screen, darkening the surrounding environment, and illuminating objects.

[0168] Although the examples of adjusters and indicators shown above are graphical entities with one-dimensional intensity to be adjusted, those skilled in the art will understand that adjusters can include two-dimensional intensity, whereby each dimension corresponds to a different attribute. For example, by selecting the coordinates X and Y of a point in a plane, a user can manipulate the intensity of two scaling attributes. In summary, in implementations, index values ​​can be discrete, continuous, vector, or matrix.

[0169] Exemplary implementations of use cases with indexes having continuous 1D values ​​could involve selecting the welding path of a welding tool, thereby Figure 8 The regulator 850 may include a second knob (not shown) such that the first knob defines the start of the welding path and the second knob defines the end of the welding path; then, the increase / decrease selection will be the path between the start and the end.

[0170] An exemplary implementation of a use case with 2D / 3D indexes having continuous values ​​may involve selecting surface areas to be sprayed, where some part surfaces are to be selected. In this case, an adjuster (not shown) may be, for example, a “blanket surface” that can be stretched / shortened by user manipulation to increase / decrease the selection of surface areas to be sprayed.

[0171] In implementation, scaling rules can be applied to several categories simultaneously, such as mirroring more or fewer operations / locations, robotic tools, replacement or copying (binary), etc.

[0172] Figure 9 A VR control according to a disclosed implementation is illustrated schematically.

[0173] like Figure 9 As shown, in the VR environment (not shown), a user can use VR controls (such as VR controls from HTC) to select a scaling rule (910) from menu (A to E) on the control touchpad using menu 911. They can use ray 921 and control trigger 922 to select objects (920). They can scale the object selection (930) by manipulating adjusters (931) on the control touchpad, and trigger 932 is used when their scaling action is complete. After the scaling action is complete, engineering actions can be easily performed, such as moving the selection, simulating via the play / pause / stop buttons, etc.

[0174] Figure 10 A graphical interface on a touchscreen according to a disclosed embodiment is illustrated schematically.

[0175] like Figure 10As shown, the user can use the tablet touchscreen to select a scaling rule from menu 1011 of options (A to E). They can select an object 1020 by touching it on the touchscreen. They can zoom the object selection (not shown) 1030 by manipulating the adjuster 1031 on the touchscreen. They can perform industrial actions 1040 by playing a simulation 1041.

[0176] Of course, those skilled in the art will recognize that, unless the sequence of operations specifically indicates or requires, certain steps in the above process may be omitted, performed simultaneously or sequentially, or performed in a different order.

[0177] Those skilled in the art will recognize that, for simplicity and clarity, this document does not illustrate or describe the full structure and operation of all data processing systems suitable for use with this disclosure. Rather, only data processing systems specific to or necessary for understanding this disclosure are shown and described. The remainder of the construction and operation of data processing system 100 may conform to any of the various current implementations and practices known in the art.

[0178] It is important to note that although this disclosure is described in the context of a full-function system, those skilled in the art will understand that at least a portion of this disclosure can be distributed in the form of instructions contained in any form of machine-usable, computer-usable, or computer-readable medium, regardless of the specific type of the instruction or signal-bearing medium or storage medium used to actually implement the distribution. Examples of machine-usable / readable or computer-usable / readable media include: non-volatile, hard-coded media, such as read-only memory (ROM) or erasable, electrically programmable read-only memory (EEPROM), and user-recordable media, such as floppy disks, hard disk drives, and optical disc read-only memory (CD-ROM) or digital versatile discs (DVDs).

[0179] While exemplary embodiments of this disclosure have been described in detail, those skilled in the art will understand that various changes, substitutions, modifications, and improvements may be made to this disclosure without departing from the spirit and scope of the broadest form of this disclosure.

[0180] Nothing described in this application should be construed as implying that any particular element, step, or function is an essential element that must be included within the scope of the claims: the scope of the patent subject matter is defined only by the granted claims.

Claims

1. A method for selecting one or more objects in a digital scene of an industrial environment via a data processing system, comprising the steps of: a) Display a digital scene that includes multiple objects; b) Receive the data for the first selection of the first group of objects; c) Receive data on the selection of scaling rules; the rules define the effect of scaling index manipulation on the resulting selection of a set of objects; d) Displays a zoom adjuster that allows the user to manipulate the current zoom index within the zoom range; e) Receive manipulated data of the current scaling index via the regulator to obtain a second selection of the second set of objects; f) Determine a second selection of objects by applying the scaling rules to the received index manipulation; g) Display a subset of the determined second set of objects within the digital scene.

2. The method according to claim 1, further comprising: The step of displaying a "scaling indicator" that indicates the current value of the scaling index via an entity different from the entity of the "scaling adjuster".

3. The method according to claim 1, wherein, The scaling rules are selected from the following group: - Scaling rules for logical types; - Scaling rules for hierarchy types; - Scaling rules for shape types; - Scaling rules for path types; - Scaling rules for material types; - Scaling rules with modifiable range; - Scaling rules with a fixed range; - Other rules of different types; - Any combination of the rules above.

4. The method according to claim 1, wherein, The first group of objects is evaluated based on the context of the selection rules.

5. The method according to claim 1, wherein, Step c) is performed before step b).

6. The method according to claim 1 or 2, wherein the environment of the zoom adjuster and / or the indicator is selected from the group consisting of: - Window, mouse, and / or keyboard environment; - Touchscreen environment; - Virtual reality environment; - Augmented reality environment; - Another XR environment; - Any combination of the above environments.

7. The method according to claim 1, further comprising: Perform industrial actions on the second group of objects.

8. The method according to claim 1, wherein, The scaling rules or industrial actions are determined by the predictor based on any criteria selected from the following groups: - history; - The action previously performed; - System preferences; - Predefined templates; - The type of the selected object; - User-defined standards; - The context of the application; - Artificial intelligence algorithms: - Other standards and technologies; - Any combination of the above.

9. A data processing system, comprising: processor; as well as Accessible memory, the data processing system is specifically configured to: a) Display a digital scene that includes multiple objects; b) Receive the data for the first selection of the first group of objects; c) Receive data on the selection of scaling rules; the rules define the effect of scaling index manipulation on the resulting selection of a set of objects; d) Displays a zoom adjuster that allows the user to manipulate the current zoom index within the zoom range; e) Receive manipulated data of the current scaling index via the regulator to obtain a second selection of the second set of objects; f) Determine a second selection of objects by applying the scaling rules to the received index manipulation; g) Display a subset of the determined second set of objects within the digital scene.

10. The data processing system of claim 9 is further configured to: display a "scaling indicator" indicating the current value of the scaling index via an entity different from the entity of the "scaling adjuster".

11. The data processing system according to claim 9, wherein, The scaling rules are selected from the following group: - Scaling rules for logical types; - Scaling rules for hierarchy types; - Scaling rules for shape types; - Scaling rules for path types; - Scaling rules for material types; - Scaling rules with modifiable range; - Scaling rules with a fixed range; - Other rules of different types; - Any combination of the rules above.

12. The data processing system according to claim 9, wherein, The environment of the zoom adjuster and / or the indicator is selected from the group including: - Window, mouse, and / or keyboard environment; - Touchscreen environment; - Virtual reality environment; - Augmented reality environment; - Another XR environment; - Any combination of the above environments.

13. The data processing system according to claim 9 is further configured to: perform industrial actions on the second group of objects.

14. The data processing system according to claim 9, wherein, The scaling rules or industrial actions are determined by the predictor based on any criteria selected from the following groups: - history; - The action previously performed; - System preferences; - Predefined templates; - The type of the selected object; - User-defined standards; - The context of the application; - Artificial intelligence algorithms: - Other standards and technologies; - Any combination of the above.

15. A non-transitory computer-readable medium encoded with executable instructions, which, when executed, cause one or more data processing systems to: a) Display a digital scene that includes multiple objects; b) Receive the data for the first selection of the first group of objects; c) Receive data on the selection of scaling rules; the rules define the effect of scaling index manipulation on the resulting selection of a set of objects; d) Displays a zoom adjuster that allows the user to manipulate the current zoom index within the zoom range; e) Receive manipulated data of the current scaling index via the regulator to obtain a second selection of the second set of objects; f) Determine a second selection of objects by applying the scaling rules to the received index manipulation; g) Display a subset of the determined second set of objects within the digital scene.

16. The non-transitory computer-readable medium of claim 15, further comprising executable instructions that, when executed, cause one or more data processing systems to display a "scaling indicator" indicating the current value of the scaling index via an entity different from the entity of the "scaling adjuster".

17. The non-transitory computer-readable medium according to claim 15, wherein, The scaling rules are selected from the following group: - Scaling rules for logical types; - Scaling rules for hierarchy types; - Scaling rules for shape types; - Scaling rules for path types; - Scaling rules for material types; - Scaling rules with modifiable range; - Scaling rules with a fixed range; - Other rules of different types; - Any combination of the rules above.

18. The non-transitory computer-readable medium according to claim 15, wherein, The environment of the zoom adjuster and / or the indicator is selected from the group including: - Window, mouse, and / or keyboard environment; - Touchscreen environment; - Virtual reality environment; - Augmented reality environment; - Another XR environment; - Any combination of the above environments.

19. The non-transitory computer-readable medium of claim 15, further comprising executable instructions that, when executed, cause one or more data processing systems to perform industrial actions on a second set of objects.

Images