Information processing systems, information processing methods, and programs

Abstract

According to one aspect of the present invention, an information processing system is provided, comprising at least one processor capable of executing a program such that the following steps are performed: an acquisition step, which acquires object information relating to at least one object placed in a representation space, wherein the representation space is configured to visually represent the object, the object information includes element information relating to a plurality of elements constituting the object, an assignment step, which assigns attributes different from an identification code to the elements based on the object information, wherein the identification code is a non-natural language code that is mechanically uniquely assigned to each of the plurality of elements when the object is defined in the representation space, the attributes are described in natural language, and an update step, which updates the attributes of the target element based on the latest attributes of adjacent elements adjacent to the target element, which is one of the plurality of elements.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses a technique capable of facilitating data management in a design / manufacturing support system. In the design / manufacturing support system including a design system (CAD system), an evaluation system (CAE system), and a production management system according to the technique, an association between a CAD data file and a component is defined in a parts list, and the data granularity (contents of data included in one data file) used in each system is unified. Thereby, it is disclosed that registration of CAD data from the design system to the evaluation system and the production management system can be performed automatically without processing.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Incidentally, the elements that make up an object placed in a representation space are uniquely distinguished by mechanically assigned identification codes, and in some cases, users can perform operations on the specified element by non-verbally specifying that element through a graphical user interface (GUI) in that representation space. However, some users may want to complete operations on elements using a character user interface (CUI). Users assign meaning to the elements that make up an object placed in a representation space through various contexts such as shape, function, and relationships with other elements, and recognize the elements through this meaning assignment. Therefore, it has been difficult to specify elements identified by mechanically assigned identification codes through a CUI. [Means for solving the problem]

[0005] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor capable of executing a program such that the following steps are performed: an acquisition step, which acquires object information relating to at least one object placed in a representation space, wherein the representation space is configured to visually represent the object, the object information includes element information relating to a plurality of elements constituting the object, an assignment step, which assigns attributes different from an identification code to the elements based on the object information, wherein the identification code is a non-natural language code that is mechanically uniquely assigned to each of the plurality of elements when the object is defined in the representation space, the attributes are described in natural language, and an update step, which updates the attributes of the target element based on the latest attributes of adjacent elements adjacent to the target element, which is one of the plurality of elements.

[0006] This configuration can improve the accuracy of semantic identification of elements in the representation space. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] This is a block diagram showing the hardware configuration of user terminal 3. [Figure 4] This figure shows an example of the data structure of the model data. [Figure 5] This is a diagram used to show the relationships between the elements that make up an object. [Figure 6] This figure shows an example of reference information IF1 that includes the naming attribute. [Figure 7] This figure shows an example of reference information IF2 that includes location attributes. [Figure 8] This figure shows an example of reference information IF3 that includes shape attributes. [Figure 9] This figure shows an example of reference information IF4 that includes appearance attributes. [Figure 10] This is an activity diagram showing an example of the information processing flow for assigning attributes, which is performed in Information Processing System 1. [Figure 11] This figure shows an example of attributes assigned to the object shown in Figure 5. [Figure 12] Figure 10 is an activity diagram showing an example of the update process in Action A7. [Figure 13] This figure shows an example of a graph network G configuration for describing the relationships between elements. [Figure 14] Figure 13 shows an example of the information associated with nodes and edges used in graph network G. [Figure 15] This is an activity diagram showing the information processing flow for performing operations on an object. [Figure 16] This figure shows the relationship between changes in topology information and attributes when a specific operation is performed on the object shown in Figure 11. [Figure 17] This is an example of an operation screen displayed on the display unit in information processing for performing operations on an object. [Modes for carrying out the invention]

[0008] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.

[0009] Incidentally, the program or program product for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided so that the program or program product is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.

[0011] In addition, in one embodiment, the "unit" may include, for example, hardware resources implemented by a circuit in a broad sense and information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and these information are represented, for example, by physical values of signal values representing voltage and current, the high and low of signal values as a binary bit aggregate composed of 0 or 1, or quantum superposition (so-called quantum bits), and communication and operations can be executed on a circuit in a broad sense.

[0012] Furthermore, a circuit in a broad sense is a circuit realized by appropriately combining at least a circuit (Circuit), circuitry (Circuitry), a processor (Processor), a memory (Memory), etc. Also, the processor may be a general-purpose processor or a dedicated circuit. That is, it includes application-specific integrated circuits (ASICs), programmable logic devices (for example, simple programmable logic devices (SPLDs), complex programmable logic devices (CPLDs), and field-programmable gate arrays (FPGAs)), etc.

[0013] 1. Hardware Configuration In this section, the hardware configuration will be described.

[0014] <Information Processing System 1> Figure 1 is a configuration diagram representing information processing system 1. Information processing system 1 comprises an information processing device 2 and a user terminal 3. The information processing device 2 and the user terminal 3 are configured to communicate with each other via a telecommunications line. In one embodiment, information processing system 1 consists of one or more devices or components. For example, if it consists only of the information processing device 2, then information processing system 1 can be the information processing device 2. These components will be described below.

[0015] <Information Processing Device 2> Figure 2 is a block diagram showing the hardware configuration of the information processing device 2. The information processing device 2 comprises a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23, and these components are electrically connected within the information processing device 2 via the communication bus 20. Each component will be described further.

[0016] The communication unit 21 preferably uses wired communication methods such as USB, IEEE1394, Thunderbolt®, and wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and Bluetooth® communication as needed. In other words, it is more preferable to implement it as a collection of these multiple communication methods. That is, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] The memory unit 22 stores various types of information as defined above. This can be implemented, for example, as a storage device such as a solid-state drive (SSD) that stores various programs related to the information processing device 2 executed by the processor 23, or as memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. The memory unit 22 stores various programs and variables related to the information processing device 2 executed by the processor 23.

[0018] For example, the memory unit 22 stores the data structure DS and reference information IF1 to IF4, which will be described later.

[0019] The processor 23 performs processing and control of the overall operation related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading predetermined programs stored in the memory unit 22. That is, information processing by software stored in the memory unit 22 can be concretely realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being a single unit, and may be implemented with multiple processors 23 for each function, or a combination thereof.

[0020] The processor 23 is configured as an acquisition unit to acquire information from the user terminal 3 or other devices. The processor 23 is configured to acquire various information by reading various information stored in the storage area, which is at least a part of the memory unit 22, and writing the read information to the work area, which is at least a part of the memory unit 22. The storage area is, for example, the area of ​​the memory unit 22 that is implemented as a storage device such as an SSD. The work area is, for example, the area that is implemented as memory such as RAM. The acquisition by the processor 23 includes acquiring the output results of each functional unit included in the processor 23.

[0021] The processor 23 is configured as a display processing unit to display various information. This information can be presented to the user via the display unit 34 of the user terminal 3 (described later) or other devices. In such cases, for example, the processor 23 controls the display unit 34 of the user terminal 3 to display visual information such as screens, images including still images or videos, icons, and messages. The processor 23 may generate only rendering information for displaying the visual information on the user terminal 3. The processor 23 may also present the outputted information to the user without going through the user terminal 3 or other devices.

[0022] <User Terminal 3> Figure 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 comprises a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. These components are electrically connected within the user terminal 3 via the communication bus 30. The explanation of the communication unit 31, the storage unit 32, and the processor 33 is the same as the explanation of each part in the information processing device 2 and is therefore omitted.

[0023] The display unit 34 may be included in the user terminal 3 housing or it may be an external component. The display unit 34 displays a user-operable GUI screen. This is preferably done by using different display devices such as a CRT display, liquid crystal display, organic EL display, and plasma display, depending on the type of user terminal 3.

[0024] The input unit 35 is configured to accept input from the user. The input unit 35 may be included in the casing of the user terminal 3 or it may be an external component. For example, the input unit 35 may be integrated with the display unit 34 and implemented as a touch panel. If it is a touch panel, the user can input tap operations, swipe operations, etc. Of course, instead of a touch panel, a switch button, mouse, QWERTY keyboard, voice recognition device, gesture detection device, gaze detection device, biosignal detection device, imaging device, etc. may be used. In other words, the input unit 35 accepts operation input made by the user. In response, the input unit 35 transmits a signal corresponding to the operation input to the processor 33 via the communication bus 30. The processor 33 can then perform predetermined controls and calculations as needed.

[0025] <Sound collection device 4> The information processing system 1 may further include a sound collection device 4. The sound collection device 4 is a so-called microphone configured to convert external sounds into signals. The sound collection device 4 may be directly connected to the user terminal 3 or connected to the information processing device 2. Alternatively, the sound collection device 4 may be built into the user terminal 3.

[0026] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in the memory of the user terminal and does not necessarily need to be stored non-volatilely in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via the network.

[0027] The sound collection device 4 is not particularly limited, but it collects sound in the human audible range, specifically between 20 Hz and 20,000 Hz, and converts it into an electrical signal. The sound may be recorded in mono or stereo. Examples of sampling rates when digitally processing the sound data include 48,000 Hz, 44,100 Hz, 32,000 Hz, 22,050 Hz, 16,000 Hz, 11,025 Hz, 11,000 Hz, and 8,000 Hz. Any of the values ​​exemplified here may be used. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, thereby improving the accuracy of speech recognition.

[0028] Furthermore, the data collected by the sound collection device 4 may be compressed as appropriate by the processor 33 of the user terminal 3, and the compression format may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. Compression can reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.

[0029] 2. Regarding information processing This section describes the information processing performed in the aforementioned information processing system 1. As an example, this information processing is configured to make objects placed in the representation space uniquely identifiable using natural language.

[0030] 2.1. Data and Data Structure First, the various data and data structures used in this information processing will be described. These data and data structures may be stored in the storage unit 22, the storage unit 32, or in other external databases, external servers, etc.

[0031] <Model Data> Model data is data used to represent a model. The model may be a spatial model such as a three-dimensional, two-dimensional, or one-dimensional spatial model, a hyperdimensional spatial model of four dimensions or more, a spatiotemporal model that includes a time axis in addition to spatial degrees of freedom, or a model that includes other degrees of freedom (for example, field degrees of freedom such as electromagnetic fields or thermal fields). The model data itself may be binary data configured to be decoded based on the data structure DS described above, or it may be managed to explicitly have the data structure DS described later. A model can be represented by at least one object. Therefore, model data is an example of object information relating to at least one object placed in the representation space. The representation space is a space for visually representing objects, and can be described using parameters relating to degrees of freedom that can be represented in the representation space, such as position coordinates that represent spatial positions. As described above, the representation space may be an N-dimensional Euclidean space, a Minkowski space that includes a time axis, a complex space, or a space described by quaternions, and may include any space in which a state can be described.

[0032] <Object Information> Object information includes element information about the elements that constitute the object. The elements that make up an object may include, for example, zero-dimensional elements that describe the object (e.g., points, especially vertices), one-dimensional elements (e.g., line segments, curves, arcs, circles, etc.), two-dimensional elements (e.g., planes and surfaces), and three-dimensional elements (e.g., cubes, spheres, etc.). These elements of different dimensions are managed in relation to one another. These relationships will be described later.

[0033] <Data structure> Figure 4 shows an example of the data structure of model data. As shown in Figure 4, the model data has a data structure DS called a drawing database. The drawing database is configured to read the model data as a data structure. The data structure DS includes, for example, a block table. The block table is configured to describe the model data mutually in different types of representation spaces, such as model space, paper space, and block definitions that can be arbitrarily defined by various users. The model space is configured to represent the model data as a three-dimensional solid (for example, a CG model such as a polygon). The paper space is configured to represent the model data as at least one two-dimensional drawing data. The model represented by the model space may be configured to describe elements of other dimensions based on the relationships between one-dimensional elements, such as multiple line segments (first line segment, second line segment, third line segment, etc.), arcs, and circles, as elements that constitute the model. In addition, the elements that constitute an object may include the object itself, which is the whole set. In other words, the elements that make up an object may include a portion of the three-dimensional region that constitutes the object, at least a portion of the two-dimensional region that constitutes the three-dimensional region, or at least a portion of the one-dimensional region that constitutes the two-dimensional region. Furthermore, object information may include information about the boundaries of the elements that make up the object.

[0034] Furthermore, in this embodiment, an object may include an object object defined as a physical object in the representation space, and a field object that describes the field of physical quantities in the representation space. With this configuration, by treating not only an object object defined as a physical object in the representation space, but also a field object that describes the field of physical quantities, as objects, geometric objects and physical quantity distributions can be managed within a common framework. Therefore, the scope of application of semantic assignment and update processing can be extended.

[0035] An object is an object that has a physical existence in the model space described above, and is a three-dimensional solid (for example, a CG model such as a polygon). A field object is a pseudo-object that represents values ​​distributed on the model space or on an object (or its elements). A field object may represent a scalar field, a vector field, or a tensor field. Examples of field objects include a physical quantity distribution object that represents the distribution of physical quantities, a state quantity distribution that represents the distribution of state quantities, and an evaluation quantity distribution that represents an evaluation quantity in the model space. As a field for physical quantities, various forms that can be represented in the model space can be adopted, such as a temperature field, pressure field, stress field, flow velocity field, and electric potential field. As a field corresponding to a state quantity distribution, for example, it may include a state quantity field that expresses the state quantities of an object (such as hardness or material) as values. As a field corresponding to an evaluation quantity distribution, for example, it may include a confidence field that represents the distribution of the likelihood of the meaning of the elements when performing the processing described later, an importance field that represents the distribution of weighting coefficients related to calculations in space, and a distance field that represents the distance between elements.

[0036] <Relationships between elements> Figure 5 is a diagram illustrating the relationships between the elements that make up an object. As shown in Figure 5, object Ob1 is a three-dimensional solid in the shape of a rectangular prism and has six faces. Each face is enclosed by four edges, each edge described as a line segment connecting two vertices. Here, object Ob1 treats the four edges as directed line segments and describes other elements using the order relationships of each directed line segment. Each directed line segment can be described, for example, by the relative position coordinates of its start and end points. An order relationship from the first directed line segment to the second directed line segment can be defined by connecting the end point of the first directed line segment to the start point of the second directed line segment. When multiple such directed line segments form a closed loop, the closed surface bounded by this loop can be described as a two-dimensional element, a continuous surface (especially a plane). Furthermore, the adjacency of faces can be described by the relationship between two directed line segments describing two different faces. Furthermore, vertices can be formed by assigning the identifier "Vertex" to the connection relationship between two directed line segments. Curves can be formed by connecting infinitesimal directed line segments. Surfaces can be formed by smoothly connecting infinitesimal surfaces.

[0037] <Identification code> Each element constituting these objects is assigned an identification code. The identification code is described by a non-natural language code (e.g., a number or a random string) that uniquely identifies the element constituting the object, and is called, for example, an entity ID. The processor 23 can accept the user's specification of an element constituting an object through non-textual input operations (e.g., touch operations, tap operations, click operations, etc.) on the GUI displayed on the display unit 34, identify the identification code of the element to be operated on according to the input operation, and execute the operation. However, the identification code itself is determined independently of the context of the element (e.g., randomly). Context refers to the meaning of the elements constituting the object as perceived or intended by the user (e.g., common name, function, part, appearance, relative positional relationship from the user's perspective, etc.). Therefore, by allowing the user to identify the elements constituting the object to be operated on using text input (especially natural language input), it may be possible to perform more efficient operations on the object through the CUI.

[0038] <Reference Information> Reference information is information for assigning attributes to the elements that make up an object, allowing them to be managed under a predetermined namespace, and it shows the correspondence between each attribute and the state of the object's elements. An attribute is a code that represents the "context" mentioned above, and is written, for example, in natural language. Attributes may include naming attributes related to the naming of the object or the elements that make up the object. Figure 6 shows an example of reference information IF1 that includes naming attributes. The reference information IF1 shown in Figure 6 hierarchically represents the concept of a structure called "house". For example, reference information IF1 indicates that the structure called "house" is associated with the highest level of hierarchy (for example, the hierarchy that represents an entire set of objects), and that the house includes (or is composed of) walls, a roof, a front door, windows, pillars, and rooms. Reference information IF1 is specified to show the correspondence between these natural language identifiers and the elements that make up the object included in the imported model. In other words, reference information can be associated with naming attributes corresponding to each element that makes up an object. With such a configuration, natural naming can be used when specifying an object, thus improving the convenience for users when specifying objects. Reference information IF1 is configured to allow the assignment of naming attributes to each object included in the imported model by associating it with model data that has these naming attributes pre-assigned to the positional relationships of each element included in a house (for example, walls extend from the floor, the roof is placed on the floor, the entrance is located on one of the walls, etc.) and to the elements that make up multiple sample objects representing a "house".

[0039] Furthermore, attributes may include positional attributes that describe the positional relationships within or between the elements constituting an object using natural language. Figure 7 shows an example of reference information IF2 that includes positional attributes. As shown in Figure 7, reference information IF2 may include positional relationships such as center, front, back, left, right, top, bottom, outer edge, inner edge, etc. For example, a partial element contained within an element (e.g., face A) can be expressed using reference information IF2 as "the central part of face A," "the front part of face A," "the outer edge of face A," "the inner edge of face A," etc. Each of these parts may be assigned a non-natural language identifier. Each of these positional attributes may be associated with a predetermined determination formula (e.g., coordinate comparison between objects). Positional attributes can describe the positional relationships between the elements constituting an object (e.g., one object is located to the left of another object). Note that the positional relationships within the elements constituting an object can also be expressed as the positional relationships between those elements.

[0040] Furthermore, attributes may include shape attributes that describe the shape of the elements constituting an object using natural language. Figure 8 shows an example of reference information IF3 that includes shape attributes. As shown in Figure 8, reference information IF3 can describe shape attributes using modifying natural language clauses that represent the shape, such as "flat," "wide," "angular," "curved," and "bent." These shape attributes may be defined in association with conditions that assign the attribute, such as determination conditions based on the connection relationships of line segment elements (for example, indicating "angular" if the angle between line segment elements is greater than or equal to a default value).

[0041] Furthermore, attributes may include appearance attributes relating to the appearance of the elements that make up an object. Figure 9 shows an example of reference information IF4 that includes appearance attributes. Appearance attributes may include, for example, attributes that represent colors such as "red," "yellow," "blue," and "green," attributes that represent patterns such as "striped," "checkerboard," and "triangular grid," and attributes that represent textures such as "smooth," "rough," and "rough." These attributes may be defined in association with discrimination criteria based on texture information assigned to the object (e.g., RGB values, material texture type, etc.).

[0042] Thus, reference information can be configured to hierarchically represent the relative relationships of the elements that make up an object (for example, the "wall" of the "house," the "left part" of the "floor" of the "house," etc.).

[0043] The specific form of the reference information may be a table linking "terms" and "judgment conditions" as described above, a hierarchical network model describing knowledge, or a pre-trained model (especially a generative AI model). A pre-trained model can be learned, for example, by machine learning using a training dataset that includes model data describing objects to which natural language attributes, different from the identification code, have been assigned in advance.

[0044] 2.2. Flow of information processing for assigning attributes Figure 10 is an activity diagram showing an example of the flow of information processing for assigning attributes, which is performed in Information Processing System 1. Note that this information processing may include arbitrary exception handling not shown. Exception handling includes interrupting the information processing or omitting individual processes. The identification or input performed in this information processing may be based on user operation or may be performed automatically without user operation.

[0045] [Action A1] As shown in Figure 10, first, in action A1, the processor 33 accepts model data input from the user. Note that the model data input can be existing model data created by the user, or model data from the template model to be generated.

[0046] [Action A2] Next, in action A2, processor 33 defines the model space as a representation space based on the model data. This virtually provides a space where objects can be placed.

[0047] [Action A3] Next, in action A3, the processor 33, as an acquisition unit, acquires object information relating to at least one object placed in the representation space for visually representing the object. If the model data is blank model data with no objects placed on it, the processor 33 may appropriately acquire information about a new object to be placed in the model space as object information.

[0048] [Action A4] Next, in action A4, processor 33 assigns an identification code to each element constituting the acquired object. The identification code may be defined, for example, by a natural number. Note that the identification code may be assigned a different value each time the model space is defined.

[0049] [Action A5] Next, in action A5, the processor 33, as an assignment unit, assigns the above-mentioned attributes, which are different from the identification code, to multiple elements constituting the object, based on the object information. As mentioned above, the attributes are described in natural language. For example, the processor 33 assigns attributes to elements based on object information and reference information. Specifically, for example, the processor 33 obtains a list of attributes of elements constituting the object (for example, attributes representing each element constituting the "house" shown in Figure 6), which is output by inputting object information and a context representing the entire object to a predefined trained model, identifies the corresponding criteria for reference information from the list, and assigns the corresponding attributes to object information (for example, shape, positional relationship, etc.) that satisfies the criteria. In other words, the processor 33 may assign the output of the trained model (for example, generative AI) regarding each element constituting the object as attributes to the elements constituting the object by inputting information about the object to a predefined trained model that has been trained based on predefined reference information. With such a configuration, when one object contains a large number of elements, the time required to assign attributes to each of those elements can be reduced. In this case, it is preferable for the processor 33 to assign attributes to all elements that make up the object. With such a configuration, the possibility of elements that make up an object that cannot be specified can be reduced.

[0050] In this case, for example, the processor 33 may assign positional attributes to the elements constituting the object based on the positional relationships between the elements constituting the object. Alternatively, the processor 33 may assign attributes according to the boundaries of the elements constituting the object. With such a configuration, information about shape can be assigned as attributes through natural language, allowing elements to be specified more intuitively using language. For example, the processor 33 may assign shape attributes such as squares, triangles, and circles according to the configuration of the loop connection of directed line segments, which are the smallest units of the elements constituting the object.

[0051] Furthermore, when assigning attributes, the processor 33 may acquire semantic information that suggests the meaning of the object and assign attributes to the element based on that semantic information. The semantic information may be metadata representing the object's name, role, function, etc., or it may be data from the specifications of the component represented by the object. In other words, any information that suggests what meaning or role the object has for the user is arbitrary. In this case, the semantic information may include not only the shape of the object, but also the social significance realized by that object. For example, the semantic information may be the standard name of the component represented by the object, the function of the component, or the use of the component.

[0052] Furthermore, the semantic information may include historical information regarding the state changes of elements (including the objects themselves) in the representation space. State changes of elements may include changes in elements over time (for example, not limited to changes in the appearance of an object such as its shape, but may include changes in properties at different layers such as physical properties) and changes in elements due to arbitrary operations on elements (not limited to Boolean operations, but may include any operations such as translation operations, rotation operations, operations that modify elements, operations that reference elements, and topologically isomorphic transformations). Changes in elements over time may be described as a history of operations such as translation and rotation operations. Such historical information can be used, for example, when an object represents a dynamic object such as a machine. In other words, the processor 33 may further assign attributes to multiple elements based on historical information regarding the state changes of elements in the representation space.

[0053] Subsequently, the processor 33 determines whether to terminate the attribute assignment. If it determines to continue the attribute assignment, the processor 33 proceeds to action A7 and executes the update process. The update process updates the attributes of the target element, which is one of several elements, based on the latest attributes of adjacent elements adjacent to the target element. The specific details of the update process will be described later.

[0054] [Action A6] Once the attribute assignment is complete, the process proceeds to action A6. In action A6, the processor 33 executes management processing to manage the elements constituting the object in the namespace corresponding to the assigned attributes in a manner that can be uniquely identified using natural language. With this configuration, since the elements constituting the object can be identified using natural language, for example, a user can intuitively specify the elements of an object placed in the representation space using natural language rather than non-natural language such as an identification number.

[0055] For example, the processor 33 can manage the elements of an object in a hierarchical namespace relating to the attributes of the elements that make up the object, based on the hierarchical structure of the elements that constitute the object. With such a configuration, for example, even if there are multiple elements of the same type of object and the attributes of each individual element of that object are identical, they can be managed in different namespaces based on the hierarchical structure. Therefore, it becomes easier to manage elements of the same type of object. For example, the processor 33 can uniquely identify the modifying relationships of each element that makes up an object representing "house" in a hierarchical namespace, such as "house / floor", "house / wall", "house / roof", "house / entrance", and "house / window".

[0056] As an alternative, the processor 33 may manage the elements constituting an object assigned the same designation attribute in a manner that allows for unique identification using natural language on a namespace based on positional attributes. With such a configuration, for example, when there are objects with the same designation, the elements can be managed uniquely while allowing the user to intuitively specify them. For example, the processor 33 may manage the elements of each "pillar" on a namespace so that for an object assigned the same designation attribute (for example, a "house" with multiple "pillars"), additional positional attributes such as "center pillar," "right pillar," "front pillar," and "pillar closest to the entrance" can be assigned according to their relative positional relationships. In this case, the "center pillar" can be managed hierarchically, for example, as "house / room / pillar / center," and the "right pillar" can be managed, for example, as "house / room / pillar / right." Note that front, back, left, and right can be identified according to the positional relationships of the elements constituting the object, regardless of the coordinate system. For example, in the case of a "house," the direction in which the object corresponding to the "entrance" is positioned from a point in the "room" is defined as "front," the opposite direction of this "front" is defined as "back (rear)," the direction in which the "floor" is located is defined as "down," the direction in which the "roof" is located is defined as "up," and the directions orthogonal to these are defined as "left" and "right." Therefore, the processor 33 can manage the attributes assigned to elements regardless of the coordinate system of the representation space. With this configuration, since the attributes are maintained even when the coordinate system changes, for example, when multiple users share an object, the terminology used to specify the object can be standardized even if the users use different coordinate systems. The coordinate system can be arbitrary, such as the Cartesian coordinate system, cylindrical coordinate system, or polar coordinate system. By iteratively performing the assignment or modification of such attributes, the accuracy of attribute assignment can be further improved.In particular, the processor 33, based on the input of a command statement from the user to a trained model such as a generative AI that includes the attributes of the entire object (especially the naming attribute), causes the generative AI to mechanically output initial values ​​for the attributes of the elements constituting the object, modifies these initial values ​​to match the user's intuition by comparing them with predetermined dictionary data, and can hierarchically manage these attributes so that they can be uniquely identified in the namespace by iteratively generating or modifying such intuition-aligned attributes. With this configuration, it is possible to manage objects using character information in a unified manner with higher accuracy than when simply using a generative AI.

[0057] After that, processor 33 terminates this information processing.

[0058] 2.3. Relationship between attributes and the elements that make up an object Next, we will explain the relationships between the attributes assigned to the elements constituting the object through the above information processing. Figure 11 shows an example of the attributes assigned to the object shown in Figure 5. As shown in Figure 11, each of the loop structures corresponding to the six faces of the rectangular "box" object Ob1 is assigned an attribute that can identify the "face," and these nominal attributes of "face" (which are also shape attributes) are subordinate to the nominal attribute of "box." Also, since there are six overlapping "face" attributes, these faces are assigned unique attributes such as "front," "right face," and "top face" in conjunction with the position attribute, so as not to overlap. In this case, attributes such as "front," "right," and "top (up)" are subordinate to the "faces" belonging to the "box," for example. In addition, the boundary represented by the loop structure, which is an example of an element constituting the object, is assigned the attribute "outer perimeter (edge)." Since this attribute indicates the boundary of a "face," it is subordinate to the "face." Furthermore, the "perimeter" is defined by four directed line segments that constitute a loop structure, and each of these four directed line segments is assigned the attribute of "end". The "end" is an element that constitutes the "face" or "perimeter" and is subordinate to these attributes. Here, since the four "ends" overlap with the "front," each directed line segment is managed in a way that makes it uniquely identifiable using positional attributes such as "bottom end," "top end," "right end," and "left end." Also, each part of the directed line segment can be divided and assigned attributes such as "center," "right part," and "left part." In addition, the X, Y, and Z directions in the model space can also be redefined based on the elements that constitute the object. In other words, the processor 33 may assign attributes to the direction describing the representation space based on the attributes (especially positional attributes) assigned to the elements that constitute the object. Such relationships are not limited to the scope of a single object, but can also exist between different objects separated within the representation space.

[0059] 2.4. Example of an update process Next, we will explain an example of the update process described in Action A7 of Figure 10. Figure 12 is an activity diagram showing an example of the update process in Action A7 of Figure 10.

[0060] [Action A71] As shown in Figure 12, first, in action A71, the processor 33, as an evaluation unit, calculates a confidence score indicating the likelihood of the attribute assigned to each element. The confidence score is information indicating the degree to which the attribute assigned to or considered for assignment to the target element is valid. For example, if the attribute of an element called "closed curve A" is "the upper end of a column extending vertically in the living room of a house," the confidence score is an index that shows how well the positional relationship of the closed curve A with other elements (for example, whether it is inside the area enclosed by the wall that defines the living room of the house, the connection relationship with the element corresponding to the side of the column extending from closed curve A, the relative coordinates of the side of the column with respect to other end faces, etc.) fits with the attributes of other elements. The confidence score may be expressed as a stepped index such as 0 to 100, or as a Boolean variable that takes False when there is a clear contradiction with other elements. The confidence level can be calculated, for example, by comparing the mathematical properties represented by the attributes with the coordinates and shape of the elements using a pre-trained model such as an LLM, or by using various decision formulas in which the likelihood of an attribute is defined as a condition (for example, the angle of inclination relative to the direction associated with the attribute "up and down," or a distance threshold indicating that it is connected).

[0061] Furthermore, the processor 33 may calculate the degree to which the state of the object or the elements constituting the object (e.g., position, orientation, shape, area or volume, curvature, contact state, boundary sharing state, inclusion relationship, etc.) matches the attributes represented in the reference information. Such a determination can be made, for example, by comparing natural language with numerical data using a large-scale language model such as a generative AI.

[0062] [Action A72] Next, in action A72, the processor 33 identifies one of several elements as the target element based on the confidence level calculated in action A71. For example, the processor 33 may preferentially select elements with a confidence level below a predetermined value as the new target element. With this configuration, confidence level information indicating the likelihood of the attribute assigned to each element is calculated, and elements with confidence level information below a predetermined value are preferentially selected as update targets, thereby prioritizing attribute revisions for elements with high uncertainty. Therefore, the convergence efficiency of the entire iterative process and the accuracy of attribute determination can be improved.

[0063] [Action A73] Next, in action A73, the processor 33 identifies adjacent elements to the target element. For example, the processor 33 identifies adjacent elements to the target element based on the relative relationship between the target element and multiple other elements. With this configuration, for newly generated elements, adjacent elements can be identified based on their relative relationship with existing elements, and the attributes of the new element can be updated based on their latest attributes, thereby allowing newly generated elements to be given meaning in light of their surrounding context. Therefore, it is easier to ensure the continuity of meaning in the representation space after the operation. The relative relationship may be a spatial relationship or a relationship described as a mathematical model in terms of field values ​​such as physical quantities. For example, adjacent elements may include at least one of the following: neighboring elements located near the target element, contacting elements that are in contact with the target element, shared elements that share a boundary with the target element, and containing elements that encompass the target element. With this configuration, the relationship with the target element can be captured in various ways. Therefore, flexible attribute updates that do not rely solely on simple contact relationships can be performed. For example, the processor 33 determines whether an element other than the target element is an adjacent element based on whether it satisfies an adjacency determination condition that represents a relative relationship with respect to the target element in the representation space. The adjacency determination condition is defined such that, for example, if at least a part of the other element exists within a predetermined distance range from the target element, then that other element is determined to be adjacent to the target element. Alternatively, the adjacency determination condition may be defined so that when an element and the target element are projected onto a plane having a predetermined direction as their normal vector, at least a part of these elements overlap. Alternatively, the adjacency determination condition may be defined so that it is satisfied when an element is connected to (in other words, overlaps with) the target element. Alternatively, the adjacency determination condition may be defined so that it is satisfied when the boundary of an element and the boundary of the target element are at least partially shared. Alternatively, the adjacency determination condition may be defined so that an element contains (or vice versa) the target element. Alternatively, the adjacency determination condition may be defined based on the update history of past attributes. Elements that satisfy such adjacency determination conditions can be identified as adjacent elements.

[0064] The processor 33 may describe the relationships between multiple elements using a graph network, where each of the elements is a node distinguished by an identification code, and the adjacency relationships between elements are represented by edges, based on various information for describing the relationships between multiple elements, such as the adjacency determination conditions described above. Figure 13 shows an example of the configuration of a graph network G for describing the relationships between elements. As shown in Figure 13, the processor 33 uses each of the multiple elements A to E in the representation space S as a node, and connects nodes that satisfy connection conditions based on their relative positional relationship in the representation space (for example, at least some of the adjacency determination conditions described above) using edges. In this case, there may be an upper limit on the number of edges extending from a given node. In this case, the processor 33 may prioritize edges with high scores for determining whether or not the connection conditions are met. Also, the types of edges may differ depending on the type and number of adjacency determination conditions that are satisfied. For example, edges can include "contact" edges where elements are in contact, "boundary-sharing" edges where elements share a boundary, "neighborhood" edges where elements are not in contact but are located in the same vicinity, "encompassing" edges where one element encloses another, and "history-aware" edges where the relationship changes little with the attribute modification history (in other words, they are temporally adjacent). Such temporal edges, such as history edges, can describe the temporal identity of objects in the representation space as a graph network.

[0065] Furthermore, the processor 33 may use the graph network constructed in this manner to identify neighboring elements to the target element. For example, the processor 33 may identify nodes whose inter-vertex distance from the target element's node is less than or equal to a threshold as neighboring element nodes. If there are multiple types of edges, the processor 33 may calculate the inter-vertex distance using only some of these nodes, or it may weight these edges by type and calculate the inter-vertex distance as a weighted graph. In other words, the above-mentioned adjacency determination conditions can be used to indicate the relationship between the nearest neighbor nodes on the graph network G.

[0066] Figure 14 shows an example of the information associated with nodes and edges used in the graph network G of Figure 13. As shown in Figure 14(a), nodes are associated with, for example, identification codes A to E, the type of element itself (for example, dimensions such as a face), attributes assigned to the element (top face, side face, bottom face, etc.), and the confidence level of these elements. Also, as shown in Figure 14(b), edges are associated with pairs of endpoint nodes, the type of relationship represented by the edge, the weight of the edge, the direction of the edge, and reference information (other) regarding the basis for determining the relationship. In addition, each edge may be associated with reference information describing the relationship of that edge in natural language. The processor 33 may manage this information as a graph network database.

[0067] [Action A74] Returning to Figure 12, in action A74, the processor 33 updates the attributes of the target element based on the latest attributes of the adjacent elements. With this configuration, attributes described in natural language are assigned to each element, and the attributes of the target element are updated based on the latest attributes of the adjacent elements adjacent to the target element. This allows for semantic assignment that reflects the relationship with surrounding elements, compared to evaluating individual elements in isolation. Therefore, the accuracy of semantic identification of elements in the representation space can be improved. For example, the processor 33 may fix the attributes of adjacent elements and output new attributes (candidates) by referring to reference information based on the absolute position of adjacent elements and the target element in the representation space, the attributes of the adjacent elements, the relationship between adjacent elements and the target element, etc., and update the attributes of the target element based on the output result. In this case, it is optional whether or not to use the attributes of the target element itself.

[0068] [Action A75] Subsequently, the processor 33 determines whether the attribute updates of other elements have been completed. If the attribute updates of other elements have not been completed, in action A75, the processor 33 identifies other elements as new target elements. At this time, the processor 33 may identify the next target element from among the adjacent elements adjacent to the target element whose attributes were updated immediately before. This makes it easier to reflect the effects of previous updates in subsequent updates by propagating the attribute updates. Subsequently, in action A73, the processor 33 sequentially identifies target elements from among multiple elements and updates their attributes based on their adjacency relationships. The method for determining whether the attribute updates of other elements have been completed is arbitrary, but for example, the processor 33 may determine that the update is complete when the attributes of a certain number of elements have been updated, or it may determine that the update is complete when the attributes of elements that have an adjacency relationship within a certain range to the initial target element (for example, elements of nodes on the graph network G that are within a certain distance range) have been updated. Alternatively, the processor 33 may determine that the attribute update is complete when the attribute updates of all elements have been performed at least once.

[0069] [Action A76] On the other hand, if it is determined that the updating of attributes of other target elements is complete, the processor 33 executes a convergence determination process in action A76. The convergence determination process is a process for determining whether or not the attribute changes accompanying the above-mentioned updates have converged. For example, in the convergence determination process, the processor 33 may further determine, as a convergence determination unit, that the attributes of an element have converged if, for at least some of the multiple elements, the attribute assigned by the previous update matches the attribute assigned by the latest update, or if the element can be uniquely identified in the namespace, for a predetermined number of consecutive times. With such a configuration, stable attribute determination can be performed while suppressing excessive update processing. This determination means that the change in position in the semantic space of natural language is below an acceptable value. For example, the processor 33 may recalculate the confidence level of each element after updating the attributes, and determine that the attributes have converged if the latest confidence level is within an acceptable range. In other words, the presence or absence of the above-mentioned attribute changes is not limited to natural language elements, but may also be determined based on parameters such as confidence levels.

[0070] Subsequently, if the convergence determination process determines that the attributes have not converged, the process returns to action A71, and the processor 33 recalculates the confidence level of each element based on the latest attributes and updates the attributes. For example, the processor 33 may, as an iterative unit, iteratively perform the following steps: identify one of the adjacent elements as a new target element, and update the attributes of the new target element, so that the attributes of each of the multiple elements converge in a manner that allows the element to be uniquely identified using natural language in the namespace corresponding to the attributes. With such a configuration, it is possible to easily construct a consistent semantic system among multiple elements.

[0071] On the other hand, if it is determined that the attributes have converged, the processor 33 terminates the update process in action A77.

[0072] 2.5. Information processing for performing operations on objects Next, we will describe the information processing required to perform operations on an object using the attributes assigned through the information processing described above. Figure 15 is an activity diagram showing the flow of information processing required to perform operations on an object.

[0073] [Action A11] As shown in Figure 15, first, in action A11, the processor 33 performs an input reception process, and as an input reception unit, it receives input from the user as a command statement in the form of character information to specify the elements that constitute an object. The command statement is defined in natural language (not limited to complete sentences, but may include sequences of words). In other words, in this information processing, the processor 33 accepts character input via CUI. Note that this does not mean that it refuses to accept input via GUI. The command statement may include first character information that can identify an attribute, and second character information that can identify the operation content on the elements that constitute the object. The first character information is configured to identify an attribute that is attached to the object and managed in the namespace, for example, "right column". The first character information can identify elements that constitute multiple objects that can be operated on. The second character information defines the type of operation, for example, and may include combining, deleting, smoothing, changing color, etc.

[0074] The method for inputting commands is not limited to input via a keyboard, which is one example of input unit 35; it may also be voice input via a microphone or the like. In other words, the processor 33 may accept voice input from the user as a voice input unit. In this case, the processor 33 may generate character information based on the voice input and accept the generated character information as a command. With such a configuration, object operations can be performed naturally by voice, thus providing a system that offers more convenient object operations.

[0075] [Action A12] Next, in action A12, the processor 33 performs an operation identification process to identify the elements that constitute the object to be operated on (for example, the object identified by the first character information) and the operation to be performed on the object, based on the received instruction statement. For example, the processor 33 performs natural language processing on the received instruction statement to identify the elements that constitute the object having the attributes represented by the first character information as the object to be operated on (or a candidate for the object to be operated on), and to identify an operation that can perform the operation corresponding to the second character information. Note that any form of instruction statement is not limited to instruction statements containing the first and second character information, as long as the operation can be uniquely identified.

[0076] Here, operations on the object being operated on may include specific operations. A specific operation is one that requires the assignment of a new identification code to the elements constituting the object on which the specific operation is performed. Such operations include, for example, a first operation that generates a new element in the representation space, a second operation that removes part of an existing element from the representation space (in other words, an operation that requires the addition or deletion of nodes in the graph network G described above to the representation space). Specifically, for example, such a specific operation includes an operation based on a Boolean operation on the elements constituting the object. With such a configuration, even if the identification code of an existing object is changed by an operation that adds elements constituting a new object using a Boolean operation (for example, an operation that creates a hole or groove in a plane, or an object composition), the existing object can be specified based on the attributes that were assigned before the specific operation. Boolean operations may include, for example, an union operation that combines two shapes, a difference operation that cuts out one shape from another, and an intersection operation that removes the part where two shapes intersect. Such operations significantly change the topology information of the object because the shapes before and after the operation cannot be reversibly obtained through continuous deformation. Therefore, because new topologies arise with changes in topology information, existing identification codes cannot be reused.

[0077] [Action A13] Next, in action A13, processor 33 executes an operation based on the result of the operation identification process performed in action A12. This allows processor 33 to execute the operation specified by the second character information on an element defining an object belonging to a namespace defined by an attribute identified by the first character information. With this configuration, operations on elements constituting an object intuitively associated by attributes can be intuitively executed using natural language input. Therefore, for example, a more user-friendly CUI can be provided. If the identified operation is a specific operation, processor 33 can execute that specific operation on the elements constituting the object.

[0078] [Action A14] Subsequently, if the operation is a Boolean operation, in action A14, the processor 33 reassigns an identification code to the element with the new topology information. Since the identification code is reassigned regardless of whether the topology has changed, in order to identify the target of the operation in the CUI based on the identification code, it is necessary to keep track of the newly assigned identification code one by one.

[0079] [Action A15] Subsequently, in action A15, the processor 33 assigns attributes to the elements constituting the object based on the object information, so as to maintain the attributes. At this time, the processor 33 may restrict the attributes to be assigned again based on existing attributes so as to maintain the attributes that have already been assigned to the elements constituting the object. In other words, the processor 33 can manage the elements constituting the object in a namespace using common attributes before and after a particular operation in a manner that can be uniquely identified using natural language. With such a configuration, for example, even if it is necessary to assign a new identification code by changing the topology of the object, the elements constituting the object to which attributes have already been assigned can be specified using common natural language before and after a particular operation. Note that the specific manner of this processing is arbitrary as long as the already assigned attributes are maintained, and it is not necessary to use object information or reference information.

[0080] Furthermore, when assigning attributes, the processor 33 may acquire semantic information that suggests the meaning of the object and assign attributes to the element based on that semantic information. The semantic information may be metadata representing the object's name, role, function, etc., or it may be data from the specifications of the component represented by the object. In other words, any information that suggests what meaning or role the object has for the user is arbitrary. In this case, the semantic information may include not only the shape of the object, but also the social significance realized by that object. For example, the semantic information may be the standard name of the component represented by the object, the function of the component, or the use of the component.

[0081] Furthermore, the semantic information may include historical information regarding the state changes of elements (including the objects themselves) in the representation space. State changes of elements may include changes in elements over time (for example, not limited to changes in the appearance of an object such as its shape, but may include changes in properties at different layers such as physical properties) and changes in elements due to arbitrary operations on elements (not limited to Boolean operations, but may include any operations such as translation operations, rotation operations, operations that reference elements, and topologically isomorphic transformations). Changes in elements over time may be described as a history of these operations, such as translation and rotation operations. Such historical information can be used, for example, when an object represents a dynamic object such as a machine. In other words, the processor 33 may further assign attributes to multiple elements based on historical information regarding the state changes of elements in the representation space.

[0082] For example, the processor 33 may associate objects (or their elements) of similar shapes that exist at close coordinates in the representation space before and after an operation as corresponding elements and assign attributes to them. Also, if one of two independent objects A and B is moving towards the other object B through a time change (or translation operation), the processor 33 may take this time change into consideration and assign an attribute to the moving object A that includes the meaning of "an object moving towards object B". Furthermore, if object A is performing vibration or circular motion on its own, the processor 33 may assign an attribute that includes the meaning of "vibrating motion" or "circular motion around a certain object (position)" to describe that motion. On the other hand, the processor 33 may assign the attribute "stationary" to a stationary object B.

[0083] Furthermore, when an object splits (a new object is created) during a time change, such a time change is topologically discontinuous, and a new identification code is assigned, similar to a Boolean operation. At the same time, a new meaning needs to be assigned to the newly created object, and it can be difficult to determine from a single snapshot in the representation space that the two split objects are derived from a single object that existed before. In such cases, the processor 33, based on the history information, can assign an attribute that includes the meaning that the objects split (derived) from the single object, since the single object that existed before the splitting operation was located in the vicinity where the two split objects appeared. Such correspondences of objects during time changes (especially temporal identity) can be represented, for example, as a graph network representing the temporal adjacency relationships described above.

[0084] [Action A20] At this time, if a new attribute is assigned in action A15 (i.e., if an attribute is updated), the processor 33 may perform an update process in action A20. The specific form of the update process is the same as the process of action A7 (see Figure 12) described above. In particular, when a specific operation such as the first operation or the second operation described above is performed, the processor 33 may preferentially identify elements that have been newly added or deleted, etc., and that reflect changes in topology before and after the operation, as target elements, and update the attributes of those target elements. In other words, as an update unit, when the first operation described above is performed, the processor 33 may set one of the new elements as a target element and identify adjacent elements that are adjacent to the target element based on the relative relationship between the target element and multiple elements. In this case, the processor 33 may update the attributes of the target element based on the latest attributes of the identified adjacent elements. With such a configuration, newly generated elements can be given meaning based on their surrounding context by identifying adjacent elements based on their relative relationship with existing elements and updating the attributes of the new elements based on their latest attributes. Therefore, it is easier to ensure the continuity of meaning in the representation space after the operation. From another perspective, when a second operation is performed on the representation space to remove a portion of an existing element, the processor 33 may set one of the elements adjacent to the removed element as the target element, identify adjacent elements adjacent to the target element based on the relative relationship between the target element and the remaining elements, and update the attributes of the target element based on the latest attributes of the identified adjacent elements. With such a configuration, when a portion of an existing element is removed, the remaining element adjacent to the removed portion is set as the target element, and its attributes are updated based on the latest attributes of the surrounding adjacent elements, thereby enabling a reset of meaning that is adapted to the local structural changes after the deletion operation. Therefore, it is easier to maintain the semantic consistency of the element after deletion.Furthermore, when the relationships between elements are described in the graph network G described above, the processor 33, as an update unit, may update the attributes of elements corresponding to nodes within a certain range for the added or deleted nodes when an operation requiring the addition or deletion of nodes in the graph network is performed on the representation space. With this configuration, the relationships between elements are managed in the graph network, and by updating the attributes of elements within a certain range from the point where the node addition or deletion occurred, the affected local area can be efficiently identified and the update target can be limited. Therefore, attribute updates in response to structural changes can be achieved while keeping the processing load down.

[0085] Furthermore, if the processor 33 assigns attributes to multiple elements based on historical information regarding the state changes of elements in the representation space, as described above, it may update the attributes of the target element based on the latest attributes of adjacent elements that were assigned based on the historical information. With such a configuration, it is possible to assign meaning to objects and their elements from temporal changes such as translation operations, for example, thus enabling more realistic and meaningful interpretations. The processor 33 may also use historical information when updating the attributes of the target element. This allows for a more stable convergence of attribute interpretations. In other words, the assignment step and the update step may be integrated as a single continuous process.

[0086] Furthermore, the identification of the target element that serves as the starting point for such processing may be performed independently of changes in topology. For example, the processor 33 may identify the target element based on the change in confidence level due to the operation. Specifically, for example, the processor 33 may preferentially identify as the target element an element whose confidence level has fallen outside the acceptable range due to the operation.

[0087] [Action A16] Subsequently, in action A16, the processor 33 manages the attributes in a namespace. This allows the attributes assigned to the new topology information to be consistently managed in the namespace obtained before the operation and the corresponding namespace (including the same namespace). At this time, the processor 33, as a history management unit, may record the attributes of elements and their correspondences before and after the (specific) operation as history information. In this case, the processor 33, as a management unit, may manage the elements generated or remaining after the operation that triggered the attribute update (e.g., a specific operation) and the elements before the operation in a manner that allows tracking of their semantic identity based on the history information. With such a configuration, by recording the attributes and correspondences of elements before and after the operation as history information, and managing the semantic identity between the generated or remaining elements and the elements before the operation in a traceable manner based on the history information, the semantic continuity of elements can be grasped even after a structural change. Therefore, reference stability across operation histories can be improved. For example, the processor 33 may associate the history of attribute changes with the node and edge information shown in Figure 14, making it possible to track semantic identity.

[0088] [Action A17] Subsequently, in action A17, the processor 33 displays the operation result on the display unit 34. If the operation performed in action A13 is not a specific operation, the processor 33 skips the processing of actions A14 to A16 and executes the processing of action A17.

[0089] After that, processor 33 terminates this information processing.

[0090] 2.6. Changes in topology information for specific operations and management on attribute-based namespaces Next, we will explain the changes in topology information and their management in the namespace based on attributes when a specific operation is performed in the information processing described in the previous section. Figure 16 shows the relationship between the changes in topology information and attributes when a specific operation is performed on the object shown in Figure 11. The specific operation performed here is an operation that includes a difference operation, which creates a round hole on the top surface of the "box" object Ob1. In Figure 16, for the sake of explanation, the object before the specific operation is denoted as Ob1, and the object after the specific operation is denoted as Ob2.

[0091] As shown in Figure 16, the top surface of object Ob1 before the operation is defined using only one loop defined by four directed line segments, whereas the top surface of object Ob2 after the operation has a discontinuous boundary defined by a circle inside that loop. As a result, various elements that constitute objects related to the hole are newly generated, such as a circle separating the hole from the top surface, the side of the hole continuous with the hole, and the space defined by the side.

[0092] The processor 33 assigns new attributes to object information, including elements that constitute the new object generated by such a specific operation, based on attributes managed in an existing namespace. During the update process, the processor 33 iteratively updates the attributes that have been assigned once, taking into account the relationships between them, for the attributes of the entire object (cuboid) and the attributes of the elements that constitute it (faces, line segments, vertices). In this embodiment, as a result of converging the attributes through iterative updates, the processor 33 assigns the attribute "round hole" subordinate to "top face" to a newly generated circle, and assigns the attribute "side" (side of the round hole) subordinate to "round hole" to a face generated as a result of a difference operation extending from the circle. The processor 33 may also manage the new attributes in the namespace in a multi-level hierarchical structure, such as "round hole" in the "center" of the "top face".

[0093] 2.7. Screen Example Next, an example of a screen displayed on the display unit 34 in the information processing for performing operations on the above-mentioned object will be described. Figure 17 is an example of an operation screen displayed on the display unit in the information processing for performing operations on the object. As shown in Figure 17, the operation screen 5 displayed on the display unit 34 may include an object display area 51, a history area 52, and an input area 53.

[0094] The object display area 51 is an area for visually displaying the representation space (e.g., model space) and the objects placed within that representation space. Users can input GUI operations on objects by performing mouse or touch operations on the object display area 51.

[0095] The history area 52 is an area that displays a history of operations performed on an object (which may include not only operations instructed by the user, but also operations performed automatically by the system).

[0096] The input area 53 is an area for the user to input a command. This input can be as text information, for example, "Please make a round hole on the top of the box." The processor 33 executes an operation based on the input content in the input area 53. In this case, "top of the box" corresponds to the first piece of text information, and "Please make a round hole" corresponds to the second piece of text information. In this case, a round hole of appropriate dimensions may be formed in any area of ​​the top of the box. For example, the object Ob2 shown in Figure 16 is displayed in the object display area 51 as an object representing the latest model.

[0097] The input area 53 may include a voice input button 531. When the voice input button 531 is operated, the processor 33 requests voice input using the sound collection device 4 and displays a command statement identified based on the input voice information in the input area 53.

[0098] The processor 33, through the above information processing, assigns natural language attributes that the user can recognize as concepts to the elements constituting an object, and manages these attributes in a namespace according to a conceptual hierarchical structure, thereby providing the user with intuitive CUI operations on a vast number of elements constituting an object.

[0099] In other words, the processor 33 assigns attributes to the elements that make up an object so as to form a projection from the semantic space represented by the namespace to the instance space. This allows the processor 33 to make the namespace corresponding to the assigned attributes function as a semantic space, and to manage the elements that make up the object while preserving relationships, hierarchy, order, symmetry, and so on.

[0100] 3. Others The above embodiment may be modified as follows.

[0101] The processor 33 may accept input regarding attribute changes. For example, the processor 33 may accept a command statement that changes an attribute related to direction or positional relationship, such as "Set the currently largest displayed face to the front of the box." In this case, the processor 33 may update the positional attributes representing up, down, front, back, left, and right, based on the relationship between the attribute of the largest displayed face among the displayed contents in the object display area 51 and the other elements constituting the object relative to that face.

[0102] The above reference information is optional, provided that attributes can be assigned to the elements that make up the object, and may be specified to include generated AI.

[0103] The above information processing is not limited to computer-aided design, but can be applied to any design or simulation, such as disaster evacuation routes, air conditioning simulations, heat conduction, electromagnetic interference, and structural design under specific environmental conditions.

[0104] In the above embodiment, a so-called on-premise configuration was adopted in which the user terminal 3 performs information processing. However, the above information processing may also be performed in a cloud configuration by the information processing device 2 based on the information input to the user terminal 3. As a cloud-based information processing device 2, for example, the above functions and processing may be provided in the form of SaaS (Software as a Service) or cloud computing.

[0105] In the above embodiment, the information processing device 2 performed various storage and control functions, but instead of the information processing device 2, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like.

[0106] The above embodiment is not limited to the information processing system 1, but may also be an information processing method or an information processing program. The information processing method includes each step of the information processing system 1. The program causes at least one computer to execute each step of the information processing system 1.

[0107] The above-mentioned information processing system 1, etc., may be provided in any of the following embodiments.

[0108] (1) An information processing system comprising at least one processor capable of executing a program such that the following steps are performed, wherein in the acquisition step, object information relating to at least one object placed in a representation space is acquired, the representation space is configured to visually represent the object, the object information includes element information relating to a plurality of elements constituting the object, in the assignment step, attributes different from an identification code are assigned to the elements based on the object information, the identification code is a non-natural language code that is mechanically uniquely assigned to each of the plurality of elements when the object is defined in the representation space, the attributes are described in natural language, and in the update step, the attributes of the target element are updated based on the latest attributes of adjacent elements adjacent to the target element, which is one of the plurality of elements.

[0109] With this configuration, attributes described in natural language are assigned to each element, and the attributes of the target element are updated based on the latest attributes of adjacent elements adjacent to the target element. This allows for semantic assignment that reflects the relationships with surrounding elements, compared to evaluating individual elements in isolation. Therefore, the accuracy of semantic identification of elements in the representation space can be improved.

[0110] (2) In the information processing system described in (1) above, the iterative step iteratively performs the following: identifying one of the adjacent elements as a new target element, and performing the update step for the new target element, such that the attributes of each of the plurality of elements converge in a manner that makes it possible to uniquely identify the element using natural language in the namespace corresponding to the attribute.

[0111] This type of configuration makes it easier to construct a consistent semantic system across multiple elements.

[0112] (3) In the information processing system described in (2) above, the convergence determination step further determines that the attributes of an element have converged if, for at least some of the multiple elements, the attribute assigned by the most recent update matches the attribute assigned by the latest update, or if the element can be uniquely identified in the namespace, a predetermined number of consecutive times.

[0113] This configuration allows for stable attribute determination while suppressing excessive update processing.

[0114] (4) In the information processing system described in (2) or (3) above, the evaluation step further calculates a confidence score indicating the likelihood of the attribute assigned to each element, and the update step preferentially identifies elements whose confidence score is less than a predetermined value as the new target elements.

[0115] With this configuration, a confidence score indicating the likelihood of an attribute assigned to each element is calculated, and elements with a confidence score below a predetermined value are prioritized for updating. This allows for prioritizing attribute revisions for elements with high uncertainty. Therefore, the convergence efficiency of the entire iterative process and the accuracy of attribute determination can be improved.

[0116] (5) In the information processing system described in any one of (1) to (4) above, the update step includes setting one of the new elements as the target element when a first operation is performed to generate a new element in the representation space, identifying adjacent elements adjacent to the target element based on the relative relationship between the target element and the plurality of elements, and updating the attributes of the target element based on the latest attributes of the identified adjacent elements.

[0117] With this configuration, newly generated elements can be given meaning based on their surrounding context by identifying adjacent elements based on their relative relationship with existing elements and updating the attributes of the new element based on the latest attributes of those adjacent elements. Therefore, it is easier to ensure the continuity of meaning in the representation space after the operation.

[0118] (6) In the information processing system described in any one of (1) to (5) above, in the update step, if a second operation is performed to remove a portion of existing elements from the representation space, one of the elements adjacent to the removed element is set as the target element, adjacent elements adjacent to the target element are identified based on the relative relationship between the target element and the remaining plurality of elements, and the attributes of the target element are updated based on the latest attributes of the identified adjacent elements.

[0119] With this configuration, if a portion of an existing element is removed, the remaining element adjacent to the removed portion is used as the target element, and its attributes are updated based on the latest attributes of the surrounding adjacent elements. This allows for a re-establishment of meaning that adapts to the local structural changes after the deletion operation. Therefore, it becomes easier to maintain the semantic consistency of the element after deletion.

[0120] (7) An information processing system according to any one of (1) to (6) above, wherein the relationships between the plurality of elements are described using a graph network in which each of the plurality of elements is a node distinguished by the identification code and the adjacency relationships between the elements are edges, and in the update step, when an operation is performed on the representation space that requires the addition or deletion of nodes of the graph network, the system updates the attributes of the elements corresponding to nodes within a certain range for the added or deleted nodes.

[0121] With this configuration, the relationships between elements are managed using a graph network, and attribute updates are performed on elements within a certain range from the point where a node is added or deleted. This allows for efficient identification of the affected local area and limits the scope of updates. Therefore, attribute updates in response to structural changes can be achieved while keeping the processing load down.

[0122] (8) An information processing system according to any one of (5) to (7) above, wherein in the update step, when an operation based on a Boolean operation is performed on the elements constituting the object, the system identifies the target element based on the change in topology due to the Boolean operation, identifies adjacent elements adjacent to the target element based on the relative relationship between the target element and the plurality of elements, and updates the attributes of the target element based on the latest attributes of the identified adjacent elements.

[0123] With this configuration, by applying the aforementioned operation to an operation based on Boolean operations, even when the element configuration changes due to shape generation, excision, joining, etc., attribute updates using adjacency relationships can be performed on the changed elements. Therefore, the continuity of meaning after Boolean operations can be improved.

[0124] (9) An information processing system according to any one of (1) to (8) above, wherein in the assignment step, the attribute is further assigned based on historical information relating to the state change of the element in the representation space, and in the update step, the attribute of the target element is updated based on the latest attribute of the adjacent element assigned based on the historical information.

[0125] With this configuration, it becomes possible to assign meaning to objects and their elements based on temporal changes such as translational operations, thus enabling more realistic and meaningful interpretations.

[0126] (10) An information processing system according to any one of (1) to (9) above, wherein the object includes an object object defined as an object in the representation space and a field object that describes a field of physical quantities in the representation space.

[0127] With this configuration, in addition to object objects defined as physical objects in the representation space, field objects that describe the field of physical quantities are treated as objects, allowing geometric objects and physical quantity distributions to be managed within a common framework. Therefore, the scope of application for semantic assignment and update processing can be extended.

[0128] (11) An information processing system according to any one of (1) to (10) above, wherein the adjacent element includes at least one of the following: a neighboring element located near the target element, a contacting element in contact with the target element, a shared element that shares a boundary with the target element, and an inclusion element that encompasses the target element.

[0129] This configuration allows for capturing relationships with target elements in various ways. Therefore, it enables flexible attribute updates that do not rely solely on simple contact relationships.

[0130] (12) An information processing method comprising each step of the information processing system described in any one of (1) to (11) above.

[0131] (13) A program that causes at least one computer to perform each step of the information processing system described in any one of (1) to (11) above. Of course, this is not always the case.

[0132] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]

[0133] 1: Information Processing System 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Processor 3: User terminal 30: Communications bus 31: Communications Department 32: Storage section 33: Processor 34:Display section 35: Input section 4: Sound collection device 5: Operation screen 51: Object display area 52: History area 53: Input area 531: Voice input button DS: Data Structure IF1~IF4: Reference Information Ob1, Ob2: Objects G: Graph Network

Claims

1. An information processing system, The system comprises at least one processor capable of executing a program so that each of the following steps is performed, In the acquisition step, object information about at least one object placed in the representation space is acquired. The aforementioned representation space is configured to visually represent the object, The object information includes element information relating to multiple elements that constitute the object, In the assignment step, based on the object information, an attribute different from the identification code is assigned to the element, The identification code is a non-natural language code that is mechanically uniquely assigned to each of the multiple elements when the object is defined in the representation space. The aforementioned attributes are described in natural language, In the update step, the system updates the attributes of a target element, which is one of the aforementioned multiple elements, based on the latest attributes of adjacent elements adjacent to the target element.

2. In the information processing system described in claim 1, A system that, in an iterative step, iteratively performs the following steps: identify one of the adjacent elements as a new target element, and perform the update step on the new target element, such that the attributes of each of the multiple elements converge in a manner that makes it possible to uniquely identify the element using natural language in the namespace corresponding to the attribute.

3. In the information processing system described in claim 2, Furthermore, in the convergence determination step, the system determines that the attributes of an element have converged if, for at least some of the multiple elements, the attribute assigned by the previous update matches the attribute assigned by the latest update, or if the element can be uniquely identified in the namespace, a predetermined number of consecutive times.

4. In the information processing system described in claim 2, Furthermore, in the evaluation step, the confidence level indicating the likelihood of the attribute assigned to each element is calculated, The system, in the update step, prioritizes identifying elements whose confidence level is less than a predetermined value as the new target elements.

5. In the information processing system described in claim 1, In the aforementioned update step, When a first operation is performed to generate new elements in the representation space, one of the new elements is set as the target element. Based on the relative relationship between the target element and the plurality of elements, adjacent elements adjacent to the target element are identified. A system that updates the attributes of a target element based on the latest attributes of the identified adjacent elements.

6. In the information processing system described in claim 1, In the aforementioned update step, When a second operation is performed on the representation space to remove a portion of the existing elements, one of the elements adjacent to the removed element is set as the target element. Based on the relative relationship between the target element and the remaining plurality of elements, adjacent elements adjacent to the target element are identified. A system that updates the attributes of a target element based on the latest attributes of the identified adjacent elements.

7. In the information processing system described in claim 1, The relationships between the aforementioned multiple elements are described using a graph network in which each of the multiple elements is a node distinguished by the identification code, and the adjacency relationships between the elements are edges. The update step includes updating the attributes of the elements corresponding to nodes within a certain range for the added or deleted nodes when an operation is performed on the representation space that requires the addition or deletion of nodes in the graph network.

8. In the information processing system described in claim 5, In the aforementioned update step, When an operation based on a Boolean operation is performed on the elements constituting the object, the target element is identified based on the change in topology caused by the Boolean operation. Based on the relative relationship between the target element and the plurality of elements, adjacent elements adjacent to the target element are identified. A system that updates the attributes of a target element based on the latest attributes of the identified adjacent elements.

9. In the information processing system described in claim 1, In the aforementioned assignment step, the attribute is further assigned based on historical information relating to the state changes of the element in the representation space, The system updates the attributes of the target element in the update step based on the latest attributes of the adjacent elements assigned based on the historical information.

10. In the information processing system described in claim 1, The object is a system that includes an object object defined as an object in the representation space and a field object that describes a field of physical quantities in the representation space.

11. In the information processing system described in claim 1, The adjacent element is a system in which at least one of the following is included: a neighboring element located near the target element, a contacting element in contact with the target element, a shared element that shares a boundary with the target element, and an inclusive element that encompasses the target element.

12. Information processing method, A method comprising each step of the information processing system described in any one of claims 1 to 11.

13. It is a program, A program that causes at least one computer to perform each step of the information processing system described in any one of claims 1 to 11.

Images