Information processing systems, information processing methods, and programs
The system addresses the challenge of specifying elements in a CUI by assigning natural language attributes, facilitating intuitive and efficient object management and operations.
Patent Information
- Application Number
- JP2025077163
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2026-11-19
- Estimated Expiration
- 2045-05-07
AI Technical Summary
Users find it difficult to specify elements in a representation space using character user interfaces (CUI) due to mechanically assigned identification codes not aligning with their intuitive understanding of the elements' context, such as shape, function, and relationships.
An information processing system that assigns attributes to elements based on object information and reference information, using natural language, allowing unique identification and management in a namespace.
Enables intuitive specification and management of elements using natural language, improving user convenience and efficiency in operations on objects.
Smart Images

Figure 2026181671000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an information processing system, an information processing method, and a program.
Background Art
[0002] Patent Document 1 discloses a technique that can facilitate data management in a design and manufacturing support system. In the design and manufacturing support system according to the technique, which is composed of a design system (CAD system), an evaluation system (CAE system), and a production management system, the association between CAD data files and components is defined in a bill of materials, and the granularity of data used in each system (the content of data contained in one data file) is unified. This discloses that CAD data can be automatically registered from the design system to the evaluation system and the production management system without processing.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] Incidentally, the elements that make up an object placed in a representation space are uniquely distinguished by mechanically assigned identification codes, and in some cases, users can perform operations on the specified element by non-verbally specifying that element through a graphical user interface (GUI) in that representation space. However, some users may want to complete operations related to operations on elements using a character user interface (CUI). However, since users assign meaning to the elements that make up an object placed in a representation space through various contexts such as shape, function, and relationships with other elements, and recognize elements through this meaning, it has been difficult to specify elements identified by mechanically assigned identification codes through a CUI. [Means for solving the problem]
[0005] According to one aspect of the present invention, an information processing system is provided, comprising at least one processor capable of executing a program such that the following steps are performed: an acquisition step acquires object information relating to at least one object placed in a representation space for visually representing an object, the object information includes information relating to the elements constituting the object; an assignment step assigns attributes different from an identification code to the elements constituting the object based on the object information and predetermined reference information, the attributes are described in natural language, and the reference information indicates the correspondence between each attribute and the state of the object's elements; and a management step manages the elements constituting the object in a namespace corresponding to the attributes in a manner that allows for unique identification using natural language.
[0006] With this configuration, it is possible to identify the elements that make up an object using natural language. For example, a user can intuitively specify the elements of an object placed in the representation space using natural language, rather than using non-natural language such as identification numbers. [Brief explanation of the drawing]
[0007] [Figure 1] This is a diagram showing the configuration of Information Processing System 1. [Figure 2] This is a block diagram showing the hardware configuration of the information processing device 2. [Figure 3] This is a block diagram showing the hardware configuration of user terminal 3. [Figure 4] This figure shows an example of the data structure of the model data. [Figure 5] This is a diagram used to show the relationships between the elements that make up an object. [Figure 6] This figure shows an example of reference information IF1 that includes the naming attribute. [Figure 7] This figure shows an example of reference information IF2 that includes location attributes. [Figure 8] This figure shows an example of reference information IF3 that includes shape attributes. [Figure 9] This figure shows an example of reference information IF4 that includes appearance attributes. [Figure 10] This is an activity diagram showing an example of the information processing flow for assigning attributes, which is performed in Information Processing System 1. [Figure 11] This figure shows an example of attributes assigned to the object shown in Figure 5. [Figure 12] This diagram shows the activity flow for performing operations on an object. [Figure 13] This figure shows the relationship between changes in topology information and attributes when a specific operation is performed on the object shown in Figure 11. [Figure 14] This is an example of an operation screen displayed on the display unit in information processing for performing operations on an object. [Modes for carrying out the invention]
[0008] Embodiments of the present invention will be described below with reference to the drawings. The various features shown in the embodiments below can be combined with each other.
[0009] Incidentally, the program for implementing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or it may be provided as a downloadable medium from an external server, or it may be provided so that the program is launched on an external computer and its functions are realized on a client terminal (so-called cloud computing).
[0010] Furthermore, in various information processing according to one embodiment, an input and an output corresponding to the input can be realized. Here, as long as an output is obtained as a result of the input, the form of the information referenced in such information processing (hereinafter referred to as "reference information") is not limited. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression equation constructed by a statistical method), or a trained model that has been pre-trained to learn the correlation between input and output, or a generative AI such as a large-scale language model that can output a desired result by inputting a prompt (these models include parameters that construct the correlation relationship between input and output) or a visual language model.
[0011] Furthermore, in one embodiment, "part" may include, for example, hardware resources implemented by a circuit in a broad sense, and the information processing of software that can be specifically realized by these hardware resources. Also, in one embodiment, various types of information are handled, and this information can be represented, for example, by the physical values of signal values representing voltage and current, the high or low values of signal values as a set of binary bits composed of 0s or 1s, or by quantum superposition (so-called qubits), and communication and calculations can be performed on a circuit in a broad sense.
[0012] Furthermore, a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit (Circuit), circuitry (Circuitry), a processor (Processor), a memory (Memory), and the like. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an Application Specific Integrated Circuit (ASIC), a programmable logic device (for example, a Simple Programmable Logic Device (SPLD), a Complex Programmable Logic Device (CPLD), a Field Programmable Gate Array (FPGA)), and the like.
[0013] 1. Hardware Configuration This section describes the hardware configuration.
[0014] <Information Processing System 1> Figure 1 is a configuration diagram illustrating the information processing system 1. The information processing system 1 includes an information processing apparatus 2 and a user terminal 3. The information processing apparatus 2 and the user terminal 3 are configured to be capable of communicating via a telecommunication line. In one embodiment, the information processing system 1 is composed of one or more apparatuses or components. For example, if the system consists only of the information processing apparatus 2, the information processing system 1 may be the information processing apparatus 2. Hereinafter, these components will be described.
[0015] <Information Processing Apparatus 2> Figure 2 is a block diagram showing the hardware configuration of the information processing apparatus 2. The information processing apparatus 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23, and these components are electrically connected to one another inside the information processing apparatus 2 via the communication bus 20. Each of the components will be further described.
[0016] The communication unit 21 preferably uses wired communication methods such as USB, IEEE1394, Thunderbolt®, and wired LAN network communication, but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, and Bluetooth® communication as needed. In other words, it is more preferable to implement it as a collection of these multiple communication methods. That is, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.
[0017] The memory unit 22 stores various types of information as defined above. This can be implemented, for example, as a storage device such as a solid-state drive (SSD) that stores various programs related to the information processing device 2 executed by the processor 23, or as memory such as random access memory (RAM) that stores temporarily necessary information (arguments, arrays, etc.) related to program calculations. The memory unit 22 stores various programs and variables related to the information processing device 2 executed by the processor 23.
[0018] For example, the memory unit 22 stores the data structure DS and reference information IF1 to IF4, which will be described later.
[0019] The processor 23 performs processing and control of the overall operation related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading predetermined programs stored in the memory unit 22. That is, information processing by software stored in the memory unit 22 can be concretely realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being a single unit, and may be implemented with multiple processors 23 for each function, or a combination thereof.
[0020] The processor 23 is configured as an acquisition unit to acquire information from the user terminal 3 or other devices. The processor 23 is configured to acquire various information by reading various information stored in the storage area, which is at least a part of the memory unit 22, and writing the read information to the work area, which is at least a part of the memory unit 22. The storage area is, for example, the area of the memory unit 22 that is implemented as a storage device such as an SSD. The work area is, for example, the area that is implemented as memory such as RAM. The acquisition by the processor 23 includes acquiring the output results of each functional unit included in the processor 23.
[0021] The processor 23 is configured as a display processing unit to display various information. This information can be presented to the user via the display unit 34 of the user terminal 3 or other devices, as described later. In such cases, for example, the processor 23 controls the display unit 34 of the user terminal 3 to display visual information such as screens, images including still images or videos, icons, and messages. The processor 23 may generate only rendering information for displaying the visual information on the user terminal 3. The processor 23 may also present the outputted information to the user without going through the user terminal 3 or other devices.
[0022] <User Terminal 3> Figure 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 comprises a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35. These components are electrically connected within the user terminal 3 via the communication bus 30. The explanation of the communication unit 31, the storage unit 32, and the processor 33 is the same as the explanation of each part in the information processing device 2 and is therefore omitted.
[0023] The display unit 34 may be included in the user terminal 3 housing or it may be an external component. The display unit 34 displays a user-operable GUI screen. This is preferably done by using different display devices such as a CRT display, liquid crystal display, organic EL display, and plasma display, depending on the type of user terminal 3.
[0024] The input unit 35 is configured to accept input from the user. The input unit 35 may be included in the casing of the user terminal 3 or it may be an external component. For example, the input unit 35 may be integrated with the display unit 34 and implemented as a touch panel. If it is a touch panel, the user can input tap operations, swipe operations, etc. Of course, instead of a touch panel, a switch button, mouse, QWERTY keyboard, voice recognition device, gesture detection device, gaze detection device, biosignal detection device, imaging device, etc. may be used. In other words, the input unit 35 accepts operation input made by the user. In response, the input unit 35 transmits a signal corresponding to the operation input to the processor 33 via the communication bus 30. The processor 33 can then perform predetermined controls and calculations as needed.
[0025] <Sound collection device 4> The information processing system 1 may further include a sound collection device 4. The sound collection device 4 is a so-called microphone configured to convert external sounds into signals. The sound collection device 4 may be directly connected to the user terminal 3 or connected to the information processing device 2. Alternatively, the sound collection device 4 may be built into the user terminal 3.
[0026] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in the memory of the user terminal and does not necessarily need to be stored non-volatilely in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via the network.
[0027] The sound collection device 4 is not particularly limited, but it collects sound in the human audible range, specifically between 20 Hz and 20,000 Hz, and converts it into an electrical signal. The sound may be recorded in mono or stereo. When digitally processing the sound data, the sampling rate may be, for example, 48,000 Hz, 44,100 Hz, 32,000 Hz, 22,050 Hz, 16,000 Hz, 11,025 Hz, 11,000 Hz, 8,000 Hz, etc. Any of the values exemplified here may be used. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, thereby improving the accuracy of speech recognition.
[0028] Furthermore, the data collected by the sound collection device 4 may be compressed as appropriate by the processor 33 of the user terminal 3, and the compression format may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. Compression can reduce communication traffic caused by data transfer from the user terminal to the information processing device 2.
[0029] 3. Regarding information processing This section describes the information processing performed in the aforementioned information processing system 1. As an example, this information processing is configured to make objects placed in the representation space uniquely identifiable using natural language.
[0030] 3.1. Data and Data Structure First, the various data and data structures used in this information processing will be described. These data and data structures may be stored in the storage unit 22, the processor 33, or in an external database, external server, or the like.
[0031] <Model Data> Model data is data used to represent a model. The model may be a spatial model such as a three-dimensional, two-dimensional, or one-dimensional spatial model, a hyperdimensional spatial model of four dimensions or more, a spatiotemporal model that includes a time axis in addition to spatial degrees of freedom, or a model that includes other degrees of freedom (for example, field degrees of freedom such as electromagnetic fields or thermal fields). The model data itself may be binary data configured to be decoded based on the data structure DS described above, or it may be managed to explicitly have the data structure DS described later. A model can be represented by at least one object. Therefore, model data is an example of object information relating to at least one object placed in the representation space. The representation space is a space for visually representing objects, and can be described using parameters relating to degrees of freedom that can be represented in the representation space, such as position coordinates that represent spatial positions. As described above, the representation space may be an N-dimensional Euclidean space, a Minkowski space that includes a time axis, a complex space, or a space described by quaternions, and may include any space in which a state can be described.
[0032] <Object Information> Object information includes information about the elements that make up the object. These elements can include, for example, zero-dimensional elements that describe the object (e.g., points, especially vertices), one-dimensional elements (e.g., line segments, curves, arcs, circles, etc.), two-dimensional elements (e.g., planes and surfaces), and three-dimensional elements (e.g., cubes, spheres, etc.). These elements of different dimensions are managed in relation to one another. These relationships will be described later.
[0033] <Data structure> Figure 4 shows an example of the data structure of model data. As shown in Figure 4, the model data has a data structure DS called a drawing database. The drawing database is configured to read the model data as a data structure. The data structure DS includes, for example, a block table. The block table is configured to describe the model data mutually in different types of representation spaces, such as model space, paper space, and block definitions that can be arbitrarily defined by various users. The model space is configured to represent the model data as a three-dimensional solid (for example, a CG model such as a polygon). The paper space is configured to represent the model data as at least one two-dimensional drawing data. The model represented by the model space may be configured to describe elements of other dimensions based on the relationships between one-dimensional elements, such as multiple line segments (first line segment, second line segment, third line segment, etc.), arcs, and circles, as elements that constitute the model. In addition, the elements that constitute an object may include the object itself, which is the whole set. In other words, the elements that make up an object may include a portion of the three-dimensional region that constitutes the object, at least a portion of the two-dimensional region that constitutes the three-dimensional region, or at least a portion of the one-dimensional region that constitutes the two-dimensional region. Furthermore, object information may include information about the boundaries of the elements that make up the object.
[0034] <Relationships between elements> Figure 5 is a diagram illustrating the relationships between the elements that make up an object. As shown in Figure 5, object Ob1 is a three-dimensional solid in the shape of a rectangular parallelepiped and has six faces. Each face is enclosed by four edges, each edge described as a line segment connecting two vertices. Here, object Ob1 treats the four edges as directed line segments and describes other elements using the order relationships of each directed line segment. Each directed line segment can be described, for example, by the relative position coordinates of its start and end points. An order relationship from the first directed line segment to the second directed line segment can be defined by connecting the end point of the first directed line segment to the start point of the second directed line segment. When multiple such directed line segments form a closed loop, the closed surface bounded by this loop can be described as a two-dimensional element, a continuous surface (especially a plane). Furthermore, the adjacency of faces can be described by the relationship between two directed line segments describing two different faces. Furthermore, vertices can be formed by assigning the identifier "Vertex" to the connection relationship between two directed line segments. Curves can be formed by connecting infinitesimal directed line segments. Surfaces can be formed by smoothly connecting infinitesimal surfaces.
[0035] <Identification code> Each element constituting these objects is assigned an identification code. The identification code is described by a non-natural language code (e.g., a number or a random string) that uniquely identifies the element constituting the object, and is called, for example, an entity ID. The processor 23 can accept the user's specification of an element constituting an object through non-textual input operations (e.g., touch operations, tap operations, click operations, etc.) on the GUI displayed on the display unit 34, identify the identification code of the element to be operated on according to the input operation, and execute the operation. However, the identification code itself is determined independently of the context of the element (e.g., randomly). Context refers to the meaning of the elements constituting the object as perceived or intended by the user (e.g., common name, function, part, appearance, relative positional relationship from the user's perspective, etc.). Therefore, by allowing the user to identify the elements constituting the object to be operated on using text input (especially natural language input), it may be possible to perform more efficient operations on the object through the CUI.
[0036] <Reference Information> Reference information is information for assigning attributes to the elements that make up an object, allowing them to be managed under a predetermined namespace, and it shows the correspondence between each attribute and the state of the object's elements. An attribute is a code that represents the "context" mentioned above, and is written, for example, in natural language. Attributes may include naming attributes related to the naming of the object or the elements that make up the object. Figure 6 shows an example of reference information IF1 that includes naming attributes. The reference information IF1 shown in Figure 6 hierarchically represents the concept of a structure called "house". For example, reference information IF1 indicates that the structure called "house" is associated with the highest level of hierarchy (for example, the hierarchy that represents an entire set of objects), and that the house includes (or is composed of) walls, a roof, a front door, windows, pillars, and rooms. Reference information IF1 is specified to show the correspondence between these natural language identifiers and the elements that make up the object included in the imported model. In other words, reference information can be associated with naming attributes corresponding to each element that makes up an object. With such a configuration, natural naming can be used when specifying an object, thus improving the convenience for users when specifying objects. Reference information IF1 is configured to allow the assignment of naming attributes to each object included in the imported model by associating it with model data that has these naming attributes pre-assigned to the positional relationships of each element included in a house (for example, walls extend from the floor, the roof is placed on the floor, the entrance is located on one of the walls, etc.) and to the elements that make up multiple sample objects representing a "house".
[0037] Furthermore, attributes may include positional attributes that describe the positional relationships within or between elements constituting an object using natural language. Figure 7 shows an example of reference information IF2 that includes positional attributes. As shown in Figure 7, reference information IF2 may include positional relationships such as center, front, back, left, right, top, bottom, outer edge, inner edge, etc. For example, a partial element contained within an element (e.g., face A) can be expressed using reference information IF2 as "the central part of face A," "the front part of face A," "the outer edge of face A," "the inner edge of face A," etc. Each of these parts may be assigned a non-natural language identifier. Each of these positional attributes may be associated with a predetermined determination formula (e.g., coordinate comparison between objects). Positional attributes can describe the positional relationships between elements constituting an object (e.g., one object is located to the left of another object). Note that the positional relationships within elements constituting an object can also be expressed as the positional relationships between those elements.
[0038] Furthermore, attributes may include shape information that describes the shape of the elements constituting an object using natural language. Figure 8 shows an example of reference information IF3 that includes shape attributes. As shown in Figure 8, reference information IF3 can describe shape attributes using modifying natural language clauses that represent the shape, such as "flat," "wide," "angular," "curved," and "bent." These shape attributes may be defined in association with conditions that assign the attribute, such as determination conditions based on the connection relationships of line segment elements (for example, indicating "angular" if the angle between line segment elements is greater than or equal to a default value).
[0039] Furthermore, attributes may include appearance attributes relating to the appearance of the elements that make up an object. Figure 9 shows an example of reference information IF4 that includes appearance attributes. Appearance attributes may include, for example, attributes that represent colors such as "red," "yellow," "blue," and "green," attributes that represent patterns such as "striped," "checkerboard," and "triangular grid," and attributes that represent textures such as "smooth," "rough," and "rough." These attributes may be defined in association with discrimination criteria based on texture information assigned to the object (e.g., RGB values, material texture type, etc.).
[0040] Thus, reference information can be configured to hierarchically represent the relative relationships of the elements that make up an object (for example, the "wall" of the "house," the "left part" of the "floor" of the "house," etc.).
[0041] The specific form of the reference information may be a table linking "terms" and "judgment conditions" as described above, a hierarchical network model describing knowledge, or a pre-trained model (especially a generative AI model). A pre-trained model can be learned, for example, by machine learning using a training dataset that includes model data describing objects to which natural language attributes, different from the identification code, have been assigned in advance.
[0042] 3.2. Flow of information processing for assigning attributes Figure 10 is an activity diagram showing an example of the flow of information processing for assigning attributes, which is performed in Information Processing System 1. Note that this information processing may include arbitrary exception handling not shown. Exception handling includes interrupting the information processing or omitting individual processes. The selections or inputs performed in this information processing may be based on user operation or performed automatically without user operation.
[0043] [Action A1] As shown in Figure 10, first, in action A1, the processor 33 accepts model data input from the user. Note that the model data input can be existing model data created by the user, or model data from the template model to be generated.
[0044] [Action A2] Next, in action A2, processor 33 defines the model space as a representation space based on the model data. This virtually provides a space where objects can be placed.
[0045] [Action A3] Next, in action A3, the processor 33, acting as an acquisition unit, acquires object information relating to at least one object placed in the representation space for visually representing the object. If the model data is blank model data with no objects placed on it, the processor 33 may appropriately acquire information about objects to be newly placed in the model space as object information.
[0046] [Action A4] Next, in action A4, processor 33 assigns an identification code to each element constituting the acquired object. The identification code may be defined, for example, by a natural number. Note that the identification code may be assigned a different value each time the model space is defined.
[0047] [Action A5] Next, in action A5, the processor 33, as an assignment unit, assigns the above-mentioned attributes, which are different from the identification code, to the elements constituting the object, based on the object information and predefined reference information. As described above, the attributes are described in natural language. For example, the processor 33 can obtain a list of attributes of the elements constituting the object (for example, attributes representing each element constituting the "house" shown in Figure 6) which is output by inputting object information and a context representing the entire object to a predefined trained model, identify the corresponding criteria for the reference information from the list, and assign the corresponding attributes to the object information (for example, shape, positional relationship, etc.) that satisfies the criteria. In other words, by inputting information about the object to the processor 33 and a trained model trained based on predefined reference information, the output of the trained model (for example, a generative AI) regarding each element constituting the object may be assigned as attributes to the elements constituting the object. With such a configuration, when one object contains a large number of elements, the time required to assign attributes to each of those elements can be reduced. In this case, it is preferable for the processor 33 to assign attributes to all of the elements constituting the object. This configuration reduces the possibility of elements constituting an object that cannot be specified.
[0048] In this case, for example, the processor 33 may assign positional attributes to the elements constituting the object based on the positional relationships between the elements constituting the object. Alternatively, the processor 33 may assign attributes according to the boundaries of the elements constituting the object. With such a configuration, information about shape can be assigned as attributes through natural language, allowing elements to be specified more intuitively using language. For example, the processor 33 may assign shape attributes such as squares, triangles, and circles according to the configuration of the loop connection of directed line segments, which are the smallest units of the elements constituting the object.
[0049] Subsequently, the processor 33 determines whether to terminate the attribute assignment. If it determines to continue assigning attributes, the processor 33 continues processing action A5 and iteratively assigns or modifies attributes. For example, the processor 33 may assign a higher-level naming attribute (in other words, a higher-level conceptual naming) to the elements constituting the object, and then assign lower-level naming attributes belonging to that higher-level naming attribute. After that, the processor 33 may assign naming attributes up to a predetermined hierarchy, and then assign position attributes, etc., in association with the naming attributes at each hierarchy. The processor 33 may also assign new attributes in response to user input.
[0050] [Action A6] Once the attribute assignment is complete, the process proceeds to action A6. In action A6, the processor 33 executes management processing to manage the elements constituting the object in the namespace corresponding to the assigned attributes in a manner that can be uniquely identified using natural language. With this configuration, since the elements constituting the object can be identified using natural language, for example, a user can intuitively specify the elements of an object placed in the representation space using natural language rather than non-natural language such as an identification number.
[0051] For example, the processor 33 can manage the elements of an object in a hierarchical namespace relating to the attributes of the elements that make up the object, based on the hierarchical structure of the elements that constitute the object. With such a configuration, for example, even if there are multiple elements of the same type of object and the attributes of each individual element of that object are identical, they can be managed in different namespaces based on the hierarchical structure. Therefore, it becomes easier to manage elements of the same type of object. For example, the processor 33 can uniquely identify the modifying relationships of each element that makes up an object representing "house" in a hierarchical namespace, such as "house / floor", "house / wall", "house / roof", "house / entrance", and "house / window".
[0052] As an alternative, the processor 33 may manage the elements constituting an object assigned the same designation attribute in a manner that allows for unique identification using natural language on a namespace based on positional attributes. With such a configuration, for example, when objects with the same designation exist, the user can intuitively specify elements while managing them uniquely by their positional relationships. For example, the processor 33 may manage the objects of each "pillar" on a namespace so that for objects assigned the same designation attribute (for example, multiple "pillars" in a "house"), additional positional attributes such as "center pillar," "right pillar," "front pillar," and "pillar closest to the entrance" are assigned according to their relative positional relationships. In this case, the "center pillar" may be managed hierarchically, for example, as "house / room / pillar / center," and the "right pillar" as "house / room / pillar / right." Note that front, back, left, and right can be identified according to the positional relationships of the elements constituting the object, regardless of the coordinate system. For example, in the case of a "house," the direction in which the object corresponding to the "entrance" is positioned from a point in the "room" is defined as "front," the opposite direction of this "front" is defined as "back (rear)," the direction in which the "floor" is located is defined as "down," the direction in which the "roof" is located is defined as "up," and the directions orthogonal to these are defined as "left" and "right." Therefore, the processor 33 can manage the attributes assigned to elements regardless of the coordinate system of the representation space. With this configuration, since the attributes are maintained even when the coordinate system changes, for example, when multiple users share an object, the terminology used to specify the object can be standardized even if the users use different coordinate systems. The coordinate system can be arbitrary, such as the Cartesian coordinate system, cylindrical coordinate system, or polar coordinate system. By iteratively performing the assignment or modification of such attributes, the accuracy of attribute assignment can be further improved.In particular, the processor 33, based on the input of a command statement from the user to a trained model such as a generative AI that includes the attributes of the entire object (especially the naming attribute), causes the generative AI to mechanically output the initial values of the attributes of the elements constituting the object. By comparing these initial values with predetermined dictionary data, the processor modifies them to attributes that align with the user's intuition. By iteratively generating or modifying such intuitive attributes, the processor can hierarchically manage the attributes so that they can be uniquely identified in the namespace. With this configuration, it is possible to manage objects using character information with higher accuracy and in a unified manner than when simply using a generative AI.
[0053] After that, processor 33 terminates this information processing.
[0054] 3.3. Relationship between attributes and the elements that make up an object Next, we will explain the relationships between the attributes assigned to the elements constituting the object through the above information processing. Figure 11 shows an example of the attributes assigned to the object shown in Figure 5. As shown in Figure 11, each of the loop structures corresponding to the six faces of the rectangular "box" object Ob1 is assigned an attribute that can identify the "face," and these nominal attributes (which are also shape attributes) called "faces" are subordinate to the nominal attribute called "box." Also, since the six "face" attributes are duplicated, these faces are assigned attributes such as "front," "right face," and "top face" in conjunction with the position attribute to ensure that they are uniquely determined and do not overlap. In this case, attributes such as "front," "right," and "top (up)" are subordinate to the "faces" belonging to the "box," for example. In addition, the boundary represented by the loop structure, which is an example of an element constituting the object, is assigned the attribute called "outer perimeter (edge)." Since this attribute indicates the boundary of a "face," it is subordinate to the "face." Furthermore, the "perimeter" is defined by four directed line segments that constitute a loop structure, and each of these four directed line segments is assigned the attribute of "end". The "end" is an element that constitutes the "face" or "perimeter" and is subordinate to these attributes. Here, the four "ends" overlap with the "front," so each directed line segment is managed in a way that makes it uniquely identifiable using positional attributes such as "bottom end," "top end," "right end," and "left end." Also, each directed line segment can be divided and assigned attributes such as "center," "right part," and "left part." In addition, the X, Y, and Z directions in the model space can also be redefined based on the elements that constitute the object. In other words, the processor 33 may assign attributes to the directions that describe the representation space based on the attributes (especially positional attributes) assigned to the elements that constitute the object.
[0055] 3.4. Information processing for performing operations on objects Next, we will describe the information processing required to perform operations on an object using the attributes assigned through the information processing described above. Figure 12 is a diagram showing the activity flow for performing operations on an object.
[0056] [Action A11] As shown in Figure 12, first, in action A11, the processor 33 performs an input reception process, and as an input reception unit, it receives input from the user as a command statement in the form of character information to specify the elements that constitute an object. The command statement is defined in natural language (not limited to complete sentences, but may include sequences of words). In other words, in this information processing, the processor 33 accepts character input via CUI. Note that this does not mean that it refuses to accept input via GUI. The command statement may include first character information that can identify an attribute and second character information that can identify the operation content on the elements that constitute the object. The first character information is configured to identify an attribute that is attached to the object and managed in the namespace, for example, "right column". The first character information can select elements that constitute multiple objects that can be operated on. The second character information defines the type of operation, for example, and may include combining, deleting, smoothing, changing color, etc.
[0057] The method for inputting commands is not limited to input via a keyboard, which is one example of input unit 35; it may also be voice input via a microphone or the like. In other words, the processor 33 may act as a voice input unit and accept voice input from the user. In this case, the processor 33 may generate character information based on the voice input and accept the generated character information as a command. With such a configuration, object operations can be performed naturally by voice, thus providing a system that enables more convenient object operations.
[0058] [Action A12] Next, in action A12, the processor 33 performs an operation identification process to identify the elements that constitute the object to be operated on (for example, the object identified by the first character information) and the operation to be performed on the object, based on the received instruction statement. For example, the processor 33 performs natural language processing on the received instruction statement to identify the elements that constitute the object having the attributes represented by the first character information as the object to be operated on (or a candidate for the object to be operated on), and identifies an operation that can perform the operation corresponding to the second character information. Note that any form of instruction statement is not limited to instruction statements containing the first and second character information, as long as the operation can be uniquely identified. Here, the operation on the object to be operated on may include an identification operation. An identification operation is an operation that requires the process of assigning a new identification code to the elements that constitute the object on which the identification operation was performed, and includes, for example, an operation based on a Boolean operation on the elements that constitute the object. With such a configuration, even if the identification code of an existing object is changed by an operation that adds elements that constitute a new object using a Boolean operation (for example, an operation that creates holes or grooves in a plane, or an object composition), the existing object can be specified based on the attributes that were assigned before the identification operation. Boolean operations may include, for example, union operations that combine two shapes, difference operations that cut one shape out of another, and multiplication operations that remove the intersection of two shapes. Such operations significantly alter the object's topology because the shapes before and after the operation cannot be reversibly obtained through continuous deformation. As a result, a new topology is created due to the change in topology, making it impossible to reuse existing identification codes.
[0059] [Action A13] Next, in action A13, processor 33 executes an operation based on the result of the operation identification process performed in action A12. This allows processor 33 to execute the operation specified by the second character information on an element defining an object belonging to a namespace defined by an attribute identified by the first character information. With this configuration, operations on elements constituting an object intuitively associated by attributes can be intuitively executed using natural language input. Therefore, for example, a more user-friendly CUI can be provided. If the identified operation is a specific operation, processor 33 can execute that specific operation on the elements constituting the object.
[0060] [Action A14] Subsequently, if the operation is a Boolean operation, in action A14, processor 33 reassigns an identification code to processor 33 with the new topology information. Since the identification code is reassigned regardless of whether the topology has changed or not, in order to identify the target of the operation using the CUI based on the identification code, it is necessary to keep track of the newly assigned identification code one by one.
[0061] [Action A15] Subsequently, in action A15, processor 33 assigns attributes to the elements constituting the object based on object information and reference information, so as to maintain the attributes. At this time, processor 33 may restrict the attributes to be assigned again based on existing attributes so as to maintain the attributes that have already been assigned to the elements constituting the object. In other words, processor 33 can manage the elements constituting the object in a namespace using common attributes before and after a particular operation in a manner that can be uniquely identified using natural language. With such a configuration, for example, even if it is necessary to assign a new identification code by changing the topology of the object, the elements constituting the object to which attributes have already been assigned can be specified using common natural language before and after a particular operation. Note that the specific manner of this processing is arbitrary as long as the already assigned attributes are maintained, and it is not necessary to use object information or reference information.
[0062] [Action A16] Next, in action A16, processor 33 manages the attributes in a namespace. This allows the attributes assigned to the new topology information to be consistently managed in the namespace obtained before the operation and the corresponding namespace (including the same namespace).
[0063] [Action A17] Subsequently, in action A17, the processor 33 displays the operation result on the display unit 34. If the operation performed in action A13 is not a specific operation, the processor 33 skips the processing of actions A14 to A16 and executes the processing of action A17.
[0064] After that, processor 33 terminates this information processing.
[0065] 3.5. Changes in topology information for specific operations and management on attribute-based namespaces Next, we will explain the changes in topology information and their management in the namespace based on attributes when a specific operation is performed in the information processing described in the previous section. Figure 13 shows the relationship between the changes in topology information and attributes when a specific operation is performed on the object shown in Figure 11. The specific operation performed here is an operation that includes a difference operation, which creates a round hole on the top surface of the "box" object Ob1. In Figure 13, for the sake of explanation, the object before the specific operation is denoted as Ob1, and the object after the specific operation is denoted as Ob2.
[0066] As shown in Figure 13, the top surface of object Ob1 before the operation is defined using only one loop defined by four directed line segments, whereas the top surface of object Ob2 after the operation has a discontinuous boundary defined by a circle inside that loop. As a result, various elements that constitute objects related to the hole are newly generated, such as a circle separating the hole from the top surface, the side of the hole continuous with the hole, and the space defined by the side.
[0067] The processor 33 assigns new attributes to object information, including elements that constitute the new object generated by such a specific operation, based on attributes managed in an existing namespace. In this embodiment, the processor 33 assigns the attribute "round hole" subordinate to "top surface" to the newly generated circle, and assigns the attribute "side surface" (side surface of the round hole) subordinate to "round hole" to the surface generated as a result of a difference operation extending from the circle. The processor 33 may also manage the new attributes in the namespace in a multi-level hierarchical structure, such as "round hole" in the "center" of the "top surface".
[0068] 3.6. Screen Example Next, an example of a screen displayed on the display unit 34 in the information processing for performing operations on the above-mentioned object will be described. Figure 14 is an example of an operation screen displayed on the display unit in the information processing for performing operations on the object. As shown in Figure 14, the operation screen 5 displayed on the display unit 34 may include an object display area 51, a history area 52, and an input area 53.
[0069] The object display area 51 is an area for visually displaying the representation space (e.g., model space) and the objects placed within that representation space. Users can input GUI operations on objects by performing mouse or touch operations on the object display area 51.
[0070] The history area 52 is an area that displays a history of operations performed on an object (which may include not only operations instructed by the user, but also operations performed automatically by the system).
[0071] The input area 53 is an area for the user to input a command statement. This input can be as text information, for example, "Please make a round hole on the top of the box." The processor 33 executes an operation based on the input content in the input area 53. In this case, "top of the box" corresponds to the first piece of text information, and "Please make a round hole" corresponds to the second piece of text information. In this case, a round hole of appropriate dimensions may be formed in any area of the top of the box. For example, the object Ob2 shown in Figure 13 is displayed in the object display area 51 as an object representing the latest model.
[0072] The input area 53 may include a voice input button 531. When the voice input button 531 is operated, the processor 33 requests voice input using the sound collection device 4 and displays a command statement identified based on the input voice information in the input area 53.
[0073] The processor 33, through the above information processing, assigns natural language attributes that the user can recognize as concepts to the elements constituting an object, and manages these attributes in a namespace according to a conceptual hierarchical structure, thereby providing the user with intuitive GUI operations on a vast number of elements constituting an object.
[0074] In other words, the processor 33 assigns attributes to the elements that make up an object so as to form a projection from the semantic space represented by the namespace to the instance space. This allows the processor 33 to make the namespace corresponding to the assigned attributes function as a semantic space, and to manage the elements that make up the object while preserving relationships, hierarchy, order, symmetry, etc.
[0075] 4. Others The above embodiment may be modified as follows.
[0076] The processor 33 may accept input regarding attribute changes. For example, the processor 33 may accept a command statement that changes an attribute related to direction or positional relationship, such as "Set the currently largest displayed face to the front of the box." In this case, the processor 33 may update the positional attributes representing up, down, front, back, left, and right, based on the relationship between the attribute of the largest displayed face among the displayed contents in the object display area 51 and the other elements constituting the object relative to that face.
[0077] The above reference information is optional, provided that attributes can be assigned to the elements that make up the object, and may be specified to include generated AI.
[0078] The above information processing is not limited to computer-aided design, but can be applied to any design or simulation, such as disaster evacuation routes, air conditioning simulations, heat conduction, electromagnetic interference, and structural design under specific environmental conditions.
[0079] In the above embodiment, a so-called on-premise configuration was adopted in which the user terminal 3 performs information processing. However, the above information processing may also be performed in a cloud configuration by the information processing device 2 based on the information input to the user terminal 3. As a cloud-based information processing device 2, for example, the above functions and processing may be provided in the form of SaaS (Software as a Service) or cloud computing.
[0080] In the above embodiment, the information processing device 2 performed various storage and control functions, but instead of the information processing device 2, multiple external devices may be used. That is, various information and programs may be stored in a distributed manner across multiple external devices using blockchain technology or the like.
[0081] The above embodiment is not limited to the information processing system 1, but may also be an information processing method or an information processing program. The information processing method includes each step of the information processing system 1. The program causes at least one computer to execute each step of the information processing system 1.
[0082] The above-mentioned information processing system 1, etc., may be provided in any of the following embodiments.
[0083] (1) An information processing system comprising at least one processor capable of executing a program such that the following steps are performed, wherein in the acquisition step, object information relating to at least one object placed in a representation space for visually representing an object is acquired, the object information includes information relating to the elements constituting the object, and in the assignment step, attributes different from an identification code are assigned to the elements constituting the object based on the information relating to the object and predetermined reference information, the attributes are described in natural language, and the reference information indicates the correspondence between each attribute and the state of the elements of the object. In the management step, the elements constituting the object are managed in a manner that allows for unique identification using natural language in a namespace corresponding to the attributes.
[0084] With this configuration, it is possible to identify the elements that make up an object using natural language. For example, a user can intuitively specify the elements of an object placed in the representation space using natural language, rather than using non-natural language such as identification numbers.
[0085] (2) In the information processing system described in (1) above, the reference information is configured to hierarchically represent the relative relationships of the elements constituting the object, and in the management step, the elements of the object are managed in a hierarchical namespace relating to the attributes of the elements constituting the object, based on the hierarchical structure of the elements constituting the object.
[0086] With this configuration, for example, even if multiple elements of the same type of object exist and the attributes of each individual element of that object are identical, they can be managed in different namespaces based on the hierarchical structure. Therefore, it becomes easier to manage elements of the same type of object.
[0087] (3) In the information processing system described in (1) or (2) above, the attribute includes a naming attribute relating to the naming of an object represented by the object or the elements constituting the object, and the reference information is associated with the naming attribute corresponding to each of the elements constituting the object.
[0088] This configuration allows for the use of natural terminology when specifying objects, thereby improving the user's convenience when selecting objects.
[0089] (4) In the information processing system described in (3) above, the attribute includes a position attribute that describes the positional relationship among the elements constituting the object, or the positional relationship between the elements constituting the object, using natural language, and in the management step, the system further assigns the position attribute to the elements constituting the object based on the positional relationships between the elements constituting the object, thereby managing the elements constituting the object that have been assigned the same naming attribute in a manner that allows them to be uniquely identified in a namespace using natural language based on the position attribute.
[0090] With this configuration, for example, when objects with the same name exist, they can be uniquely managed based on their positional relationship, while the user can intuitively specify the element.
[0091] (5) An information processing system according to any one of (1) to (4) above, wherein the object information includes information relating to the boundaries of the elements constituting the object, and in the assignment step, the attributes are assigned according to the boundaries of the elements constituting the object.
[0092] With this configuration, information about the shape can be assigned as attributes through natural language, allowing elements to be specified more intuitively using language.
[0093] (6) An information processing system according to any one of (1) to (5) above, wherein the management step manages the attributes assigned to the elements regardless of the coordinate system of the representation space.
[0094] With this configuration, attributes are maintained even when the coordinate system changes. For example, when multiple users share an object, the terminology used to specify the object can be standardized even if the users use different coordinate systems.
[0095] (7) An information processing system according to any one of (1) to (6) above, wherein the specific operation step is to perform a specific operation on the elements constituting the object, the specific operation is an operation that requires the process of newly assigning the identification code that has been assigned to the elements constituting the object on which the specific operation was performed, and the management step is to manage the elements constituting the object in a namespace using the common attribute before and after the specific operation in a manner that can be uniquely identified using natural language.
[0096] With this configuration, for example, even when it is necessary to assign a new identification code by changing the topology of an object, the elements that make up an object to which attributes have already been assigned can be specified using a common natural language before and after a particular operation.
[0097] (8) In the information processing system described in (7) above, the specific operation includes an operation based on a Boolean operation on the elements constituting the object.
[0098] With this configuration, even if the identification code of an existing object is changed by an operation that adds elements to a new object using Boolean operations (for example, an operation that creates holes or grooves in a plane, or an operation that combines objects), the existing object can still be identified based on the attributes that were assigned before the particular operation.
[0099] (9) An information processing system according to any one of (1) to (8) above, wherein in the input reception step, the system receives input of character information from a user for specifying the elements constituting the object, the character information is defined in natural language and includes first character information that can identify the attribute and second character information that can identify the content of an operation on the elements constituting the object, and in the operation step, the system causes the system to perform the operation specified by the second character information on the element that defines the object belonging to the namespace defined by the attribute specified by the first character information.
[0100] With this configuration, operations on the elements constituting an object, which are intuitively mapped by attributes, can be performed intuitively using natural language input. Therefore, for example, a more user-friendly command-line interface (CUI) can be provided.
[0101] (10) In the information processing system described in (9) above, the system further includes a voice input step in which voice input is received from the user, and an input reception step in which character information is generated based on the voice input.
[0102] With this configuration, object manipulation can be performed naturally using voice commands, thus providing a system that enables more convenient object manipulation.
[0103] (11) In an information processing system described in any one of (1) to (10) above, the assignment step involves inputting information about the object into a trained model trained based on predetermined reference information, thereby assigning the output of the AI for each element constituting the object as an attribute to the element constituting the object.
[0104] This configuration reduces the time required to assign attributes to each element when an object contains a large number of elements.
[0105] (12) An information processing system according to any one of (1) to (11) above, wherein the elements constituting the object include a part of the three-dimensional region constituting the object, at least a part of the two-dimensional region constituting the three-dimensional region, or at least a part of the one-dimensional region constituting the two-dimensional region.
[0106] (13) An information processing method comprising each step of an information processing system described in any one of (1) to (12) above.
[0107] (14) A program that causes at least one computer to perform each step of the information processing system described in any one of (1) to (12) above. Of course, this is not always the case.
[0108] Finally, while various embodiments relating to this disclosure have been described, these are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims and their equivalents. [Explanation of symbols]
[0109] 1: Information Processing System 2: Information Processing Device 20: Communications bus 21: Communications Department 22: Storage section 23: Processor 3: User terminal 30: Communications bus 31: Communications Department 32: Storage section 33: Processor 34: Display section 35: Input section 4: Sound collection device 5: Operation screen 51: Object display area 52: History area 53: Input area 531: Voice input button DS: Data Structure IF1~IF4: Reference Information Ob1, Ob2: Objects
Claims
1. An information processing system, The system comprises at least one processor capable of executing a program so that each of the following steps is performed, In the acquisition step, object information is obtained about at least one object that is placed in a representation space for visually representing an object. The object information includes information about the elements that constitute the object, In the assignment step, based on information about the object and predefined reference information, attributes different from the identification code are assigned to the elements constituting the object. The aforementioned attributes are described in natural language, The aforementioned reference information is a system that shows the correspondence between the state of the elements of the object for each attribute. In the management step, the elements constituting the object in the namespace corresponding to the attribute are managed in a manner that can be uniquely identified using natural language.
2. In the information processing system described in claim 1, The aforementioned reference information is configured to hierarchically represent the relative relationships of the elements constituting the object. The management step involves a system that manages the elements of an object in a hierarchical namespace relating to the attributes of the elements constituting the object, based on the hierarchical structure of the elements constituting the object.
3. In the information processing system described in claim 1, The aforementioned attributes include a naming attribute relating to the naming of an object represented by the object or an element constituting the object, The system is such that the aforementioned reference information is associated with the aforementioned naming attribute corresponding to each of the elements that constitute the object.
4. In the information processing system described in claim 3, The aforementioned attributes include positional attributes that describe the positional relationships among the elements constituting the object, or the positional relationships between the elements constituting the object, using natural language. The management step further involves assigning the positional attribute to the elements constituting the object based on the positional relationships between the elements constituting the object, thereby managing the elements constituting the object that have been assigned the same naming attribute in a manner that allows them to be uniquely identified in a namespace using natural language based on the positional attribute.
5. In the information processing system described in claim 1, The object information includes information regarding the boundaries of the elements that constitute the object. The system, in the aforementioned assignment step, assigns the attributes according to the boundaries of the elements constituting the object.
6. In the information processing system described in claim 1, The management step involves a system that manages the attributes assigned to the elements, regardless of the coordinate system of the representation space.
7. In the information processing system described in claim 1, In the specific operation step, a specific operation is performed on the elements that constitute the object. The aforementioned specific operation is an operation that requires the process of newly assigning the identification code that has been assigned to the elements constituting the object on which the specific operation was performed. The management step involves a system that manages the elements constituting the object in a namespace using the common attribute before and after the specific operation, in a manner that allows for unique identification using natural language.
8. In the information processing system described in claim 7, The system includes operations based on Boolean operations on the elements constituting the object.
9. In the information processing system described in claim 1, In the input acceptance step, the system accepts input of character information from the user to specify the elements constituting the object, wherein the character information is defined in natural language and includes first character information that can identify the attribute and second character information that can identify the operation performed on the elements constituting the object. In the operation step, the system causes the system to perform an operation specified by the second character information on an element that defines the object belonging to the namespace defined by the attribute specified by the first character information.
10. In the information processing system described in claim 9, Furthermore, in the voice input step, voice input from the user is received. The system generates the character information based on the voice input in the input reception step.
11. In the information processing system described in claim 1, In the aforementioned assignment step, the system assigns the output of the trained model for each of the elements constituting the object as attributes to the elements constituting the object, by inputting information about the object into a trained model that has been trained based on predefined reference information.
12. In the information processing system described in claim 1, A system in which the elements constituting the object include a part of the three-dimensional region constituting the object, at least a part of the two-dimensional region constituting the three-dimensional region, or at least a part of the one-dimensional region constituting the two-dimensional region.
13. Information processing method, A method comprising each step of the information processing system according to any one of claims 1 to 12.
14. It is a program, A program that causes at least one computer to perform each step of the information processing system described in any one of claims 1 to 12.
Citation Information
Patent Citations
Design / manufacturing support system, design data management method, and design data management program
JP2007193665A
