Information processing system, information processing method and program

Abstract

To use natural language to operate on the elements that make up an object placed in a representation space. [Solution] The information processing system has at least one processor capable of executing a program to perform each of the following steps: in the acquisition step, object information about at least one object placed in a representation space for visually representing the object is acquired, the object information including information about the elements that make up the object; in the assignment step, attributes different from identification codes are assigned to the elements that make up the object based on information about the object and predefined reference information; the attributes are described in natural language; and the reference information indicates the correspondence between each attribute and the state of the object's elements.

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 discloses a technology that can facilitate data management in a design and manufacturing support system. In the design and manufacturing support system, which is comprised of a design system (CAD system), an evaluation system (CAE system), and a production management system, the association between CAD data files and parts is defined in a parts list, and the granularity of the data used in each system (the content of data contained in one data file) is unified. This enables CAD data to be registered automatically from the design system to the evaluation system and production management system without any processing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-193665 Summary of the Invention [Problem to be solved by the invention]

[0004] The elements that make up an object placed in a representation space are uniquely distinguished by mechanically assigned identification codes, and in some cases, users can perform operations on the specified elements by non-verbally specifying the elements in the representation space through a graphical user interface (GUI). However, some users prefer to complete operations related to element manipulation using a character user interface (CUI). However, because users assign meaning to the elements that make up an object placed in a representation space through various contexts such as shape, function, and relationships with other elements, and recognize elements through these meanings, it has been difficult to specify elements identified by mechanically assigned identification codes through a CUI. [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided an information processing system including at least one processor capable of executing a program to perform the following steps: in the acquisition step, object information regarding at least one object arranged in a representation space for visually representing the object is acquired, the object information including information regarding elements that constitute the object; and in the assignment step, The object information and predetermined reference information, attributes different from the identification code are assigned to the elements that make up the object, the attributes being described in a natural language, and the reference information is Elements that make up the object The correspondence between the state of and in the management step, the elements constituting the object in the namespace corresponding to the attribute are managed in a manner that allows them to be uniquely identified using natural language. system is provided .

[0006] With this configuration, the elements that make up an object can be identified using natural language, so that, for example, a user can intuitively specify the elements of an object placed in the representation space through natural language rather than using non-natural language such as an identification number. [Brief explanation of the drawings]

[0007] [Figure 1]1 is a configuration diagram illustrating an information processing system 1. FIG. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of an information processing device 2. [Figure 3] FIG. 2 is a block diagram showing the hardware configuration of a user terminal 3. [Figure 4] FIG. 2 is a diagram illustrating an example of a data structure of model data. [Figure 5] FIG. 2 is a diagram showing the relationship between elements that make up an object. [Figure 6] FIG. 10 is a diagram showing an example of reference information IF1 including a pronunciation attribute. [Figure 7] FIG. 10 is a diagram showing an example of reference information IF2 including a location attribute. [Figure 8] FIG. 10 is a diagram showing an example of reference information IF3 including a shape attribute. [Figure 9] FIG. 10 is a diagram showing an example of reference information IF4 including appearance attributes. [Figure 10] 1 is an activity diagram showing an example of the flow of information processing for assigning attributes, which is executed in the information processing system 1. FIG. [Figure 11] FIG. 6 is a diagram showing an example of attributes assigned to the object shown in FIG. 5. [Figure 12] FIG. 10 is a diagram showing an activity indicating the flow of information processing for executing an operation on an object. [Figure 13] 12 is a diagram showing the relationship between the change in topology information and the attributes when a specific operation is performed on the object shown in FIG. 11. FIG. [Figure 14] 10 is an example of an operation screen displayed on a display unit in information processing for executing an operation on an object. DETAILED DESCRIPTION OF THE INVENTION

[0008] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other.

[0009] Incidentally, the program for realizing the software appearing in one embodiment may be provided as a non-transitory computer-readable medium, or may be provided so that it can be downloaded from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0010] Furthermore, various information processing according to an embodiment may realize input and output corresponding to the input. Here, the form of information referenced in such information processing (hereinafter referred to as reference information) is not limited as long as an output is obtained as a result of the input. The reference information may be, for example, rule-based information such as a database, a lookup table, or a predetermined function (including a decision formula such as a regression formula constructed using a statistical method), a trained model that has previously learned the correlation between input and output, or a generative AI such as a large-scale language model (these models include parameters that establish the correlation between input and output) or a visual language model that can output a desired result in response to a prompt.

[0011] In one embodiment, a "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In one embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculations can be performed on a circuit in the broad sense.

[0012] Furthermore, a circuit in the broad sense is a circuit realized by at least an appropriate combination of a circuit, circuitry, processor, memory, etc. The processor may be a general-purpose processor or a dedicated circuit. That is, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration This section explains the hardware configuration.

[0014] <Information Processing System 1> FIG. 1 is a configuration diagram showing an information processing system 1. The information processing system 1 includes an information processing device 2 and a user terminal 3. The information processing device 2 and the user terminal 3 are configured to be able to communicate with each other via a telecommunications line. In one embodiment, the information processing system 1 is made up of one or more devices or components. For example, if the information processing system 1 is made up of only the information processing device 2, the information processing system 1 can be the information processing device 2. These components will be described below.

[0015] <Information processing device 2> 2 is a block diagram showing the hardware configuration of the information processing device 2. The information processing device 2 includes a communication bus 20, a communication unit 21, a storage unit 22, and a processor 23, and these components are electrically connected via the communication bus 20 inside the information processing device 2. Each component will be further described below.

[0016] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as 3G / LTE / 5G, BLUETOOTH (registered trademark) communication, etc. as needed. In other words, it is more preferable to implement it as a collection of multiple communication means. In other words, the information processing device 2 may communicate various information from the outside via the communication unit 21 and the network.

[0017] The storage unit 22 stores various pieces of information defined above. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the information processing device 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the program operations. The storage unit 22 stores various programs, variables, etc. related to the information processing device 2 executed by the processor 23.

[0018] For example, the storage unit 22 stores a data structure DS and reference information IF1 to IF4, which will be described later.

[0019] The processor 23 processes and controls the overall operations related to the information processing device 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the information processing device 2 by reading out predetermined programs stored in the storage unit 22. In other words, information processing by software stored in the storage unit 22 is specifically realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23. These will be described in more detail in the next section. Note that the processor 23 is not limited to being single, and multiple processors 23 may be provided for each function. A combination of these may also be used.

[0020] The processor 23 is configured to function as an acquisition unit and acquire information from the user terminal 3 or another device. The processor 23 is configured to be able to acquire various pieces of information by reading out various pieces of information stored in a storage area that is at least a part of the memory unit 22 and writing the read information into a working area that is at least a part of the memory unit 22. The storage area is, for example, an area of ​​the memory unit 22 that is implemented as a storage device such as an SSD. The working area is, for example, an area that is implemented as a memory such as a RAM. Note that acquisition by the processor 23 includes acquiring output results from each functional unit included in the processor 23.

[0021] The processor 23 is configured to display various types of information as a display processing unit. The information can be presented to the user via a display unit 34 of the user terminal 3 (described later) or another device. In such a case, for example, the processor 23 controls the display unit 34 of the user terminal 3 to display visual information such as a screen, an image including a still image or a video, an icon, or a message. The processor 23 may generate only rendering information for displaying the visual information on the user terminal 3. Note that the processor 23 may present the output information to the user without going through the user terminal 3 or another device user.

[0022] <User terminal 3> 3 is a block diagram showing the hardware configuration of the user terminal 3. The user terminal 3 includes a communication bus 30, a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35, and these components are electrically connected via the communication bus 30 inside the user terminal 3. The description of the communication unit 31, the storage unit 32, and the processor 33 is omitted because they are the same as the description of each unit in the information processing device 2.

[0023] The display unit 34 may be included in the housing of the user terminal 3 or may be externally attached. The display unit 34 displays a GUI screen that can be operated by the user. This is preferably implemented by selectively using display devices such as a CRT display, a liquid crystal display, an organic EL display, or a plasma display depending on the type of user terminal 3.

[0024] The input unit 35 is configured to be able to accept input from a user. The input unit 35 may be included in the housing of the user terminal 3 or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. The touch panel allows the user to input tapping, swiping, and the like. Of course, instead of a touch panel, a switch button, a mouse, a QWERTY keyboard, a voice recognition device, a gesture detection device, a gaze detection device, a biosignal detection device, an imaging device, and the like may be used. That is, the input unit 35 accepts an operation input made by the user. In response, the input unit 35 transfers a signal corresponding to the operation input to the processor 33 via the communication bus 30. The processor 33 can execute predetermined control and calculations as necessary.

[0025] <Sound collection device 4> The information processing system 1 may further include a sound collection device 4. The sound collection device 4 is a so-called microphone configured to be able to convert external sounds into signals. The sound collection device 4 may be directly connected to the user terminal 3 or may be connected to the information processing device 2. The sound collection device 4 may also be built into the user terminal 3.

[0026] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal and does not have to be stored non-volatilely in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the information processing device 2 via a network.

[0027] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sound may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48,000 Hz, 44,100 Hz, 32,000 Hz, 22,050 Hz, 16,000 Hz, 11,025 Hz, 11,000 Hz, 8,000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. Increasing the sampling rate allows for more precise discretization of the temporal timing of the sound, improving the accuracy of voice recognition.

[0028] Furthermore, the data collected by the sound collection device 4 may be appropriately compressed by the processor 33 of the user terminal 3, and the compression format at this time may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. Compression can reduce communication traffic due to data transfer from the user terminal to the information processing device 2.

[0029] 3. Information Processing This section describes information processing executed in the above-mentioned information processing system 1. As an example, this information processing is configured to enable an object placed in a representation space to be uniquely identified using natural language.

[0030] 3.1. Data and Data Structure First, various data and data structures used in this information processing will be described. These data and data structures may be stored in the storage unit 22, the processor 33, or an external database, an external server, or the like.

[0031] <Model data> Model data is data used to represent a model. The model may be a spatial model such as a three-dimensional space model, a two-dimensional space model, or a one-dimensional space model; a hyperdimensional space model of four or more dimensions; a space-time model including a time axis in addition to spatial degrees of freedom; or a model including other degrees of freedom (e.g., field degrees of freedom such as electromagnetic fields or thermal fields). The model data itself may be binary data configured to be decoded based on the data structure DS, or may be managed to explicitly include the data structure DS described below. A model may be represented by at least one object. Therefore, model data is an example of object information regarding at least one object placed in a representation space. The representation space is a space for visually representing an object and may be described using parameters related to the degrees of freedom representable in the representation space, such as position coordinates representing a spatial position. As described above, the representation space may be an N-dimensional Euclidean space, a Minkowski space including a time axis, a complex space, a space described by quaternions, or any space capable of describing a state.

[0032] <Object information> Object information includes information about the elements that make up the object. The elements that make up the object can include, for example, zero-dimensional elements that describe the object (e.g., points, especially vertices), one-dimensional elements (e.g., line segments, curves, arcs, circles, etc.), two-dimensional elements (e.g., planes and curved surfaces), and three-dimensional elements (e.g., cubes, spheres, etc.). These elements with different dimensions are managed in association with each other. The relationships between them will be described later.

[0033] <Data Structure> FIG. 4 is a diagram showing an example of a data structure of model data. As shown in FIG. 4, the model data has a data structure DS called a drawing database. The drawing database is configured to read model data as a data structure. The data structure DS includes, for example, a block table. The block table is configured to enable model data to be described interactively in different types of representation spaces, such as model space, paper space, and block definitions that can be arbitrarily defined by various users. The model space is configured to represent the model data as a three-dimensional solid (e.g., a CG model such as a polygon). The paper space is configured to represent the model data as at least one two-dimensional drawing data. A model represented in the model space can be configured to describe elements of other dimensions based on the relationships between one-dimensional elements such as multiple line segments (a first line segment, a second line segment, a third line segment, etc.), arcs, and circles as elements constituting the model. Furthermore, the elements constituting an object can include the object itself, which is a universal set. In other words, the elements that make up the object may include a portion of a three-dimensional region that makes up the object, at least a portion of a two-dimensional region that makes up the three-dimensional region, or at least a portion of a one-dimensional region that makes up the two-dimensional region. Also, the object information may include information about the boundaries of the elements that make up the object.

[0034] <Relationships between elements> FIG. 5 is a diagram illustrating the relationships between elements that constitute an object. As shown in FIG. 5, object Ob1 is a three-dimensional rectangular parallelepiped solid with six faces. Each face is surrounded by four edges, and each edge is described as a line segment connecting two vertices. Here, object Ob1 treats the four edges as directed line segments, and other elements are described using the order relationship of each directed line segment. Each directed line segment can be described, for example, by the relative position coordinates of its start point and end point. The end point of a first directed line segment can be connected to the start point of a second directed line segment, thereby defining the order relationship from the first directed line segment to the second directed line segment. When such multiple directed line segments form a closed loop, the closed surface bounded by the loop can be described as a two-dimensional element called a continuous surface (particularly a plane). Furthermore, adjacent surfaces can be described by associating two directed line segments that describe two different surfaces with each other. A vertex can be formed by assigning an identifier called a "vertex" to the connection relationship between two directed line segments. A curve can be formed by connecting infinitesimal directed line segments. A curved surface can be formed by smoothly connecting infinitesimal faces.

[0035] <Identification code> Each element constituting these objects is assigned an identification code. The identification code is written using non-natural language codes (e.g., numbers or random character strings) that can uniquely identify the element constituting the object, and is called, for example, an entity ID. The processor 23 accepts the designation of an element constituting the object through a user's non-character input operation (e.g., touch operation, tap operation, click operation, etc.) on the GUI displayed on the display unit 34, identifies the identification code of the element to be operated in accordance with the input operation, and can execute the operation. However, the identification code itself is determined (e.g., randomly) regardless of the context of the element. The context is the meaning of the element constituting the object that the user recognizes or intends (e.g., general name, function, location, appearance, relative positional relationship from the user's perspective, etc.). Therefore, by allowing the user to identify the element constituting the object to be operated using character input (e.g., natural language input), it may be possible to perform more efficient operations on the object through the CUI.

[0036] <Reference information> The reference information is information for assigning attributes that allow elements constituting an object to be managed under a specified namespace, and indicates the correspondence between each attribute and the state of the object's elements. An attribute is a code representing the aforementioned "context" and is described, for example, in natural language. The attribute may include a nomenclature attribute related to the nomenclature of an object or an object represented by an element constituting the object. FIG. 6 is a diagram illustrating an example of reference information IF1 including nomenclature attributes. The reference information IF1 shown in FIG. 6 hierarchically represents the concept of a structure called a "house." For example, the reference information IF1 indicates that a structure called a "house" is associated with the highest level (e.g., a level representing an entire group of objects) and that the house includes (or is composed of) walls, a roof, a front door, windows, pillars, and rooms. The reference information IF1 is defined to indicate the correspondence between these natural language identifiers and the elements constituting the object included in the imported model. In other words, the reference information may be associated with nomenclature attributes corresponding to each element constituting the object. This configuration allows users to use natural nomenclature when specifying an object, thereby improving convenience when specifying objects. Reference information IF1 is configured to be able to assign pronoun attributes to each of the objects included in the imported model by associating the positional relationships of each element included in a house (e.g., walls extend from the floor, the roof is located on the floor, the entrance is located on one of the walls, etc.) and the elements that make up multiple sample objects representing a "house" with model data to which these pronoun attributes have been assigned in advance.

[0037] Attributes may also include position attributes that describe, in natural language, the positional relationships within elements constituting an object or the positional relationships between elements constituting an object. FIG. 7 is a diagram illustrating an example of reference information IF2 including position attributes. As shown in FIG. 7, reference information IF2 may include positional relationships such as center, front, back, left, right, top, bottom, outer edge, and inner edge. For example, partial elements contained in a certain element (e.g., surface A) may be expressed using reference information IF2 as "the center part of surface A," "the front part of surface A," "the outer edge of surface A," "the inner edge of surface A," etc. Each of these parts may be assigned a non-natural language identification code. Each of these position attributes may be associated with, for example, a predetermined determination formula (such as a coordinate comparison between objects). The position attribute may describe the positional relationships between elements constituting an object (e.g., that one object is located to the left of another object). Note that the positional relationships within elements constituting an object may also be expressed as the positional relationships between the elements.

[0038] The attributes may also include shape information that describes the shapes of the elements that make up the object using natural language. FIG. 8 is a diagram showing an example of reference information IF3 that includes shape attributes. As shown in FIG. 8, the reference information IF3 may describe natural language modifiers that describe shapes, such as "flat," "wide," "angular," "curved," and "bent," as shape attributes. These shape attributes may be specified in association with conditions for assigning the attributes, such as a determination condition based on the connection relationship between line segment elements (e.g., if the angle between line segment elements is equal to or greater than a predetermined value, it indicates "angular").

[0039] The attributes may also include appearance attributes related to the appearance of the elements that make up the object. Fig. 9 is a diagram showing an example of reference information IF4 including appearance attributes. The appearance attributes may include, for example, attributes representing colors such as "red," "yellow," "blue," and "green," attributes representing patterns such as "striped," "checked," and "triangular checkered," and attributes representing textures such as "smooth," "rough," and "unkempt." These attributes may be defined in association with discrimination conditions based on, for example, texture information assigned to the object (e.g., RGB values, type of material texture, etc.).

[0040] In this way, the reference information can be configured to hierarchically represent the relative relationships of the elements that make up an object (for example, the "walls" of a "house", the "left part" of the "floor" of a "house", etc.).

[0041] The specific form of the reference information may be a table in which "terms" and "determination conditions" are associated as described above, a hierarchical network model that describes knowledge, or a trained model (particularly a generative AI model) that has been trained in advance. The trained model may be trained by machine learning using a training dataset that includes model data that describes objects to which natural language attributes different from identification codes have been assigned in advance.

[0042] 3.2. Information processing flow for assigning attributes FIG. 10 is an activity diagram showing an example of the flow of information processing for assigning attributes, which is executed in the information processing system 1. Note that the information processing may include any exception processing not shown. Exception processing includes interruption of the information processing or omission of each process. Selection or input performed in the information processing may be based on a user operation or may be performed automatically without relying on a user operation.

[0043] [Action A1] 10, first, in action A1, processor 33 accepts input of model data from a user. Note that the input model data may be existing model data created by the user or model data of a template model to be generated.

[0044] [Action A2] Next, in action A2, the processor 33 defines a model space as a representation space based on the model data, thereby virtually obtaining a space in which objects can be arranged.

[0045] [Action A3] Next, in action A3, processor 33, as an acquisition unit, acquires object information about at least one object to be placed in a representation space for visually representing objects. Note that if the model data is blank model data with no objects placed on it, processor 33 may appropriately acquire information about an object to be newly placed in the model space as object information.

[0046] [Action A4] Next, in action A4, processor 33 assigns an identification code to each of the elements that make up the obtained object. The identification code may be defined, for example, by a natural number. Note that the identification code may be assigned as a different value each time a model space is defined.

[0047] [Action A5] Next, in action A5, the processor 33, as an attribute assigning unit, assigns the above-mentioned attribute, which is different from the identification code, to the elements constituting the object based on the object information and predefined reference information. As described above, the attributes are described in natural language. For example, the processor 33 may acquire a list of attributes of the elements constituting the object (e.g., attributes representing each element constituting the "house" shown in FIG. 6) output by inputting the object information and a context representing the entire object to a predefined trained model, identify a judgment condition for the corresponding reference information from the list, and assign the corresponding attribute to object information (e.g., shape, positional relationship, etc.) that satisfies the judgment condition. In other words, the processor 33 may input information about the object to a trained model trained based on predefined reference information, and assign the output of the trained model (e.g., a generative AI) regarding each element constituting the object as an attribute to the element constituting the object. With this configuration, when an object includes a large number of elements, the time required to assign attributes to each of the elements can be reduced. In this case, it is preferable that the processor 33 assigns attributes to all of the elements constituting the object. With this configuration, it is possible to reduce the possibility that an element that constitutes an object that cannot be designated will occur.

[0048] In this case, for example, the processor 33 may assign position attributes to the elements that make up the object based on the positional relationships between the elements that make up the object. The processor 33 may also assign attributes according to the boundaries of the elements that make up the object. This configuration allows shape-related information to be assigned as attributes using natural language, making it possible to more intuitively specify elements using language. For example, the processor 33 may assign shape attributes such as square, triangle, circle, etc., according to the form of loop connections of directed line segments, which are the smallest units of elements that make up the object.

[0049] Thereafter, processor 33 determines whether or not to terminate the assignment of attributes. If it is determined that the assignment of attributes should be continued, processor 33 continues the processing of action A5, and repeatedly assigns or modifies attributes. For example, processor 33 may assign a higher-level designation attribute (in other words, a designation based on a higher concept) to an element constituting an object, and then assign a lower-level designation attribute that belongs to the higher-level designation attribute. Thereafter, processor 33 may assign designation attributes up to a predetermined hierarchy, and then assign a position attribute or the like in association with the designation attribute of each hierarchy. Processor 33 may also assign a new attribute in response to an operation input from the user.

[0050] [Action A6] When the assignment of attributes is completed, the process proceeds to action A6. In action A6, processor 33 executes a management process, thereby serving as a management unit, managing elements constituting an object in a namespace corresponding to the assigned attributes in a manner that allows the elements to be uniquely identified using natural language. With this configuration, elements constituting an object can be identified using natural language, so that, for example, a user can intuitively specify elements of an object arranged in a representation space using natural language rather than using a non-natural language such as an identification number.

[0051] For example, processor 33 may manage the elements of an object in a hierarchical namespace related to the attributes of the elements that make up the object, based on the hierarchical structure of the elements that make up the object. With this configuration, even if there are multiple elements of the same type of object and the individual attributes of the elements of the object are the same, they can be managed in different namespaces based on the hierarchical structure. This makes it easier to manage the elements of the same type of object. For example, processor 33 may uniquely identify, in the hierarchical namespace, the qualifying relationships between each element that makes up an object representing a "house," such as "house / floor," "house / wall," "house / roof," "house / entrance," and "house / window."

[0052] As another example, the processor 33 may manage elements constituting objects assigned the same name attribute in a manner that allows them to be uniquely identified using natural language in a namespace based on a position attribute. With this configuration, for example, when there are objects with the same name, the elements can be managed uniquely based on their positional relationships, allowing a user to intuitively specify the elements. For example, the processor 33 may manage each "pillar" object in the namespace by assigning additional position attributes to objects assigned the same name attribute (e.g., multiple "pillars" in a "house"), such as "center pillar," "right pillar," "front pillar," or "pillar closest to the entrance," depending on their relative positions. In this case, the "center pillar" may be managed hierarchically, for example, as "house / room / pillar / center," and the "right pillar" may be managed hierarchically, for example, as "house / room / pillar / right." Note that front, back, left, and right can be identified based on the positional relationships of the elements constituting the object, without relying on a coordinate system. For example, in the case of a "house," the direction from a certain point in a "room" to an object corresponding to the "entrance" is defined as "front," the opposite direction of "front" is defined as "back," the direction of the "floor" relative to the front and back is defined as "down," the direction of the "roof" is defined as "up," and directions perpendicular to these are defined as "left" and "right." Therefore, processor 33 can manage attributes assigned to elements regardless of the coordinate system of the representation space. This configuration allows attributes to be maintained even if the coordinate system changes. Therefore, for example, when multiple users share objects, they can use common terms to specify objects even if their coordinate systems differ. The coordinate system can be any system, such as a Cartesian coordinate system, a cylindrical coordinate system, or a polar coordinate system. Repeatedly assigning or modifying attributes in this manner can further improve the accuracy of attribute assignment.In particular, processor 33 can cause the generation AI to mechanically output initial values ​​of the attributes of the elements that make up the object based on a command input from a user to a trained model such as a generation AI, including the attributes of the entire object (especially the pronunciation attributes), and modify the initial values ​​to match the user's intuition by comparing them with predetermined dictionary data, etc., and repeatedly generate or modify such intuitive attributes to hierarchically manage the attributes so that they can be uniquely identified in the namespace. This configuration allows for centralized character information management of objects with higher accuracy than when simply using a generation AI.

[0053] Thereafter, the processor 33 ends this information processing.

[0054] 3.3. Relationship between attributes and elements that make up an object Next, the relationship between attributes assigned to elements constituting an object through the above-described information processing will be described. FIG. 11 is a diagram showing an example of attributes assigned to the object shown in FIG. 5. As shown in FIG. 11, each of the loop structures corresponding to the six faces of the rectangular parallelepiped "box" object Ob1 is assigned an attribute capable of identifying the "face," and these "face" nominal attributes (which are also shape attributes) are subordinate to the nominal attribute "box." Furthermore, since the six "face" attributes overlap, these faces are assigned attributes, such as "front face," "right face," and "top face," in addition to the position attribute, to uniquely define them so that they do not overlap. In this case, attributes such as "front," "right," and "top" are subordinate to the "face" belonging to the "box," as an example. Furthermore, an attribute called "periphery (edge)" is assigned to the boundary represented by the loop structure, which is an example of an element constituting an object. This attribute indicates the boundary of the "face," and is therefore subordinate to the "face." Furthermore, the "perimeter" is defined by four directed line segments that form a loop structure, and each of these four directed line segments is assigned an attribute called "edge." An "edge" is an element that constitutes a "face" or "perimeter" and is dependent on these attributes. In this example, because four "edges" overlap with the "front surface," each directed line segment is managed so that it can be uniquely identified using position attributes such as "bottom edge," "top edge," "right edge," and "left edge." Furthermore, portions of each directed line segment may also be divided and assigned attributes, such as "center," "right portion," and "left portion." Furthermore, the X, Y, and Z directions in the model space may also be redefined based on the elements that constitute the object. In other words, the processor 33 may assign attributes to directions describing the representation space based on attributes (particularly position attributes) assigned to the elements that constitute the object.

[0055] 3.4. Processing information to perform operations on objects Next, an information processing for executing an operation on an object using the attribute assigned by the above-mentioned information processing will be described. Fig. 12 is a diagram showing an activity showing the flow of the information processing for executing an operation on an object.

[0056] [Action A11] As shown in FIG. 12 , first, in action A11, processor 33 executes an input acceptance process, thereby functioning as an input acceptance unit to accept input of a command statement as character information for specifying an element constituting an object from a user. The command statement is defined in a natural language (not limited to a complete sentence, but may include a list of words). That is, in this information processing, processor 33 accepts character input via a CUI. Note that this does not mean that acceptance of input via a GUI is rejected. The command statement may include first character information capable of identifying an attribute and second character information capable of identifying an operation on an element constituting the object. The first character information is configured to be capable of identifying an attribute assigned to an object and managed in a namespace, such as "right pillar." The first character information can select an element constituting a plurality of objects that can be the target of an operation. The second character information specifies the type of operation, such as join, delete, smoothing, or color change.

[0057] The input method for the command statement is not limited to input via a keyboard, which is an example of the input unit 35, but may also be voice input via a microphone or the like. In other words, the processor 33 may act as a voice input unit and accept voice input from the user. In this case, the processor 33 may generate text information based on the voice input and accept the generated text information as the command statement. With this configuration, object operations can be performed naturally by voice, thereby providing a system that enables more convenient object operation.

[0058] [Action A12] Next, in action A12, processor 33 executes an operation identification process to identify an element constituting an object to be operated (e.g., an object identified by the first character information) and an operation to be performed on the operation object based on the received command statement. For example, processor 33 executes natural language processing on the received command statement to identify an element constituting an object having an attribute represented by the first character information as an operation object (or a candidate for an operation object) and to identify an operation that can perform an operation corresponding to the second character information. Note that as long as an operation can be uniquely identified, any format of command statement can be used, not limited to a command statement including the first character information and the second character information. Here, the operation on the operation object may include an identification operation. The identification operation is an operation that requires a process of newly assigning an identification code assigned to an element constituting the object on which the identification operation is performed, and includes, for example, an operation based on a Boolean operation on the element constituting the object. With this configuration, even if the identification code of an existing object is changed by an operation of adding an element constituting a new object by a Boolean operation (e.g., an operation of forming a hole or a groove in a plane, or combining objects), the existing object can be specified based on the attribute assigned before the identification operation. Boolean operations can include, for example, a union operation that combines two shapes, a subtraction operation that removes one shape from another, and a multiplication operation that removes the intersection of two shapes. These operations significantly change the topological information of an object because the shapes before and after the operation cannot be reversibly obtained through continuous transformation. As a result, a new topology is created as the topological information changes, making it impossible to reuse existing identification codes.

[0059] [Action A13] Next, in action A13, processor 33 executes an operation based on the result of the operation identification process executed in action A12. As a result, processor 33 executes, for example, an operation specified by the second character information on an element that specifies an object belonging to a namespace specified by an attribute specified by the first character information. With this configuration, operations on elements that constitute an object that are intuitively associated by an attribute can be intuitively executed by inputting characters using natural language. Therefore, for example, a CUI with higher operability can be provided. If the identified operation is a specific operation, processor 33 can execute the specific operation on the elements that constitute the object.

[0060] [Action A14] Thereafter, if the operation is a Boolean operation, in action A14, the processor 33 reassigns an identification code to the processor 33 having the new topology information. Since the identification code is reassigned regardless of before and after the topology change, in order to identify the operation target in the CUI based on the identification code, it is necessary to grasp each newly assigned identification code.

[0061] [Action A15] Thereafter, in action A15, processor 33 assigns attributes to the elements constituting the object based on the object information and reference information so as to maintain the attributes. At this time, processor 33 may constrain the attributes to be reassigned based on existing attributes so as to maintain the attributes already assigned to the elements constituting the object. In other words, processor 33 can manage the elements constituting the object in a namespace using attributes common before and after the specific operation in a manner that allows them to be uniquely identified using natural language. With this configuration, for example, even if it is necessary to assign a new identification code by changing the topology of the object, the elements constituting the object to which attributes have already been assigned can be specified using natural language that is common before and after the specific operation. Note that the specific form of this processing is arbitrary as long as the attributes already assigned are maintained, and it is not necessary to use object information or reference information.

[0062] [Action A16] Next, in action A16, the processor 33 manages the attributes in a namespace, thereby enabling the attributes assigned to the new topology information to be consistently managed in a namespace corresponding to the namespace obtained before the operation (including the same namespace).

[0063] [Action A17] Thereafter, in action A17, processor 33 causes the operation result to be displayed on display unit 34. If the operation executed in action A13 is an operation other than the specific operation, processor 33 omits the processing of actions A14 to A16 and executes the processing of action A17.

[0064] Thereafter, the processor 33 ends this information processing.

[0065] 3.5. Changes in topology information for specific operations and management in the attribute-based namespace Next, we will explain the changes in topology information when a specific operation is performed in the information processing described in the previous section, and the management in the attribute-based namespace. Figure 13 is a diagram showing the relationship between the changes in topology information and attributes when a specific operation is performed on the object shown in Figure 11. The specific operation performed here is an operation that includes a subtraction operation, which creates a circular hole in the top surface of the "box" object Ob1. For convenience of explanation, in Figure 13, the object before the specific operation is performed is represented as Ob1, and the object after the specific operation is performed is represented as Ob2.

[0066] 13, the top surface of object Ob1 before the operation is defined using only one loop defined by four directed line segments, whereas the top surface of object Ob2 after the operation has a discontinuous boundary defined by a circle inside the loop. As a result, various elements that make up the object related to the hole are newly generated, such as the circle that separates the hole from the top surface, the side surfaces of the hole continuing from the hole, and the space defined by the side surfaces.

[0067] The processor 33 assigns new attributes to object information including elements constituting a new object generated by such a specific operation, based on attributes managed in an existing namespace. In this embodiment, the processor 33 assigns an attribute of "round hole" subordinate to the "top surface" to the newly generated circle, and assigns an attribute of "side surface" (side surface of the round hole) subordinate to the "round hole" to the surface generated as a result of the subtraction operation extending from the circle. The processor 33 may also manage new attributes in a multi-level hierarchical structure in the namespace, such as "round hole" in the "center" of the "top surface."

[0068] 3.6. Screen Examples Next, an example of a screen displayed on the display unit 34 in the information processing for executing an operation on the object will be described. Fig. 14 shows an example of an operation screen displayed on the display unit 34 in the information processing for executing an operation on the object. As shown in Fig. 14, the operation screen 5 displayed on the display unit 34 may include an object display area 51, a history area 52, and an input area 53.

[0069] The object display area 51 is an area for visually displaying a representation space (for example, a model space) and objects arranged in the representation space. The user can input GUI operations for objects by performing mouse operations or touch operations on the object display area 51.

[0070] The history area 52 is an area where the history of operations performed on an object (not limited to operations instructed by the user, but may include operations automatically performed by the system) is displayed.

[0071] The input area 53 is an area where the user inputs a command. The input may be input as text information, such as "Please drill a round hole in the top surface of the box." The processor 33 executes an operation based on the input content in the input area 53. In this case, "top surface of the box" corresponds to the first text information, and "Please drill a round hole." corresponds to the second text information. In this case, a round hole of appropriate dimensions may be formed in any area of ​​the entire top surface of the box. For example, the object Ob2 shown in FIG. 13 is displayed in the object display area 51 as an object representing the latest model.

[0072] The input area 53 may include a voice input button 531. When the voice input button 531 is operated, the processor 33 requests voice input using the sound collection device 4, and displays in the input area 53 a command statement identified based on the input voice information.

[0073] Through the above information processing, processor 33 assigns natural language attributes that users can recognize as concepts to the elements that make up the objects, and manages these attributes in a namespace according to a hierarchical structure as concepts, thereby providing users with intuitive GUI operations for the elements that make up a huge number of objects.

[0074] In other words, the processor 33 assigns attributes to the elements that make up an object so as to form a projection from the semantic space represented by the namespace to the instance space. This allows the processor 33 to make the namespace corresponding to the assigned attributes function as the semantic space, and to manage the elements that make up the object while preserving relationships, hierarchy, order, symmetry, etc.

[0075] 4.Other The above embodiment may be modified as follows.

[0076] Processor 33 may accept input related to attribute changes. For example, processor 33 may accept a command statement to change attributes related to direction or positional relationship, such as "Please set the currently largest displayed surface as the front of the box." In this case, processor 33 may update position attributes representing up, down, front, back, left, and right represented by the attributes, based on the attribute of the largest displayed surface among the display contents displayed in object display area 51 and the relationship between that surface and elements constituting other objects.

[0077] The reference information is arbitrary as long as it can assign attributes to the elements that make up the object, and may be specified to include the generation AI.

[0078] The above-mentioned information processing is not limited to computer-aided design, but can be applied to any design or simulation such as disaster evacuation routes, air conditioning simulation, heat conduction, electromagnetic interference, and structural design under field conditions.

[0079] In the above embodiment, a so-called on-premise configuration is adopted in which the user terminal 3 executes information processing, but the above information processing may also be a cloud configuration in which the information processing device 2 executes the information processing based on information input to the user terminal 3. As a cloud-based information processing device 2, the above functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0080] In the above embodiment, the information processing device 2 performs various storage and control operations, but multiple external devices may be used instead of the information processing device 2. That is, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0081] The above embodiment is not limited to the information processing system 1, and may be an information processing method or an information processing program. The information processing method includes each step of the information processing system 1. The program causes at least one computer to execute each step of the information processing system 1.

[0082] The information processing system 1 and the like may be provided in the following aspects.

[0083] (1) An information processing system, comprising at least one processor capable of executing a program to perform the following steps: in the acquisition step, acquiring object information about at least one object arranged in a representation space for visually representing the object, the object information including information about elements that constitute the object; and in the assignment step, The object information and predetermined reference information, attributes different from the identification code are assigned to the elements that make up the object, the attributes being described in a natural language, and the reference information is Elements that make up the object The correspondence between the state of and in the management step, the elements constituting the object in the namespace corresponding to the attribute are managed in a manner that allows them to be uniquely identified using natural language. system.

[0084] With this configuration, the elements that make up an object can be identified using natural language, so that, for example, a user can intuitively specify the elements of an object placed in the representation space through natural language rather than using non-natural language such as an identification number.

[0085] (2) In the information processing system described in (1) above, the reference information is configured to hierarchically represent the relative relationships of the elements that constitute the object, and in the management step, the elements of the object are managed in a hierarchical namespace related to the attributes of the elements that constitute the object based on the hierarchical structure of the elements that constitute the object.

[0086] With this configuration, even if there are multiple elements of the same type of object and the individual attributes of the object elements are the same, they can be managed in different name spaces based on the hierarchical structure, making it easier to manage elements of the same type of object.

[0087] (3) In the information processing system described in (1) or (2) above, the attributes include a name attribute relating to the name of the object represented by the object or the elements constituting the object, and the reference information is associated with the name attribute corresponding to each of the elements constituting the object.

[0088] According to this configuration, natural pronunciation can be used when specifying a target, thereby improving convenience when the user specifies an object.

[0089] (4) In the information processing system described in (3) above, the attributes include a position attribute that describes, using natural language, the positional relationship within the elements that make up the object or the positional relationship between the elements that make up the object, and in the management step, the system further assigns the position attribute to the elements that make up the object based on the positional relationship between the elements that make up the object, thereby managing the elements that make up the object that have been assigned the same designation attribute in a manner that allows them to be uniquely identified using natural language in a namespace based on the position attribute.

[0090] According to this configuration, for example, when there are objects with the same name, the elements can be managed uniquely based on their positional relationships, and the user can intuitively specify the elements.

[0091] (5) In the information processing system described in any one of (1) to (4) above, the object information includes information regarding the boundaries of the elements that constitute the object, and in the assigning step, the attribute is assigned according to the boundaries of the elements that constitute the object.

[0092] According to this configuration, information about the shape can be assigned as an attribute in natural language, so that elements can be specified more intuitively in language.

[0093] (6) In the information processing system described in any one of (1) to (5) above, the management step manages the attributes assigned to the elements regardless of the coordinate system of the representation space.

[0094] With this configuration, attributes are maintained even when the coordinate system changes, so that, for example, when multiple users share an object, the terms used to specify the object can be standardized even if the users use different coordinate systems.

[0095] (7) In an information processing system described in any one of (1) to (6) above, in the specific operation step, a specific operation is performed on an element constituting the object, and the specific operation is an operation that requires processing to newly assign the identification code that has been assigned to the element constituting the object on which the specific operation has been performed, and in the management step, the elements constituting the object are managed in a manner that allows them to be uniquely identified using natural language in a namespace using the attribute that is common before and after the specific operation.

[0096] With this configuration, even if it is necessary to assign a new identification code by changing the topology of an object, the elements that make up the object to which attributes have already been assigned can be specified using a common natural language before and after the specific operation.

[0097] (8) In the information processing system described in (7) above, the specific operation includes an operation based on a Boolean operation on the elements that make up the object.

[0098] With this configuration, even if the identification code of an existing object is changed by an operation that adds elements that make up a new object using Boolean operations (for example, an operation that creates a hole or groove in a plane, or by combining objects), the existing object can be specified based on the attributes that were assigned before the specific operation.

[0099] (9) In the information processing system described in any one of (1) to (8) above, in the input receiving step, an input of character information for specifying an element constituting the object is received from a user, the character information being specified in a natural language and including first character information capable of identifying the attribute and second character information capable of identifying an operation content for the element constituting the object, and in the operation step, an operation specified by the second character information is performed on an element defining the object belonging to the namespace specified by the attribute specified by the first character information.

[0100] With this configuration, operations on elements that make up objects that are intuitively associated by attributes can be intuitively performed by inputting characters using natural language, thereby providing a CUI with higher operability, for example.

[0101] (10) In the information processing system described in (9) above, the voice input step further receives voice input from the user, and the input receiving step generates the character information based on the voice input.

[0102] According to this configuration, an object can be naturally operated by voice, and therefore a system that allows for more convenient object operation can be provided.

[0103] (11) In the information processing system described in any one of (1) to (10) above, in the assignment step, information about the object is input into a trained model trained based on predefined reference information, and the output by the AI ​​regarding each of the elements that make up the object is assigned to the elements that make up the object as the attribute.

[0104] According to this configuration, when one object includes a large number of elements, it is possible to reduce the time required to assign attributes to each of the elements.

[0105] (12) In the information processing system described in any one of (1) to (11) above, the elements constituting the object include a portion of a three-dimensional area constituting the object, at least a portion of a two-dimensional area constituting the three-dimensional area, or at least a portion of a one-dimensional area constituting the two-dimensional area.

[0106] (13) An information processing method, comprising the steps of the information processing system according to any one of (1) to (12) above.

[0107] (14) A program that causes at least one computer to execute each step of the information processing system according to any one of (1) to (12) above. Of course, this is not the case.

[0108] Finally, while various embodiments of the present disclosure have been described, they are presented as examples and are not intended to limit the scope of the invention. The novel embodiments may be embodied in various other forms, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments and modifications are intended to be included within the scope and spirit of the invention, as well as within the scope of the inventions and their equivalents as defined in the claims. [Explanation of symbols]

[0109] 1: Information processing system 2: Information processing equipment 20: Communication bus 21: Communications Department 22: Storage section 23: Processor 3: User terminal 30: Communication bus 31: Communications Department 32: Storage section 33: Processor 34:Display section 35: Input section 4: Sound collection device 5: Operation screen 51: Object display area 52: History area 53: Input area 531: Voice input button DS: Data Structure IF1~IF4: Reference information Ob1, Ob2: Objects

Claims

1. An information processing system, at least one processor capable of executing a program to perform the following steps; In the obtaining step, object information regarding at least one object arranged in a representation space for visually representing the object is obtained; the object information includes information about elements that constitute the object, In the assigning step, an attribute different from the identification code is assigned to an element constituting the object based on the object information and predefined reference information; the identification code is a non-natural language code that is mechanically assigned to elements that constitute the object when the object is placed in the representation space, reassigned when the topology of the elements that make up the object changes; The attributes are described in natural language; the reference information indicates a correspondence relationship between each attribute and a state of an element constituting the object; In the management step, the system manages elements that make up the object in a name space corresponding to the attribute in a manner that allows them to be uniquely identified using natural language.

2. 2. The information processing system according to claim 1, the reference information is configured to hierarchically represent the relative relationships of elements that constitute the object, In the management step, the elements constituting the object are managed in a hierarchical name space related to the attributes of the elements constituting the object, based on a hierarchical structure of the elements constituting the object.

3. An information processing system, At least one processor capable of executing a program to perform the following steps: In the obtaining step, object information regarding at least one object arranged in a representation space for visually representing the object is obtained; the object information includes information about elements that constitute the object, In the assigning step, an attribute different from the identification code is assigned to an element constituting the object based on the object information and predefined reference information; the identification code is a non-natural language code that is mechanically assigned to elements that constitute the object when the object is placed in the representation space, The attributes are described in natural language; the reference information indicates a correspondence relationship between each attribute and a state of an element constituting the object; The attribute is: A name attribute relating to the name of an object represented by the object or an element constituting the object; a position attribute that describes, using a natural language, a positional relationship within an element that constitutes the object or a positional relationship between elements that constitute the object; the reference information is associated with the naming attribute corresponding to each of the elements constituting the object; In the management step, the system assigns the location attribute to the elements that make up the object based on the positional relationship between the elements that make up the object, and manages the elements that make up the object that have been assigned the same name attribute in a manner that allows them to be uniquely identified using natural language in a namespace based on the location attribute.

4. 2. The information processing system according to claim 1, the object information includes information about boundaries of elements that constitute the object; In the assigning step, the attribute is assigned according to boundaries of elements that constitute the object.

5. 2. The information processing system according to claim 1, In the management step, the system manages the attributes assigned to the elements regardless of the coordinate system of the representation space.

6. An information processing system, At least one processor capable of executing a program to perform the following steps: In the obtaining step, object information regarding at least one object arranged in a representation space for visually representing the object is obtained; the object information includes information about elements that constitute the object, In the assigning step, an attribute different from the identification code is assigned to an element constituting the object based on the object information and predefined reference information; the identification code is a non-natural language code that is mechanically assigned to elements that constitute the object when the object is placed in the representation space, The attributes are described in natural language; the reference information indicates a correspondence relationship between each attribute and a state of an element constituting the object; In the specific operation step, a specific operation is performed on an element constituting the object; the specifying operation is an operation that requires a process of newly assigning the identification code that has been assigned to an element that constitutes the object on which the specifying operation has been performed, In the management step, elements constituting the object are managed in a name space using the attribute that is common before and after the specific operation in a manner that allows the elements to be uniquely identified using a natural language; The system, wherein the specific operation includes an operation based on a Boolean operation on elements that constitute the object.

7. 2. The information processing system according to claim 1, the input receiving step receives input of character information for specifying an element constituting the object from a user, the character information being defined in a natural language and including first character information capable of identifying the attribute and second character information capable of identifying an operation content for the element constituting the object; In the operation step, the system executes an operation specified by the second character information on elements that constitute the object that belong to the namespace defined by the attribute specified by the first character information.

8. In the information processing system according to claim 7, Furthermore, in the voice input step, voice input from the user is accepted, In the input receiving step, the system generates the text information based on the voice input.

9. 2. The information processing system according to claim 1, In the assignment step, the system inputs the object information into a trained model trained based on predefined reference information, and assigns the output from the trained model regarding each of the elements that make up the object as the attribute to the elements that make up the object.

10. 2. The information processing system according to claim 1, A system in which the elements constituting the object include a portion of a three-dimensional region constituting the object, at least a portion of a two-dimensional region constituting the three-dimensional region, or at least a portion of a one-dimensional region constituting the two-dimensional region.

11. An information processing method, comprising: A method comprising the steps of the information processing system according to any one of claims 1 to 10.

12. A program, A program causing at least one computer to execute each step of the information processing system according to any one of claims 1 to 10.

Images