Information generation device, information generation method, computer program, and learning model

Abstract

The information generation device of the present disclosure includes a generation unit that inputs document data indicating the operation content to be executed by the robot to be controlled into a learning model and causes the learning model to generate control-related information indicating information related to the control of the robot to be controlled, and a storage unit that stores the control-related information. The learning model is constructed by learning a learning data set. The learning data set includes learning document data indicating the operation content to be executed by the learning robot and learning control-related information that is information related to the control of the learning robot. The learning control-related information includes control information of the learning robot created based on the learning document data.

Description

Technical Field

[0001] The present disclosure relates to an information generation device, an information generation method, a computer program, and a learning model.

Background Art

[0002] In recent years, it has been put into practical use to utilize an autonomous mobile robot capable of autonomous driving for transporting goods in facilities such as factories and warehouses. A control technology of a guidance system for running an autonomous mobile robot along a predetermined rail track laid in a facility to a predetermined target position is disclosed in, for example, Patent Document 1. Further, a technology of an autonomous driving system for performing driving control to a destination by self-position estimation and environmental map creation is described in, for example, Patent Document 2.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in order to control the autonomous mobile robot described in Patent Document 1, it is necessary to create control information of the autonomous mobile robot by a person based on the required specifications. Therefore, for the creator of the control information, for example, a great deal of labor, rich experience, and accumulated knowledge are required. This is not limited to the autonomous mobile robot, and the same applies when creating control information for other robots. Further, this is not limited to the control information of the robot, and the same applies when creating the required specifications of the robot.

[0005] Therefore, the present disclosure has been made in view of the above problems, and an object thereof is to provide an information generation device, an information generation method, a computer program, and a learning model that can easily generate control-related information that is information related to the control of a robot to be controlled.

Means for Solving the Problems

[0006] According to the present disclosure, there is provided an information generation device including: a generation unit that inputs document data indicating an operation content to be executed by a robot to be controlled into a learning model and causes the learning model to generate control-related information indicating information related to the control of the robot to be controlled; and a storage unit that stores the control-related information, wherein the learning model is constructed by learning a learning data set, the learning data set includes learning document data indicating an operation content to be executed by a learning robot, and learning control-related information that is information related to the control of the learning robot, and the learning control-related information includes control information of the learning robot created based on the learning document data.

[0007] According to the present disclosure, there is provided an information generation method including: inputting document data indicating an operation content to be executed by a robot to be controlled into a learning model and causing the learning model to generate control-related information indicating information related to the control of the robot to be controlled; and acquiring the control-related information, wherein the learning model is constructed by learning a learning data set, the learning data set includes learning document data indicating an operation content to be executed by a learning robot, and learning control-related information that is information related to the control of the learning robot, and the learning control-related information includes control information of the learning robot created based on the learning document data.

[0008] According to the present disclosure, a computer is caused to execute a step of inputting document data indicating operation contents to be executed by a robot to be controlled into a learning model and causing the learning model to generate control-related information indicating information related to the control of the robot to be controlled, and a step of acquiring the control-related information. The learning model is constructed by learning a learning dataset. The learning dataset includes learning document data indicating operation contents to be executed by a learning robot and learning control-related information that is information related to the control of the learning robot. The learning control-related information includes control information of the learning robot created based on the learning document data. A computer program is provided.

[0009] According to the present disclosure, there is provided a learning model that is constructed by learning a learning dataset and causes a computer to function so as to generate control-related information indicating information related to the control of a robot to be controlled. The computer is caused to function to input document data indicating operation contents to be executed by the robot to be controlled and generate the control-related information. The learning dataset includes learning document data indicating operation contents to be executed by a learning robot and learning control-related information that is information related to the control of the learning robot. The learning control-related information includes control information of the learning robot created based on the learning document data.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to provide an information generation device, an information generation method, a computer program, and a learning model that can easily generate control-related information that is information related to the control of a robot to be controlled.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0012] With reference to the accompanying drawings below, preferred embodiments of the present disclosure will be described in detail. In this specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant descriptions are omitted. Also, in the drawings, the database may be denoted as "DB".

[0013] (Embodiment 1) With reference to FIGS. 1 to 7, a robot system SYS according to Embodiment 1 of the present disclosure will be described. FIG. 1 is a block diagram showing a configuration example of the robot system SYS. As shown in FIG. 1, the robot system SYS includes a learning device 100, a generative AI (Artificial Intelligence) device 200, an information generation device 300, a robot management device 400, at least one robot RB, at least one document database device 500, and at least one control database device 600. The robot system SYS may include at least one environment database device 700.

[0014] The document database device 500 includes at least one document database 501. The document database 501 stores document data 502. The document data 502 indicates the operation content to be executed by the robot RB. The document database 501 may store a plurality of different document data 502. Also, the document database device 500 may include a plurality of different document databases 501. For example, the document database 501 stores the document data 502 for the robot RB1, and another document database 501 stores the document data 502 for the robot RB2.

[0015] The document data 502 includes, for example, one or more types of data among text, images, and diagrams (e.g., CAD drawings). That is, the document data 502 may include data of one data format type or data of a plurality of data format types.

[0016] The document data 502 is data indicating, for example, the specifications of the robot RB. The specifications are, for example, the requirement specifications and / or the requirement definition documents of the robot RB. In other words, the document data 502 is data indicating the requirement specifications for the robot RB.

[0017] Specifically, the document data 502 includes the work requirements for the robot RB. The work requirements are requirements regarding the work content to be executed by the robot RB. The work requirements include, for example, one or more pieces of information among work process definition, work object, work content, work environment conditions, and work timing.

[0018] The work process definition includes, for example, one or more pieces of information among the definition of the target work process (for example, the parts supply process), the target equipment and line information (for example, production lines 1, 2, 3), the business flow (including, for example, the business sequence), and the response flow in case of abnormality.

[0019] The work object includes, for example, one or more of the type, shape, and size of the object (workpiece) (for example, the type of parts), the physical characteristics of the object (for example, weight, material, or flexibility), the posture and state of the object (for example, orientation or arrangement state), and precautions for handling.

[0020] The work content includes, for example, one or more pieces of information among specific work procedures (for example, picking, conveying, or supplying), work location information (for example, the position of the parts supply port), task execution ability requirements (for example, throughput or cycle time), the number of tasks that can be executed simultaneously, and the level requirement of work autonomy (for example, full manual control, semi-autonomous control, or full autonomous control). The level requirement of work autonomy indicates the level of the degree of automation of the work by the robot RB.

[0021] The work environment conditions include, for example, one or more pieces of information among process layout information (for example, the number and position of the parts supply ports on each line), work space constraints, the positional relationship with other equipment, and obstacles on the work path.

[0022] The operation timing and synchronization include, for example, one or more pieces of information among the operation order and priority rules, synchronization conditions with other processes, takt time requirements, operation cycle and timing, and task switching time requirements.

[0023] The document data 502 may include the functional requirements of the robot RB. The functional requirements are the requirements regarding the capabilities of the robot RB itself. The functional requirements include one or more pieces of information among the basic functions, sensing and recognition capabilities, and interaction.

[0024] The basic functions include, for example, one or more pieces of information among the operating range and degrees of freedom, portable weight and maximum load, moving speed and acceleration, maximum moving speed, stop accuracy, positioning accuracy and repeatability accuracy, type and function of the end effector, and power performance. The power performance includes, for example, one or more of the operating time of the battery, charging time of the battery, and battery replacement information.

[0025] The sensing and recognition capabilities include, for example, one or more pieces of information among the types of mounted sensors (e.g., cameras, LiDAR, or force sensors), object recognition capabilities, environment recognition capabilities, self-position estimation accuracy, obstacle detection capabilities and detection distances, and sensor fusion functions.

[0026] The movement and navigation include one or more pieces of information among the movement method (e.g., wheels, crawlers, or legs), navigation method, path planning algorithm, mapping capabilities, narrow passage capabilities, and step and slope adaptation capabilities.

[0027] The interaction includes one or more of the types of human interfaces, voice recognition and speech capabilities, gesture recognition and display capabilities, remote operation interfaces, teaching methods, and feedback presentation functions.

[0028] The document data 502 may include non-functional requirements of the robot RB. Non-functional requirements are requirements that represent characteristics or properties other than the functional requirements of the robot RB. That is, non-functional requirements are requirements regarding quality attributes. Non-functional requirements include, for example, information on one or more of safety, reliability and robustness, maintainability, communication and cooperation, and performance.

[0029] Safety includes, for example, information on one or more of safety functions and stop functions, risk assessment results, safety standard compliance (e.g., ISO 10218, ISO / TS 15066), fail-safe mechanisms, collision detection and avoidance capabilities, and electrical safety. Risk assessment is performed, for example, based on safety standards (e.g., ISO 10218, or ISO 12100).

[0030] Reliability and robustness include, for example, information on one or more of mean time between failures, availability (uptime ratio), failure detection functions, error recovery capabilities, environmental resistance (e.g., temperature, humidity, or dust), and durability (e.g., operating life or component life).

[0031] Maintainability includes, for example, information on one or more of maintenance cycles, ease of component replacement, self-diagnosis functions, remote maintenance functions, software update methods, and backup and restore functions.

[0032] Communication and cooperation include, for example, information on one or more of communication interfaces (wired or wireless), communication protocols, bandwidth requirements, cooperation functions with other systems, cloud cooperation functions, and data transmission and reception formats. A communication protocol is a communication standard or communication method that the robot RB supports for communicating with other devices or systems. Communication protocols are, for example, industrial communication protocols. Industrial communication protocols are, for example, EtherCAT, FL-net, EtherNet / IP, PROFINET, POWERLINK, SERCOS III, CC-LINK, or Modbus TCP.

[0033] Performance includes information on one or more of, for example, processing power and computing performance, response time, real-time performance, resource efficiency (e.g., power or computing resources), scalability, or optimization ability.

[0034] Document data 502 may include system implementation requirements for robot RB. The system implementation requirements are requirements regarding the configuration and implementation of robot RB. The system implementation requirements include information on one or more of, for example, hardware specifications, software architecture, and physical characteristics.

[0035] Hardware specifications include information on one or more of, for example, processor type and performance, memory capacity and type, storage capacity and type, power supply specifications (e.g., voltage or power consumption), battery capacity and type, and cooling system.

[0036] Software architecture includes information on one or more of, for example, operating system, middleware requirements, control algorithms, data processing pipeline, AI models and machine learning techniques, and software module configuration.

[0037] Physical characteristics include information on one or more of, for example, dimensions and size, weight, shape and appearance, installation area and ground pressure, material and surface finish, and appearance information.

[0038] Document data 502 may include operation requirements for robot RB. The operation requirements are requirements regarding the introduction, operation, and management of robot RB. The operation requirements include information on one or more of, for example, installation and introduction, operation management, scalability and customizability, compliance, and environmental adaptability.

[0039] Installation and introduction include information on one or more of installation conditions and methods, initial setup procedures, calibration methods, trial operation procedures, training requirements, and construction conditions.

[0040] Operation management includes, for example, one or more pieces of information among an operation monitoring function, a fleet management function, user management and authority setting, a scheduling function, an operation analysis and reporting function, and an operation support function. The fleet management function refers to a function of integrally managing and controlling a plurality of robots RB. Specifically, the fleet management function represents a mechanism for centrally performing operation status monitoring, work assignment, route plan optimization, battery management, and / or maintenance planning of a plurality of robots RB.

[0041] Scalability and customizability include, for example, one or more pieces of information among an expansion method, expansion module compatibility, API (Application Programming Interface) support, customizable parameters, an external cooperation function, and an upgrade path. The expansion method is, for example, an expansion method when introducing a robot RB introduced into a certain facility into another facility. The expansion module compatibility represents, for example, configurations and information related to the connection and control of additional function modules. The API support indicates information on the API provided when the robot RB cooperates with an external system (for example, a robot management device 400, an industrial device EQ, or a management system MG). The external cooperation function is a function for the robot RB to cooperate or integrate with an external device manufactured or developed by a third party other than the manufacturer or developer. The external device is, for example, an industrial device EQ.

[0042] Compliance includes, for example, information on one or more of legal regulations, certification requirements (e.g., CE or UL), personal information protection measures, security measures, environmental regulation compliance, and industry standard compliance. Certification requirements indicate the requirements for certifications that should be obtained to prove that the robot RB complies with the regulations or standards of each country or region. Security measures indicate the measures for designing and operating the robot RB to protect it from cyberattacks or unauthorized access. For example, security measures include one or more of communication encryption, control information encryption, obfuscation of control information, access control, vulnerability countermeasures, log management, and malware countermeasures. Environmental regulation compliance indicates the measures related to environmental regulations that the robot RB should follow. Industry standard compliance indicates the measures related to industry standards that the robot RB should follow. Industry standards are, for example, standards such as safety standards or communication standards. Information related to environmental regulations and information related to industry standards are information related to the environment of the robot RB.

[0043] Environmental adaptability includes, for example, information on one or more of the applicable environmental conditions (indoor or outdoor), lighting condition response range, dust and water resistance performance, temperature and humidity operating range, electromagnetic compatibility (EMC), and noise level.

[0044] The control database device 600 includes at least one control database 601. The control database 601 stores control-related information 602. The control-related information 602 indicates information related to the control of the robot RB. The control-related information 602 includes control information 603 of the robot RB created based on the corresponding document data 502. In the control database device 600 and the document database device 500, the control-related information 602 and the corresponding document data 502 are associated with each other. The control-related information 602 is typically shown in text format. Note that the control-related information 602 may be, for example, in binary format or a proprietary format. The proprietary format is a format defined by an industry, company, organization, or system.

[0045] The control database 601 may store a plurality of different control-related information 602. Further, the control database device 600 may include a plurality of different control databases 601. For example, the control database 601 stores control-related information 602 for the robot RB1, and another control database 601 stores control-related information 602 for the robot RB2.

[0046] The control information 603 of the robot RB typically includes a control setting file created based on the document data 502. The control setting file includes information defining specific parameters and operation sequences necessary for the robot RB to execute a specific task. The control setting file is, for example, structured data described in a text-based structured description language. The structured description language is, for example, YAML, TOML, or JSON. The structured description language is a language that can be read by humans. The processor of the robot RB parses (analyzes) the control setting file by a control program and processes the analysis result during operation execution to execute an operation according to the control setting file. Note that the control setting file may be processed by the processor of the robot RB after being compiled into machine language, for example.

[0047] Since the control information 603 is created based on the document data 502, the control information 603 includes information with content corresponding to each piece of information constituting the document data 502.

[0048] The control information 603 includes one or more of environment definition information, operation parameter information, task definition information, safety setting information, cooperation setting information, and operation management information. Specifically, it is preferable that the control information 603 includes environment definition information, operation parameter information, and task definition information. It is further preferable that the control information 603 further includes one or more of safety setting information, cooperation setting information, and operation management information.

[0049] The environment definition information is information regarding the physical or virtual space in which the robot RB operates. The environment definition information includes, for example, one or more pieces of information among work space information, reference point information, operable range information, operation prohibited area information, and environment state information. The work space information includes one or more pieces of information among map information of the work space and coordinate system information. The reference point information is, for example, a calibration reference point, the origin of the coordinate system, a reference point for position recognition by the robot RB, or a reference point for work coordinates. The environment state information indicates the configuration or state regarding the space in which the robot RB performs work or movement.

[0050] The operation parameter information is a group of technical parameters that control the physical behavior of the robot RB. The operation parameter information includes, for example, one or more pieces of information among operation definition information (for example, speed, acceleration, or torque), force limit profile, positioning accuracy, sensor feedback setting, control gain, and damping coefficient.

[0051] The task definition information indicates the definition regarding the work content to be executed by the robot RB. The task definition information includes, for example, one or more pieces of information among operation programs, operation sequence definitions, condition judgments, exception handling logics, task priorities, interlock conditions, work completion judgment criteria, and quality thresholds.

[0052] The safety setting information is information for ensuring the safety during the operation of the robot RB. The safety setting information includes, for example, one or more pieces of information among parameters of the collision detection sensor (for example, sensitivity, detection range, or reaction threshold), operation parameters during avoidance behavior, safety monitoring area, limit value definition, emergency stop conditions, fail-safe mode setting, speed during human collaboration, and force limit value.

[0053] The cooperation setting information is information required when the robot RB cooperates with an external device (for example, a robot management device 400, a management system MG, or an industrial device EQ). The cooperation setting information includes, for example, one or more pieces of information among a communication method with the external device, authentication information at the time of connection with the external device, security information, data format, message structure definition, communication timing, and synchronization parameters. The communication method includes, for example, one or more pieces of information among a communication protocol (for example, an industrial communication protocol), API setting information, and a URL. The API setting information is information for using an API for cooperation with an external device and is, for example, the URL of an API endpoint and an API key.

[0054] The operation management information is information for realizing efficient operation in a specific operation of a single robot RB or a plurality of robots RB. The operation management information includes, for example, one or more pieces of information among the operation priority order, scheduling, resource management, error processing, and performance evaluation of a single robot RB or a plurality of robots RB.

[0055] The environmental database device 700 will be described in Embodiments 4 and 5 described later.

[0056] The learning device 100 constructs a learning model TM by causing a learning dataset (hereinafter, "learning dataset TD") to be learned by an unlearned or pre-learned learning model TMB. The learning model TM is a learned model. The learning models TM and TMB are computer programs. Further, the learning device 100 may update the parameters of the learning model TM by causing the learning dataset TD to be re-learned by the learning model TM.

[0057] In this specification, "learning" indicates, for example, "machine learning".

[0058] The learning dataset TD includes learning document data 502 indicating the operation content to be executed by the learning robot, and learning control-related information 602 which is information related to the control of the learning robot. The learning robot corresponds to an example of the "learning robot" in the present disclosure. The learning control-related information 602 corresponds to an example of the "learning control-related information" in the present disclosure.

[0059] In this specification, the learning robot may be, for example, a robot that operates only to acquire the learning dataset TD, or a robot that operates on-site for actual work purposes. The learning document data 502 and the control-related information 602 may be document data and control-related information acquired specifically for the learning dataset, or may be document data and control-related information for a robot that operates on-site for actual work purposes. The learning document data 502 corresponds to an example of the "learning document data" in the present disclosure.

[0060] The generation AI device 200 stores the learning model TM generated by the learning device 100. The generation AI device 200 is, for example, a server.

[0061] The information generation device 300 inputs the document data 502 indicating the operation content to be executed by the controlled robot RB into the learning model TM, and causes the learning model TM to generate control-related information 602 which is information related to the control of the controlled robot RB. Therefore, according to Embodiment 1, the control-related information 602 can be easily generated as compared with the case where the control-related information 602 is created only by a person. As a result, for example, the labor of the person involved in creating the control-related information 602 can be reduced. Also, for example, it is possible to suppress the influence of the experience and knowledge of the person involved in creating the control-related information 602 on the quality of the control-related information 602. The controlled robot RB corresponds to an example of the "controlled robot" in the present disclosure.

[0062] Robot RB operates based on control information 603 included in control-related information 602. That is, robot RB operates based on control information 603. Robot RB operates on-site for actual work purposes, but in this case, document data 502 and control-related information 602 may be used as learning document data and control-related information. Also, robot RB may be operated solely for the purpose of acquiring document data 502 and control-related information for learning. From these perspectives, robot RB can be both a robot to be controlled to achieve actual work purposes and a robot for learning.

[0063] Robot RB is, for example, an unmanned robot. However, robot RB may be a manned robot. Also, robot RB is, for example, a conveyance robot such as a carrier vehicle or a conveyor, or a work robot such as an articulated robot.

[0064] Robot system SYS may include a plurality of robots RB. The plurality of robots RB may be of the same type or different types. The type of robot RB is not particularly limited.

[0065] Robot RB1 is, for example, an unmanned conveyance robot that conveys an object to be conveyed. Robot system SYS may include one or more robots RB1. The object to be conveyed is, for example, a cart, a workpiece, a package, or a robot (e.g., a conveyance robot or a work robot).

[0066] An unmanned transport robot is, for example, an unmanned transport vehicle used to transport various manufacturing parts, packages, and other objects to be transported in a manufacturing factory, a logistics warehouse, or the like. The unmanned transport vehicle is, for example, an AGV (Automatic Guided Vehicle) or an AMR (Autonomous Mobile Robot). As an example, the unmanned transport vehicle has an autonomous driving mode and / or a guided driving mode. The guided driving mode is a driving mode in which the unmanned transport vehicle moves along a real or virtual guideline. The autonomous driving mode is a driving mode in which the unmanned transport vehicle can move in an area where no guideline is arranged by estimating its own position. The autonomous driving mode is realized, for example, by a SLAM (Simultaneous Localization and Mapping) function. Further, the unmanned transport vehicle travels while towing the object to be transported. The object to be transported is connected to the unmanned transport vehicle so as to be rotatable, for example.

[0067] In particular, when the robot RB1 is an unmanned transport robot, for example, the work requirements, functional requirements, and non-functional requirements of the document data 502 may include the following information.

[0068] The work requirements (business process definition) may include, for example, the definition of the target transport process, the definition of the transport route and waypoints, the cooperation flow with the logistics management system, the alternative flow in case of an abnormality, and one or more pieces of information among the coordinated transport rules for multiple vehicles.

[0069] The work requirements (work object) may include, for example, one or more pieces of information among the type, shape, size of the object to be transported, the weight range of the package, the specifications of the pallet, trolley, container, the identification method of the package (tag, code, etc.), and the special handling requirements (temperature control, etc.).

[0070] The operation requirements (operation content) may include, for example, one or more pieces of information among an autonomous conveyance task, pickup and drop-off accuracy, high-precision docking control (positioning accuracy), acceleration / deceleration control for preventing load collapse during conveyance, load transfer cooperation control, load pickup procedure, conveyance route and priority, drop-off (unloading) procedure, battery charging timing, method of circulating to multiple destinations, and optimization of empty vehicle running. The autonomous conveyance task indicates a series of work definitions for starting, executing, and completing the conveyance of the object to be conveyed. The pickup and drop-off accuracy indicates the allowable error with respect to the transfer position.

[0071] The operation requirements (operation timing and synchronization) may include, for example, one or more pieces of information among a line synchronization function (interlocking control with the production line), traffic control protocol among multiple robots RB1, automatic distributed processing at conveyance capacity overflow, conveyance task priority setting, operation restriction during a specific time period, load transfer timing accuracy, traffic control among multiple robots RB1, and charging scheduling.

[0072] The operation requirements (operation environmental conditions) may include, for example, one or more pieces of information among the width and shape of the traveling route, charging station arrangement, layout of the load transfer location, designation of the coexistence area with people, traffic rules at intersections and merging points, and requirements for installing dedicated lanes.

[0073] The function requirements (basic functions) may include, for example, one or more pieces of information among the maximum load capacity and towing capacity, maximum traveling speed and acceleration / deceleration performance, continuous traveling time (battery driving time), minimum turning radius and turning performance, gradient traveling ability (maximum uphill angle), and upper limit of the number of carts connected (when towing multiple carts).

[0074] The function requirements (sensing and recognition capabilities) may include, for example, one or more pieces of information among the obstacle detection range and accuracy, traveling route recognition ability (e.g., magnetic tape, QR code (registered trademark), or marker), load detection / identification ability (e.g., barcode, RFID, or image recognition), charging station recognition accuracy, tracking function for people / dynamic obstacles, and docking position recognition accuracy.

[0075] Functional requirements (movement and navigation) may include, for example, one or more of the following information: the driving mode of the robot RB1, the map, the self-position estimation accuracy, the dynamic path replanning, the specific location driving control, the collision avoidance function, the alternative navigation function when the driving path cannot be recognized, the automatic connection and disconnection control of the carriage, the predictive avoidance operation for dynamic obstacles, the traffic adjustment function based on the priority at intersections, the switching rules in the hybrid driving mode, the mapping function and the map update ability, and the path planning algorithm. The driving mode may be a driving mode that implements only the autonomous driving mode, a driving mode that implements only the guided driving mode, or a hybrid driving mode. The hybrid driving mode is a driving mode that implements both the autonomous driving mode and the guided driving mode. The specific location driving control indicates the driving control at specific locations such as narrow roads or intersections. The collision avoidance function may include, for example, one or more of the following information: the obstacle detection accuracy, the collision avoidance control, and the emergency stop trigger. The obstacle detection accuracy indicates, for example, the detection performance or the detection range of stationary or dynamic obstacles. The collision avoidance function indicates the execution control such as stopping, decelerating, or selecting an avoidance path. The switching rules in the hybrid driving mode include the switching control conditions from the autonomous driving mode to the guided driving mode and the switching control conditions from the guided driving mode to the autonomous driving mode.

[0076] Non-functional requirements (safety) may include, for example, one or more of the following information: the emergency stop function (manual and automatic), the collision prevention sensor redundancy, the low-speed driving mode (during collaborative work with people), the fail-safe mechanism (when out of control), the compliance with safety standards, and the shock absorption structure during collisions.

[0077] Non-functional requirements (reliability and robustness) may include, for example, one or more of the following information: the towing stability, the compatibility with mixed-load transportation, and the continuous operation performance. The towing stability indicates, for example, the design requirements for preventing snake-like movement, tipping over, or falling off during carriage connection. The compatibility with mixed-load transportation indicates the ability to carry multiple types of items simultaneously and its control design. The continuous operation performance indicates the function of minimizing the downtime caused by battery replacement and self-charging functions during operation, or the continuous operation time.

[0078] Also, when the robot RB1 is an automated guided vehicle, for example, the environment definition information, operation parameter information, task definition information, safety setting information, cooperation setting information, and operation management information of the control information 603 may include the following information.

[0079] The environment definition information includes, for example, one or more pieces of information among map information, guideline information, drivable floor conditions, charging station positions, logistics base definitions (for example, positions of pick-up and drop-off points), drivable areas, and restricted areas.

[0080] The operation parameter information includes, for example, one or more pieces of information among drive wheel setting information (for example, control parameters of drive wheels), traction load response parameters (for example, drive force or acceleration adjustment values according to the load being carried), battery consumption optimization settings (for example, settings of power consumption adjustment parameters according to the driving pattern), and slip response settings.

[0081] The task definition information includes, for example, one or more pieces of information among operation scenarios, handling sequences, platform operation procedures, charging timing determination conditions, trolley connection and disconnection procedures, and power consumption adjustment parameters according to the driving pattern. The operation scenario is information that defines a series of operation contents to be executed by the robot RB1. The operation scenario includes, for example, information on the destination of the robot RB1, a plurality of operation contents to be executed until reaching the destination, the operation order of the plurality of operations, and the switching conditions of the plurality of operations. The handling sequence is, for example, an operation sequence for receiving and unloading goods.

[0082] The safety setting information includes, for example, one or more pieces of information among human detection range settings, speed limit information (for example, maximum speed for each area), load collapse prevention parameters (for example, acceleration limit values for preventing sudden starts or stops), and traffic rules (for example, deceleration profiles or passing priority determination rules at intersections).

[0083] The cooperation setting information includes, for example, one or more pieces of information among docking conditions, the cooperation setting between the robot RB1 and the gate or elevator, and the cooperation setting with the traffic control system. The docking conditions indicate, for example, the conditions when the robot RB docks with a charging station, a trolley, or a transfer mechanism.

[0084] The operation management information includes, for example, one or more pieces of information among the transfer task allocation policy, the travel route occupancy time management, the task restriction based on the remaining battery level, and the definition of the detour route during congestion. The transfer task allocation policy includes, for example, the optimal distribution algorithm of the transfer tasks among multiple robots RB1.

[0085] Also, the robot RB2 is, for example, a work robot that drives in a state of being arranged on an automated guided vehicle. The work robot is, for example, an articulated robot. The work robot may be, for example, a collaborative robot. A collaborative robot is a robot that executes work in cooperation with a human. The robot system SYS may include one or more robots RB2. Note that the articulated robot may be, for example, a collaborative robot.

[0086] In particular, when the robot RB2 is an articulated robot that functions as a collaborative robot, for example, the work requirements, functional requirements, and non-functional requirements of the document data 502 may include the following information.

[0087] The work requirements (business process definition) may include, for example, one or more pieces of information among the work sharing information between the human and the robot RB2, the procedure of the collaborative work, the role sharing in case of abnormality of the human and the robot RB2, and the judgment criteria for the intervention of the robot RB2.

[0088] The work requirements (work object) may include, for example, one or more pieces of information among the grasping characteristics of the workpiece (for example, shape, hardness, or weight), the assembly tolerance requirements, the position tolerance of the workpiece, and the tool operation requirements (for example, torque or accuracy).

[0089] The operation requirements (operation content) may include, for example, one or more pieces of information among the specific procedures of assembly or processing, the assembly operation by force control, the self-evaluation of work quality, and the operation sequence for the workpiece. The operation sequence may include, for example, the pickup (grasping) of the workpiece, the adjustment of the position and posture, the conveyance along the movement path, the placement at the target position, and the release (release of grasping).

[0090] The operation requirements (operation environmental conditions) may include, for example, one or more pieces of information among the shared work space of humans and robots, the tool and component supply positions, the layout, and the environmental lighting conditions (for vision).

[0091] The operation requirements (operation timing and synchronization) may include, for example, one or more pieces of information among the operation handover timing with humans, the operation adaptation according to the work progress, the synchronization with the work rhythm of humans, and the tact adjustment between multiple processes.

[0092] The functional requirements (basic functions) may include, for example, one or more pieces of information among the movable range and torque performance of each joint, the output limitation function in collaborative applications, the automatic end effector exchange function, the position repeatability accuracy, and the path following accuracy.

[0093] The functional requirements (sensing and recognition capabilities) may include, for example, one or more pieces of information among the joint torque monitoring function, the contact detection sensor with humans, the recognition of the position and posture of the workpiece, the grasping force control of the hand part, the conformity determination of assembled parts, and the adaptation detection to environmental changes.

[0094] The functional requirements (movement and navigation) may include, for example, one or more pieces of information among the obstacle avoidance in the work space, the selection of the optimal solution for joint angles, the singularity avoidance algorithm, the trajectory correction for dynamic obstacles, and the automatic calibration of the work coordinate system.

[0095] Functional requirements (interaction) may include, for example, one or more pieces of information among intention sharing interface with people, direct teaching by force control, haptic feedback function, visual display function of working state, instruction input by gesture recognition, and voice command support.

[0096] Non-functional requirements (safety) may include, for example, one or more pieces of information among collision detection based on force sense monitoring, speed and torque limit functions, power limiting control, dynamic setting of safety monitoring space, automatic deceleration function when a person approaches, and implementation of safety stop classification according to the degree of danger.

[0097] Also, when the robot RB2 is an articulated robot functioning as a collaborative robot, for example, the environmental definition information, operation parameter information, task definition information, safety setting information, cooperation setting information, and operation management information of the control information 603 may include the following information.

[0098] The environmental definition information includes, for example, one or more pieces of information among the position and orientation of the robot reference coordinate system, the tool tip coordinate system, or the work coordinate system, the reference point coordinates for calibrating the tool tip position, the collaborative work area (definition of the boundary of the work space shared by the person and the robot RB2), the shape for interference confirmation, the tool mounting position information, the tool weight, and the tool center of gravity position.

[0099] The operation parameter information includes, for example, one or more pieces of information among the joint movable range, dynamic correction values (for example, parameters for correcting the influence of inertial force, centrifugal force, or gravity during operation), flexibility control settings (control coefficients for adjusting the hardness or softness of the robot RB2), operation path settings (adjustment values for path generation to achieve smooth acceleration and deceleration), positioning accuracy adjustment (gain settings related to the position control accuracy of each axis), force detection settings (torque monitoring values of each joint), and singularity countermeasure settings (for example, control values for avoiding poses that become mechanically unstable).

[0100] The task definition information includes, for example, one or more pieces of information among basic operation type settings (e.g., speed or acceleration settings for point-to-point movement, linear movement, or arc movement), force control operation settings (e.g., control direction and reaction force settings for simultaneously controlling position and force), tool operation conditions, image linkage control settings (e.g., control settings for position correction based on camera images), spatial trajectory definitions (e.g., mathematical definitions of work paths by smooth curves), assembly operation procedures, and electrical and mechanical connection confirmation conditions.

[0101] The safety setting information includes, for example, one or more pieces of information among speed or acceleration limit values, safety evaluation reference values, speed adjustment when a person approaches, minimum safety distance calculated values (e.g., calculation settings for the safety distance to be maintained between a person and the robot RB2), safety monitoring areas, and output limit settings (e.g., maximum torque limit values for each joint and threshold values for collision detection).

[0102] The cooperation setting information includes, for example, one or more pieces of information among image processing cooperation (e.g., camera image acquisition and image data format settings) and composite sensor integration settings (e.g., information on processing settings for combining multiple sensor information).

[0103] The operation management information includes, for example, one or more pieces of information among operation priority settings (e.g., processing priority and resource allocation rules when executing multiple operations), operation record settings, inspection plan settings, remote monitoring settings, abnormal recovery procedures, and cooperation work efficiency settings (adjustment values for optimizing the work sharing between a person and the robot RB2).

[0104] The robot management device 400 manages a plurality of robots RB. The robot management device 400 includes, for example, a PLC (Programmable Logic Controller) and / or an FMS (Fleet Management System).

[0105] The learning device 100, the generation AI device 200, the information generation device 300, the robot management device 400, the robot RB, the document database device 500, the control database device 600, and the environment database device 700 are connected to the network NW and can communicate with each other. The network NW includes, for example, the Internet, a closed network, a public telephone network, a LAN (Local Area Network), and a short-range wireless network.

[0106] Also, at least one industrial device EQ may be connected to the network NW. A plurality of industrial devices EQ may be connected to the network NW. The industrial device EQ refers to a machine or device used for the purpose of manufacturing, processing, transporting, or controlling products in business facilities such as factories or warehouses. The industrial device EQ is, for example, a conveyor or an elevator, but is not particularly limited. The industrial device EQ may include a robot.

[0107] Furthermore, a management system MG may be connected to the network NW. The management system MG manages the industrial device EQ and the robot system SYS. The management system MG includes, for example, an MES (Manufacturing Execution System) and / or a WMS (Warehouse Management System).

[0108] Note that each of the generation AI device 200, the robot management device 400, the document database device 500, the control database device 600, the environment database device 700, and the management system MG includes, for example, a processing unit, a storage unit, a communication unit, an input unit, and an output unit.

[0109] Next, the learning device 100 will be described with reference to FIGS. 2 to 4. FIG. 2 is a block diagram showing a configuration example of the learning device 100. As shown in FIG. 2, the learning device 100 includes a processing unit 110, an input unit 120, an output unit 130, a communication unit 140, and a storage unit 150.

[0110] The input unit 120 is an input device for inputting various types of information to the processing unit 110. For example, the input unit 120 is a keyboard and a pointing device, or a touch panel.

[0111] The output unit 130 outputs various types of information. The output unit 130 includes, for example, a display unit that displays various types of information. The display unit is, for example, a liquid crystal display or an organic electroluminescence display.

[0112] The communication unit 140 is connected to the network NW. The communication unit 140 communicates with various devices connected to the network NW. The communication unit 140 is a communication device that performs communication according to a predetermined communication protocol, and includes, for example, a network interface controller. The predetermined communication protocol is, for example, a protocol compliant with Ethernet (registered trademark), the Internet Protocol Suite, and / or a protocol compliant with a short-range wireless communication standard.

[0113] The processing unit 110 executes various processes (various operations). The processing unit 110 controls the input unit 120, the output unit 130, the communication unit 140, and the storage unit 150. The processing unit 110 includes one or more processors. The processor is a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or an ASIC (Application Specific Integrated Circuit). The processor may operate by a computer program or may operate by hardwired logic.

[0114] The storage unit 150 includes one or more storage devices and stores data and computer programs. The storage unit 150 includes, for example, a main storage device such as a semiconductor memory and an auxiliary storage device such as a semiconductor memory or a storage drive. The storage drive is, for example, a hard disk drive or a solid state drive. The storage unit 150 may be, for example, a non-transitory computer-readable storage medium.

[0115] The processing unit 110 includes an acquisition unit 111 and a learning unit 112. Specifically, the processor of the processing unit 110 functions as the acquisition unit 111 and the learning unit 112 by executing the computer program stored in the storage unit 150.

[0116] Note that the hardware configurations of the processing unit, storage unit, communication unit, input unit, and output unit of the generation AI device 200, robot management device 400, document database device 500, control database device 600, environment database device 700, and management system MG are the same as those of the processing unit 110, storage unit 150, communication unit 140, input unit 120, and output unit 130, respectively.

[0117] FIG. 3 is a diagram for explaining the operation of the learning device 100. For the learning device 100, the robot RB is a learning robot RB. As shown in FIG. 3, the acquisition unit 111 acquires the learning document data 502 from the document database 501. Hereinafter, the learning document data 502 may be referred to as document data 502T. Further, the acquisition unit 111 acquires the learning control-related information 602 from the control database 601. The control-related information 602 includes the control information 603 of the learning robot RB. Hereinafter, the learning control-related information 602 and the control information 603 may be referred to as control-related information 602T and control information 603T, respectively. The storage unit 150 (FIG. 2) stores the document data 502T and the control-related information 602T as a learning data set TD.

[0118] The learning unit 112 accesses the generative AI device 200 and trains the learning dataset TD on the unlearned or pre-learned learning model TMB to construct the learning model TM. That is, based on the learning dataset TD, the learning unit 112 constructs a learning model TMB that generates control-related information 602 (specifically, control information 603) when document data 502 is input.

[0119] Here, in the control database 601, the control information 603T of the learning robot RB includes the control information created via the input device operated by the operator. Therefore, according to Embodiment 1, the control information 603T based on the experience and knowledge of the operator (e.g., an expert) can be learned by the learning model TMB. In this way, by including the high-quality control information 603T reflecting the experience and knowledge of the operator in the learning dataset TD, the accuracy of the obtained learning model TM is improved, and the generation of reliable control information 603 becomes possible.

[0120] Continuing with reference to FIG. 3, the details of the learning model TM will be described. The learning unit 112 executes learning using the learning dataset TD by executing a learning algorithm. The learning unit 112 generates the learning model TM by repeatedly executing learning using a plurality of learning datasets TD.

[0121] As an example, the learning unit 112 generates a learning model TM by fine-tuning a pre-trained large language model (LLM) or vision language model (VLM). The learning algorithm used for fine-tuning is, for example, self-supervised learning, reinforcement learning, supervised learning, or unsupervised learning, or a combination of one or more of them. Specifically, the learning algorithm is linear regression, neural network, deep neural network, decision tree, random forest, gradient boosting, or regularized regression. Note that, for example, the learning algorithm for generating LLM and VLM is, for example, self-supervised learning. Specifically, the learning algorithm is, for example, a neural network, a deep neural network, or a Transformer.

[0122] FIG. 4 is a flowchart showing an example of the learning method according to Embodiment 1. The learning method is executed by the learning device 100. As shown in FIG. 4, the learning method includes steps S1 to S4. Preferably, the learning method further includes steps S5 to S7.

[0123] As shown in FIG. 4, first, in step S1, the acquisition unit 111 acquires document data 502T from the document database 501.

[0124] Next, in step S2, the acquisition unit 111 acquires control-related information 602T (control information 603T) from the control database 601.

[0125] Next, in step S3, the learning unit 112 uses the learning dataset TD (document data 502T and control-related information 602T) to train the LLM or VLM by the first learning algorithm (first learning). The first learning algorithm is, for example, supervised learning. In this case, the document data 502T is the explanatory variable (input), and the control information 603T is the objective variable (output). That is, the control information 603T corresponds to the correct label.

[0126] Next, in step S4, the learning unit 112 determines whether the first end condition is satisfied. The first end condition is, for example, that the number of epochs has reached a first predetermined value. If the determination in step S4 is negative (NO), the process proceeds to step S1. On the other hand, if the determination in step S4 is positive (YES), the process proceeds to step S5.

[0127] Next, in step S5, the acquisition unit 111 acquires the document data 502T from the document database 501.

[0128] Next, in step S5, the learning unit 112 uses the document data 502T to train the LLM or VLM after the first learning is completed by the second learning algorithm (second learning). The second learning algorithm is, for example, reinforcement learning. In this case, for example, the learning unit 112 numerically evaluates the execution result and / or quality of the control information 603T generated by the LLM or VLM after the first learning is completed, calculates a reward value based on the evaluation value that is the result of the numerical evaluation, and performs reinforcement learning of the LLM or VLM using the reward value by the reinforcement learning algorithm. The execution result of the control information 603T is, for example, the operation result of the robot RB by the control information 603T.

[0129] Next, in step S6, the learning unit 112 determines whether the second end condition is satisfied. The second end condition is, for example, that the number of iterations has reached a second predetermined value. If the determination in step S6 is negative (NO), the process proceeds to step S5. On the other hand, if an affirmative determination is made in step S6 (YES), the learning method ends.

[0130] As a result, the LLM or VLM after the completion of the first learning and the second learning is generated as the learning model TM. Therefore, a highly accurate learning model TM can be generated. As described above, the learning model TM is generated by having the LLM or VLM learn the relationship between the document data 502T and the control-related information 602T.

[0131] Next, the information generation device 300 will be described with reference to FIGS. 5 to 7. FIG. 5 is a block diagram showing a configuration example of the information generation device 300. As shown in FIG. 5, the information generation device 300 includes a processing unit 310, an input unit 320, an output unit 330, a communication unit 340, and a storage unit 350. The hardware configurations of the processing unit 310, the input unit 320, the output unit 330, the communication unit 340, and the storage unit 350 are the same as the hardware configurations of the processing unit 110, the input unit 120, the output unit 130, the communication unit 140, and the storage unit 150 of the learning device 100 in FIG. 2, respectively.

[0132] The input unit 320 is an input device for inputting various information to the processing unit 310. The output unit 330 outputs various information. The output unit 330 includes, for example, a display unit that displays various information. The communication unit 340 is connected to the network NW and communicates with various devices connected to the network NW. The processing unit 310 executes various processes (various operations). The storage unit 350 includes one or more storage devices and stores data and computer programs.

[0133] The processing unit 310 includes a reception unit 311, a generation unit 312, and an update unit 313. Specifically, when the processor of the processing unit 310 executes the computer program stored in the storage unit 350, it functions as the reception unit 311, the generation unit 312, and the update unit 313.

[0134] FIG. 6 is a diagram for explaining the operation of the information generation device 300. FIG. 7 is a flowchart showing an example of the information generation method according to the first embodiment. The information generation method inputs document data 502 for the robot RB into the learning model TM, and acquires control-related information 602A of the robot RB from the learning model TM. The robot RB is a control target. The information generation method is executed by the information generation device 300. As shown in FIG. 7, the information generation method includes steps S21 to S26.

[0135] As shown in FIGS. 6 and 7, first, in step S21, the reception unit 311 of the information generation device 300 receives the document data 502 for the robot RB. For example, the reception unit 311 acquires the document data 502 from the document database 501. The storage unit 350 (FIG. 5) stores the document data 502.

[0136] Next, in step S22, the generation unit 312 inputs the document data 502 indicating the operation content to be executed by the robot RB into the learning model TM, and causes the learning model TM to generate control-related information 602A indicating information related to the control of the robot RB. Specifically, the generation unit 312 inputs the document data 502 into the learning model TM, and causes the learning model TM to generate the control information 603A of the robot RB.

[0137] Next, in step S23, the generation unit 312 acquires the control-related information 602A from the learning model TM. The control-related information 602A includes the control information 603A. The storage unit 350 stores the control-related information 602A.

[0138] Next, in step S24, the generation unit 312 transmits the control information 603A to the robot RB as the control target. The robot RB operates based on the control information 603A. According to the first embodiment, since the control information 603A can be generated at high speed by the learning model TM, the operation start time of the robot RB can be shortened.

[0139] Next, in step S25, the update unit 313 updates the control database 601 by storing the control-related information 602A in the control database 601. The control-related information 602A includes the control information 603A. Specifically, the update unit 313 transmits the control-related information 602A to the control database device 600. Then, based on the update instruction from the update unit 313, the control database device 600 stores the control-related information 602A in the control database 601. Note that the control database device 600 may transmit the control information 603A to the robot RB to be controlled. In this case, step S4 can be omitted.

[0140] Next, in step S26, the update unit 313 transmits information (hereinafter, "update notification information") indicating that the control database 601 has been updated to the learning device 100 (FIG. 2). Then, the information generation method ends.

[0141] When the acquisition unit 111 (FIG. 3) of the learning device 100 receives the update notification information, it acquires the newly stored control-related information 602A from the control database 601 as learning control-related information 602T. Further, when the acquisition unit 111 receives the update notification information, it acquires the document data 502 (document data 502 corresponding to the control-related information 602A) used when generating the control-related information 602A from the document database 501 as learning document data 502T. Then, the learning unit 112 updates the parameters of the learning model TM by having the learning model TM relearn the learning dataset TD (document data 502T and control-related information 602T). In this way, when re-learning is executed every time the update notification information is received, it may be described as "sequential re-learning". According to Embodiment 1, since the update notification information is automatically transmitted from the information generation device 300 to the learning device 100, sequential re-learning can be automatically executed.

[0142] Note that the learning unit 112 may, for example, re-learn the learning model TM regularly, or may re-learn the learning model TM when a predetermined number of update notification messages are received. Such re-learning may be described as "batch re-learning". When performing batch re-learning, the control-related information 602T added to the control database 601 after the previous re-learning, and the corresponding document data 502T are used as the learning data set TD (FIG. 3).

[0143] (Embodiment 2) With reference to FIGS. 4 and 7 to 9, a robot system SYS according to Embodiment 2 of the present disclosure will be described. In Embodiment 2, it is mainly different from Embodiment 1 in that the control-related information 602T for learning includes change history information. Hereinafter, the points in which Embodiment 2 is different from Embodiment 1 will be mainly described.

[0144] FIG. 8 is a diagram for explaining the operation of the learning device 100 of the robot system SYS according to Embodiment 2. As shown in FIG. 8, the acquisition unit 111 acquires the learning document data 502T from the document database 501. Further, the acquisition unit 111 acquires the control-related information 602T for learning from the control database 601. The control-related information 602T includes information indicating the change history of the control information 603T (hereinafter, "change history information 604T") in addition to the control information 603T of the learning robot RB. The storage unit 150 (FIG. 2) stores the document data 502T and the control-related information 602T as the learning data set TDX. For example, the control information 603T is version-managed by the change history information 604T in the control database 601. Therefore, there are one or more versions of the control information 603T. Further, the document data 502T is preferably version-managed in the control database 601 corresponding to the change history of the control information 603T. Therefore, in this case, the version of the document data 502T and the version of the control information 603T are associated with each other in the control database 601.

[0145] The learning unit 112 accesses the generative AI device 200 and trains the learning model TMB, which is either unlearned or pre-learned, with the learning dataset TDX to construct the learning model TM. Thus, according to Embodiment 2, by including the change history information 604T of the control information 603T in the learning dataset TDX, the learning model TMB can learn the dynamic and time-series change information of the control information 603T. As a result, the learning model TMB can learn the judgment criteria and improvement trends in the process of changing the control information 603T. Therefore, the accuracy of the obtained learning model TM is further improved, and more reliable control information 603 can be generated. Note that

[0146] Furthermore, in Embodiment 2, the control-related information 602T for learning preferably includes information 605T (hereinafter referred to as "change reason information 605T") regarding the reason for the change indicated by the change history of the control information 603T in addition to the control information 603T and the change history information 604T. According to this preferred example, the learning dataset TDX includes the document data 502T, the control information 603T, the change history information 604T, and the change reason information 605T. Thus, by learning the change reason information 605T, the obtained learning model TM can understand the intention and judgment criteria behind the change and generate more appropriate and explainable control information 603 according to the situation.

[0147] For example, the learning model TM generated by learning the change reason information 605T can infer the intention of the change. Therefore, flexible judgments can be made even for new situations, and highly reliable control information 603 can be generated for new situations.

[0148] For example, by learning the change reason information 605T, the learning model TM can generate explanation information (hereinafter, "explanation information EX") that explains the control information 603. The explanation information EX indicates the reason why the learning model TM has come to generate the control information 603. The explanation information EX is shown in, for example, text format. For example, the explanation information EX includes an explanation indicating "why the control information 603 was generated in this way". Therefore, the explanation information EX of the control information 603 can be provided to the user. As a result, it can be shown to the user that the reliability of the control information 603 is high. Thus, the user can introduce the control information 603 into the robot RB with high confidence. In this way, the learning model TM, which is an explainable AI (XAI: Explainable AI), can be provided.

[0149] With reference to FIGS. 4 and 8, the details of the learning model TM of Embodiment 2 will be described. The points in which Embodiment 2 differs from Embodiment 1 will be mainly described. First, as shown in FIG. 4, in step S1, the acquisition unit 111 acquires the document data 502T from the document database 501.

[0150] Next, in step S2, the acquisition unit 111 acquires control-related information 602T (control information 603T, change history information 604T, and change reason information 605T) from the control database 601. Note that the control-related information 602T may not include the change reason information 605T.

[0151] Next, in step S3, the learning unit 112 uses the learning dataset TDX (document data 502T and control-related information 602T) to train an LLM or a VLM using the first learning algorithm (first training). The first learning algorithm is, for example, self-training. In self-training, based on the change history information 604T and / or the change reason information 605T, each part of the control information 603T is automatically labeled, and training is executed. In this case, the label indicates, for example, the reliability of each part of the control information 603T. For example, a label indicating a high reliability is attached to a part with few changes, and a label indicating a low reliability is attached to a part that is frequently changed. Then, in self-training, using the labeled control information 603T and the document data 502T, the LLM or VLM is trained to establish the correspondence between the control information 603T and the document data 502T such that the control information 603 is generated when the document data 502 is input. In this case, since the reliability indicated by the label functions as the weighting during training, the LLM or VLM can train the correspondence between the control information 603T and the document data 502T more precisely.

[0152] Note that the first learning algorithm may be, for example, unsupervised learning or supervised learning. In unsupervised learning, based on the change history information 604T and / or the change reason information 605T, the transition pattern of the control information 603T with respect to the document data 502T is learned. In supervised learning, for example, the learning dataset TDX is composed of corresponding versions of the document data 502T (explanatory variable) and the control-related information 602T (objective variable).

[0153] Next, steps S4 and S5 are executed. Steps S4 and S5 are the same as steps S4 and S5 in Embodiment 1, respectively.

[0154] Next, in step S6, the learning unit 112 learns the LLM or VLM using the second learning algorithm (second learning). The second learning algorithm is reinforcement learning. This is the same as in Embodiment 1. However, the learning unit 112 can construct a more reliable learning model TM by reflecting the change history information 604T and / or the change reason information 605T in the reward value and performing reinforcement learning.

[0155] Next, step S7 is executed. Step S7 is the same as step S7 in Embodiment 1. By steps S1 to S7, the learning model TM is generated by learning the relationship between the document data 502T and the control-related information 602T in the LLM or VLM.

[0156] Next, with reference to FIG. 9, the operation of the information generation device 300 according to Embodiment 2 will be described. In this case, the information generation method according to Embodiment 2 is the same as the information generation method shown in FIG. 7. Therefore, with reference to FIG. 7 as well, the points where Embodiment 2 differs from Embodiment 1 will be mainly described. As shown in FIG. 7, the information generation method according to Embodiment 2 includes steps S21 to S26. The robot RB is the control target.

[0157] FIG. 9 is a diagram for explaining the operation of the information generation device 300 according to Embodiment 2. As shown in FIGS. 7 and 9, in step S22, the generation unit 312 inputs the document data 502 into the learning model TM and causes the learning model TM to generate the control information 603A of the robot RB. In this case, it is preferable that the generation unit 312 generates, for the learning model TM, in addition to the control information 603A, the explanatory information EX of the control information 603A. Specifically, the generation unit 312 inputs a prompt into the learning model TM and causes the learning model TM to generate the control information 603A and the explanatory information EX. The prompt includes, for example, content that requests to "generate the control information 603A based on the document data 502 and generate the explanatory information EX of the control information 603A".

[0158] Next, in step S23, the generation unit 312 acquires control-related information 602A from the learning model TM. The control-related information 602A includes control information 603A and explanatory information EX.

[0159] Next, in step S24, the generation unit 312 transmits the control information 603A to the robot RB to be controlled.

[0160] Next, in step S25, the update unit 313 updates the control database 601 by storing the control information 603A of the robot RB in the control database 601 together with the explanatory information EX. That is, the control database 601 stores control-related information 602A including the control information 603A and the explanatory information EX. Therefore, according to the second embodiment, the explanatory information EX can be included in the learning dataset TDX. As a result, the learning model TM can self-evaluate or self-verify the relationship between the output (control information 603A) and the basis (explanatory information EX), and the ability of the learning model TM to autonomously detect or avoid inconsistent or incorrect outputs is enhanced.

[0161] That is, returning to FIG. 8, the learning control-related information 602T can further include the explanatory information EX. Then, the learning unit 112 accesses the generation AI device 200 and causes the unlearned or pre-learned learning model TMB to learn the learning dataset TDX including the explanatory information EX, thereby constructing the learning model TM. Note that the explanatory information EX may be further included in the learning control-related information 602T in FIG. 3.

[0162] Note that when the learning device 100 receives the update notification information, it performs sequential re-learning or batch re-learning in the same manner as in the first embodiment.

[0163] Also, in the control database 601, the change history information 604T includes the change history information when the control information 603 was created via an input device operated by an operator. Therefore, according to the second embodiment, the change history information 604T based on the experience and knowledge of the operator (for example, a skilled person) can be learned by the learning model TMB. In this way, by including the high-quality change history information 604T reflecting the experience and knowledge of the operator in the training dataset TD, the accuracy of the obtained training model TM is further improved, and it becomes possible to generate more reliable control information 603.

[0164] Preferably, in the control database 601, the change reason information 605T includes the change reason information created via an input device operated by an operator. Therefore, according to the second embodiment, the accurate change reason information 605T based on the experience and knowledge of the operator can be learned by the learning model TMB.

[0165] The control database device 600 has a version management system. The version management system manages the change history information 604 and the change reason information 605 in the control database 601.

[0166] Here, returning to FIGS. 7 and 9, a modification example of the second embodiment will be described. In the modification example, in step S21, the reception unit 311 receives the document data 502 from the document database 501 and also receives the control information 603 to be corrected (improved) from the control database 601. Then, in step S22, the generation unit 312 inputs the document data 502 and the control information 603 into the learning model TM, and causes the learning model TM to correct (improve) the control information 603. Specifically, the generation unit 312 inputs a prompt into the learning model TM and causes the learning model TM to correct (improve) the control information 603. The prompt includes, for example, content requesting to "correct (improve) the control information 603 created based on the document data 502".

[0167] In this case, it is preferable that the generation unit 312 inputs a prompt to the learning model TM, causing the learning model TM to generate change reason information 605 for the control information 603 in addition to the modification (improvement) of the control information 603. The change reason information 605 includes information regarding the reason for the modification (improvement) of the control information 603 (the reason for changing the control information 603). "The reason for the modification (improvement)" is an example of "the reason for the change". The prompt includes, for example, the content that requests "modifying (improving) the control information 603 created based on the document data 502 and generating the change reason information 605".

[0168] Then, in step S23, the generation unit 312 acquires the modified (improved) control information 603A and the change reason information 605. Further, in step S24, the generation unit 312 transmits the control information 603A to the robot RB.

[0169] Then, in step S25, the update unit 313 updates the control database 601 by storing the modified (improved) control information 603A together with the change history information 604 in the control database 601. That is, the update unit 313 updates the control database device 600 by storing the control information 603A generated by the learning model TM together with the change history information 604 in the control database device 600. Therefore, according to the modification example, the change history information 604 can be utilized as the change history information 604T for learning. Preferably, the update unit 313 updates the control database 601 by storing the control information 603A together with the change history information 604 and the change reason information 605 in the control database 601. According to this preferred example, the change history information 604 and the change reason information 605 can be utilized as the change history information 604T for learning and the change reason information 605.

[0170] Note that it is not necessary to acquire the document data 502 in step S21 of the modification example, nor to input the document data 502 to the learning model TM in step S22 of the modification example. This example is also a kind of modification example.

[0171] Note that, as a modification of Embodiment 1 of FIG. 6, in the same manner as the modification of Embodiment 2, the generation unit 312 may cause the learning model TM to correct (improve) the control information 603. In this case, the document data 502 may or may not be input to the learning model TM.

[0172] (Embodiment 3) With reference to FIGS. 10 and 11, the robot system SYS according to Embodiment 3 of the present disclosure will be described. In Embodiment 3, it is mainly different from Embodiments 1 and 2 in that the learning model TM refers to the control database 601 to generate the control related information 602. Hereinafter, the points in which Embodiment 3 is different from Embodiments 1 and 2 will be mainly described.

[0173] FIG. 10 is a diagram for explaining the operation of the information generation device 300 according to Embodiment 3. As shown in FIG. 10, the processing unit 310 of the information generation device 300 includes a reception unit 311, a search unit 315, a generation unit 312, and an update unit 313. Specifically, the processor of the processing unit 310 functions as the reception unit 311, the search unit 315, the generation unit 312, and the update unit 313 by executing a computer program stored in the storage unit 350.

[0174] FIG. 11 is a flowchart showing an example of the information generation method according to Embodiment 3. The robot RB is a control target. The information generation method is executed by the information generation device 300. As shown in FIG. 11, the information generation method includes steps S31 to S37.

[0175] As shown in FIGS. 10 and 11, first, in step S31, the reception unit 311 receives the document data 502 for the robot RB to be controlled. This is the same as step S21 of Embodiment 1 in FIG. 7.

[0176] Next, in step S32, the search unit 315 searches the control database device 600 for control-related information 602 related to the content of the document data 502 based on a search query generated based on the document data 502. The control database device 600 stores a plurality of pieces of control-related information 602. The control-related information 602 is information related to the control of the robot RB, and includes the control information 603 and the change history information 604 of the robot RB. Preferably, the control-related information 602 further includes change reason information 605. The control-related information 602 output as a search result is referred to by the learning model TM as will be described later. Therefore, the control-related information 602 as a search target corresponds to an example of the "reference control-related information" of the present disclosure. The control database device 600 corresponds to an example of the "database device" of the present disclosure.

[0177] The search query is, for example, the entire content of the document data 502. Note that the search query may be, for example, a part of the content, characteristic content, or summary of the document data 502.

[0178] Next, in step S33, the generation unit 312 inputs the document data 502 and the search result (control-related information 602) by the search unit 315 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A). The learning model TM is the learning model TM (FIG. 9) according to Embodiment 2.

[0179] Next, in step S34, the generation unit 312 acquires the control-related information 602A from the learning model TM. The control-related information 602A includes the control information 603A. The storage unit 350 stores the control-related information 602A.

[0180] Next, steps S35 to S37 are executed. Steps S35 to S37 are the same as steps S24 to S26 in Embodiment 1 of FIG. 7, respectively. However, in step S36, the update unit 313 also updates the control vector database 601V corresponding to the control database 601. That is, the control vector database 601V is updated by storing control-related vector information 602V indicating the control-related information 602A in the control vector database 601V.

[0181] As described above with reference to FIGS. 10 and 11, according to Embodiment 3, not only the document data 502 but also the control-related information 602 as the search result by the search unit 315 is input to the learning model TM. Therefore, the learning model TM can generate the control information 603A based on the document data 502 with reference to the control-related information 602 stored in the control database device 600. Therefore, the learning model TM can incorporate the latest control-related information 602 (control information 603, change history information 604, and change reason information 605). In addition, since the search is executed by the search query based on the document data 502 for the robot RB to be controlled, the learning model TM can incorporate the control-related information 602 similar to or related to the required specifications indicated by the document data 502. As a result, the learning model TM can generate more reliable control information 603A.

[0182] Continuing with the detailed description with reference to FIG. 10. The control database device 600 stores a control vector database 601V. The control vector database 601V stores a plurality of control-related vector information 602V each indicating a plurality of control-related information 602. The control-related vector information 602V indicates information representing the characteristics of the control-related information 602 stored in the control database 601 by a plurality of numerical vectors. In this case, the characteristics are, for example, semantic and / or structural characteristics of the control-related information 602. The numerical vector is indicated by, for example, an array of multi-dimensional real values. In the control vector database 601V and the control database 601, the control-related vector information 602V and the corresponding control-related information 602 are associated with each other.

[0183] Each of the control-related vector information 602V includes control vector information 603V and change history vector information 604V. Each of the control-related vector information 602V preferably further includes change reason vector information 605V. The control vector information 603V indicates information representing the characteristics of the control information 603 by a plurality of numerical vectors. In this case, the characteristics are, for example, semantic and / or structural characteristics of the control information 603. The change history vector information 604V indicates information representing the characteristics of the change history information 604 by a plurality of numerical vectors. In this case, the characteristics are, for example, semantic and / or structural characteristics of the change history information 604. The change reason vector information 605V indicates information representing the characteristics of the change reason information 605 by a plurality of numerical vectors. In this case, the characteristics are, for example, semantic and / or structural characteristics of the change reason information 605.

[0184] Also, the processing unit 310 of the information generation device 300 further includes a preprocessing unit 314. Specifically, the processor of the processing unit 310 functions as the reception unit 311, the preprocessing unit 314, the search unit 315, the generation unit 312, and the update unit 313 by executing a computer program stored in the storage unit 350.

[0185] Furthermore, a vector generation device 800 is connected to the network NW (Fig. 1). The vector generation device 800 converts text and images into numerical vectors. The vector generation device 800 includes, for example, a multimodal embedding model. A multimodal embedding model is a machine learning model designed to commonly process and understand different types of data and convert different types of data into numerical vectors. For example, the multimodal embedding model has the feature of being able to infer the semantic correspondence between an image and text by mapping the image and text into the same embedding space and vectorizing them.

[0186] The preprocessing unit 314 inputs the document data 502 to the vector generation device 800. The vector generation device 800 converts the document data 502 into numerical vector information. The numerical vector information includes a plurality of numerical vectors. In this case, it is preferable that the preprocessing unit 314 segments the text in the document data 502. Segmentation is a process of dividing text into semantic or structural units. By segmentation, the text is divided into a plurality of segments. The preprocessing unit 314 inputs the plurality of segments to the vector generation device 800. The vector generation device 800 generates a plurality of numerical vectors respectively indicating the plurality of segments. Also, when the document data 502 includes one or more images, the preprocessing unit 314 inputs the images to the vector generation device 800. Furthermore, when the document data 502 includes one or more diagrams, the preprocessing unit 314 converts the diagrams into images and inputs the images indicating the diagrams to the vector generation device 800. The vector generation device 800 generates numerical vectors indicating the images.

[0187] Segmentation is preferably chunking. A chunk is a block of text with meaning or function. For example, the preprocessing unit 314 uses an LLM or VLM to split the text in the document data 502 into a plurality of chunks. Then, the preprocessing unit 314 converts each chunk into a numerical vector by the vector generation device 800. In chunking, the preprocessing unit 314 may perform chunking using the learning model TM, or may perform chunking using another LLM or VLM.

[0188] The search unit 315 calculates the similarity of each numerical vector between the numerical vector information indicating the document data 502 and the control-related vector information 602V in the control vector database 601V. Then, the search unit 315 calculates the similarity (hereinafter, "comprehensive similarity") between the numerical vector information indicating the document data 502 and the control-related vector information 602V based on the calculated similarities of each. The search unit 315 calculates the comprehensive similarity for each control-related vector information 602V by comparing the numerical vector information indicating the document data 502 with each control-related vector information 602V. Then, the search unit 315 selects the top N comprehensive similarities with high similarities among the plurality of comprehensive similarities. N represents an integer of 1 or more. The search unit 315 acquires the N control-related information 602 corresponding to the N control-related vector information 602V indicating the top N comprehensive similarities from the control database 601. The search unit 315 outputs the N control-related information 602 as search results to the generation unit 312.

[0189] The generation unit 312 inputs the N pieces of control-related information 602 and the document data 502 output as search results from the search unit 315 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A) corresponding to the content of the document data 502. For example, the generation unit 312 creates a prompt and inputs the prompt into the learning model TM. The prompt includes, for example, the content that requests to "generate control information 603A that can realize the required specifications indicated by the document data 502 with reference to the N pieces of control-related information 602". As a result, the learning model TM generates control information 603A corresponding to the document data 502 with reference to the N pieces of control information 603 output as search results by the search unit 315.

[0190] In addition, in Embodiment 3, as the learning model TM, the learning model TM of Embodiment 1 (including modifications) in FIG. 6 may be used. Also, when using the learning model TM of Embodiment 1 (including modifications) in FIG. 6 in Embodiment 3, or when using the learning model TM of Embodiment 3 (including modifications), the control-related information 602 in the control database 601 may not include the change history information 604 and the change reason information 605. In this case, the control-related vector information 602V in the control vector database 601V may not include the change history vector information 604V and the change reason vector information 605V. These examples are also a kind of modifications.

[0191] (Embodiment 4) With reference to FIGS. 1, 12, and 13, the robot system SYS according to Embodiment 4 of the present disclosure will be described. Embodiment 4 is mainly different from Embodiment 3 in that the learning model TM generates control-related information 602 with reference to the environmental database device 700. Hereinafter, the points where Embodiment 4 differs from Embodiment 3 will be mainly described.

[0192] As shown in FIG. 1, the environmental database device 700 includes at least one environmental database 701. The environmental database device 700 may store a plurality of environmental databases 701. The environmental database 701 stores one or more pieces of environmental information 702. In this case, for example, the environmental information 702 is stored in text format.

[0193] The environmental information 702 includes, for example, information regarding the technical environment or legal environment of the robot RB. The environmental information 702 is determined, for example, in a country, region, or business company. "Legal" is not limited to having legal binding force, and includes "quasi-legal" such as self-regulation.

[0194] The environmental information 702 regarding the technical environment is, for example, information that defines industry standards related to the robot RB (hereinafter, "industry standard information"). The industry standard information is, for example, information that defines technical standards related to the robot RB. The industry standard information is, for example, communication standard information, control-related standard information, robot performance standard information, or interface specification information. The communication standard information is, for example, EtherCAT, PROFINET, Modbus, or CANopen. The control-related standard information is, for example, PLCopen, ROS, or OPC-UA. The robot performance standard information is, for example, ISO 9283 (performance test) or ISO 10218 (robot design). The interface specification information is, for example, the USB standard.

[0195] The environmental information 702 regarding laws is, for example, information that defines legal regulations regarding the robot RB (hereinafter referred to as "legal regulation information"). The legal regulation information is, for example, information that defines legal standards regarding the robot RB. The legal regulation information is, for example, safety standard information, regional law regulation information, or chemical substance regulation information. The safety standard information is, for example, the Machinery Directive, RoHS, REACH, WEEE, or the Industrial Safety and Health Act. The regional law regulation information is, for example, the Japanese Electrical Appliance Safety Act (PSE), the US OSHA standards, or the EU CE marking requirements. The chemical substance regulation information is, for example, GHS, TSCA (USA), or PRTR (Japan).

[0196] FIG. 12 is a diagram for explaining the operation of the information generation device 300 according to Embodiment 4. FIG. 13 is a flowchart showing an example of the information generation method according to Embodiment 4. The robot RB is a control target. The information generation method is executed by the information generation device 300. As shown in FIG. 13, the information generation method includes steps S41 to S48.

[0197] As shown in FIGS. 12 and 13, first, in step S41, the reception unit 311 receives the document data 502 for the robot RB that is the control target. This is the same as step S31 in FIG. 11.

[0198] Next, in step S42, the search unit 315 searches the control database device 600 for control-related information 602 related to the content of the document data 502 based on a search query generated based on the document data 502 (search process). This is the same as step S32 in FIG. 11.

[0199] Next, in step S43, the search unit 315 searches the environment database device 700 for environment information 702 related to the content of the document data 502 based on the search query generated based on the document data 502. The environment database device 700 stores a plurality of pieces of environment information 702. The search query is, for example, the entire content of the document data 502.

[0200] Next, in step S44, the generation unit 312 inputs the document data 502 and the search results (control-related information 602 and environment information 702) by the search unit 315 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A). The learning model TM is the learning model TM (FIG. 9) according to Embodiment 2.

[0201] Next, in step S45, the generation unit 312 acquires the control-related information 602A from the learning model TM. The control-related information 602A includes the control information 603A. The storage unit 350 stores the control-related information 602A.

[0202] Next, steps S46 to S48 are executed. Steps S46 to S48 are the same as steps S35 to S37 in FIG. 11, respectively.

[0203] As described above with reference to FIGS. 12 and 13, according to Embodiment 4, not only the document data 502 but also the environment information 702 as the search result by the search unit 315 is input to the learning model TM. The environment information 702 is, for example, industry standard information or legal regulation information. Therefore, the learning model TM can generate the control information 603A that reflects the industry standard information or legal regulation information to which the robot RB to be controlled should conform. Note that the effect when the control-related information 602 as the search result by the search unit 315 is input to the learning model TM is the same as that in Embodiment 3.

[0204] Continuing with reference to FIG. 12 for a detailed description. The environmental database device 700 stores an environmental vector database 701V. The environmental vector database 701V stores a plurality of environmental vector information 702V each indicating a plurality of environmental information 702. The environmental vector information 702V indicates information in which the characteristics of the environmental information 702 stored in the environmental database 701 are represented by a plurality of numerical vectors. In this case, the characteristics are, for example, semantic and / or structural characteristics of the environmental information 702. In the environmental vector database 701V and the environmental database 701, the environmental vector information 702V and the corresponding environmental information 702 are associated with each other.

[0205] The search unit 315 calculates the similarity of each numerical vector between the numerical vector information indicating the document data 502 and the environmental vector information 702V in the environmental vector database 701V. Then, based on the calculated similarities, the search unit 315 calculates the similarity (hereinafter, "comprehensive similarity") between the numerical vector information indicating the document data 502 and the environmental vector information 702V. The search unit 315 calculates the comprehensive similarity for each environmental vector information 702V by comparing the numerical vector information indicating the document data 502 with each environmental vector information 702V. Then, the search unit 315 selects the top M comprehensive similarities with high similarities among the plurality of comprehensive similarities. M represents an integer of 1 or more. The search unit 315 acquires the M environmental information 702 corresponding to the M environmental vector information 702V indicating the top M comprehensive similarities from the environmental database 701. The search unit 315 outputs the M environmental information 702 as search results to the generation unit 312.

[0206] The generation unit 312 inputs the N pieces of control-related information 602 output as search results from the search unit 315, the M pieces of environment information 702 output as search results from the search unit 315, and the document data 502 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A) corresponding to the content of the document data 502. As a result, the learning model TM can generate control information 603A corresponding to the document data 502 with reference to the N pieces of control information 603 and the M pieces of environment information 702 output as search results by the search unit 315.

[0207] In addition, in Embodiment 4, the generation unit 312 does not necessarily input the control-related information 602 into the learning model TM. In this case, the generation unit 312 inputs the M pieces of environment information 702 output as search results from the search unit 315 and the document data 502 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A) corresponding to the content of the document data 502. In this case, the control database device 600 does not necessarily include the control vector database 601V. These examples are also a kind of modification.

[0208] Also, the search unit 315 may search for different types of environment information 702 from a plurality of environment databases 701 of different types. In this case, the generation unit 312 inputs the different types of environment information 702 as search results into the learning model TM. The different types of environment databases 701 are, for example, an environment database 701 storing environment information 702 indicating industry standard information and an environment database 701 storing environment information 702 indicating legal regulation information. These examples are also a kind of modification.

[0209] In addition, in the fourth embodiment, as the learning model TM, the learning model TM of the first embodiment (including modifications) shown in FIG. 6 may be used. Further, the control-related information 602 in the control database 601 may not include the change history information 604 and the reason-for-change information 605. In this case, the control-related vector information 602V in the control vector database 601V may not include the change history vector information 604V and the reason-for-change vector information 605V. These examples are also a kind of modifications.

[0210] (Embodiment 5) With reference to FIGS. 14 and 15, a robot system SYS according to the fifth embodiment of the present disclosure will be described. The fifth embodiment is mainly different from the third and fourth embodiments in that the learning model TM has pre-learned the environmental information 702 stored in the environmental database device 700. Hereinafter, the differences between the fifth embodiment and the third and fourth embodiments will be mainly described.

[0211] FIG. 14 is a diagram for explaining the operation of the learning device 100 according to the fifth embodiment. For the learning device 100, the robot RB is a learning robot RB. As shown in FIG. 14, before the learning unit 112 causes the learning model TMB to learn the learning data set TDX (document data 502T and control-related information 602T), the learning unit 112 causes the learning model TMB to learn the environmental information 702 stored in the environmental database 701.

[0212] Specifically, the acquisition unit 111 acquires the environmental information 702 from the environmental database 701. The storage unit 150 (FIG. 2) stores the environmental information 702. Then, the learning unit 112 accesses the generation AI device 200 to cause the learning model TMB to pre-learn the environmental information 702. The environmental information 702 is industry standard information or legal regulatory information related to the robot RB.

[0213] After the pre-training of the environmental information 702 is completed, the learning unit 112 causes the learning model TMB pre-trained with the environmental information 702 to learn the learning dataset TDX (document data 502T and control-related information 602T). As a result, a learning model TM that has learned not only the learning dataset TDX but also the environmental information 702 is generated.

[0214] Note that the acquisition unit 111 may acquire different types of environmental information 702 from a plurality of different types of environmental databases 701. In this case, the learning unit 112 causes the learning model TM to pre-learn different types of environmental information 702. The different types of environmental databases 701 are, for example, an environmental database 701 storing environmental information 702 indicating industry standard information and an environmental database 701 storing environmental information 702 indicating legal regulatory information.

[0215] Referring to FIG. 15, a learning method according to Embodiment 5 will be described. As an example, the learning unit 112 generates the learning model TM by gradually fine-tuning a pre-trained LLM or VLM.

[0216] FIG. 15 is a flowchart showing an example of the learning method according to Embodiment 5. As shown in FIG. 15, the learning method includes steps S51 to S60. Steps S51 to S53 are the first stage of fine-tuning, and steps S54 to S60 are the second stage of fine-tuning.

[0217] First, in step S51, the acquisition unit 111 acquires environmental information 702 from the environmental database 701.

[0218] Next, in step S52, the learning unit 112 uses the environmental information 702 to cause an LLM or VLM to learn by a learning algorithm. The learning algorithm is, for example, self-supervised learning.

[0219] Next, in step S53, the learning unit 112 determines whether pre-training has been completed for all the environmental information 702. If a negative determination is made in step S53 (NO), the process proceeds to step S51. On the other hand, if an affirmative determination is made in step S53 (YES), the process proceeds to step S54. In this way, the pre-learning of the environmental information 702 is completed.

[0220] Next, steps S54 to S60 are executed. Steps S54 to S60 are the same as steps S1 to S7 according to Embodiment 2 described with reference to FIG. 4.

[0221] Next, with reference to FIGS. 10 and 11, the operation of the information generation apparatus 300 according to Embodiment 5 will be described. As shown in FIGS. 10 and 11, in step S33, the generation unit 312 inputs the document data 502 and the search result (control-related information 602) by the search unit 315 into the learning model TM, and causes the learning model TM to generate control-related information 602A (specifically, control information 603A). In this case, the learning model TM has pre-learned the environmental information 702. The environmental information 702 is, for example, industry standard information or legal regulation information regarding the robot RB. Therefore, according to Embodiment 5, the learning model TM can generate control information 603A that reflects the industry standard information or legal regulation information to which the robot RB to be controlled should conform.

[0222] Note that the information generation apparatus 300 according to Embodiment 1 (including modifications), Embodiment 2 (including modifications), and Embodiment 4 (including modifications) may use the learning model TM according to Embodiment 5. These examples are also a kind of modification.

[0223] (Embodiment 6) With reference to FIGS. 8, 16, and 17, the robot system SYS according to Embodiment 6 of the present disclosure will be described. In Embodiment 6, it is mainly different from Embodiments 1 to 5 in that the learning model TM includes information that directly or indirectly indicates the possibility of incompleteness of the document data 502. Hereinafter, the differences between Embodiment 6 and Embodiment 3 will be mainly described.

[0224] As shown in FIG. 8, the learning unit 112 according to Embodiment 6 accesses the generation AI device 200 and causes the unlearned or pre-learned learning model TMB to learn the learning dataset TDX (document data 502T and control-related information 602T), thereby constructing the learning model TM. That is, the learning unit 112 constructs a learning model TM that generates the control-related information 602B when the document data 502 is input based on the learning dataset TDX. The control-related information 602B is information related to the control of the robot RB to be controlled. The control-related information 602B includes information (hereinafter, "defect suggestion information 603B") that directly or indirectly indicates the possibility of defects in the document data 502.

[0225] Referring to FIGS. 4 and 8, the details of the learning model TM of Embodiment 6 will be described. The differences between Embodiment 6 and Embodiment 2 will be mainly described. First, as shown in FIG. 4, steps S1 and S2 are executed, and the acquisition unit 111 acquires the document data 502T and the control-related information 602T (control information 603T, change history information 604T, and change reason information 605T). Note that the control-related information 602T may not include the change reason information 605T.

[0226] Next, in step S3, the learning unit 112 uses the learning dataset TDX (document data 502T and control-related information 602T) to train an LLM or a VLM using a first learning algorithm (first training). The first learning algorithm is, for example, self-training. In self-training, based on the change history information 604T and / or the change reason information 605T, the relationship between the deficiencies (e.g., ambiguity, contradiction, insufficiency, or error) in the document data 502T and the change pattern of the control information 603T corresponding to the deficiencies is learned. In this case, for example, each part of the document data 502T is automatically labeled and training is executed. The label indicates, for example, the reliability of each part of the document data 502T. For example, a label indicating low reliability is attached to the corresponding part of the document data 502T corresponding to the part of the control information 603T with a high change frequency indicated by the change history information 604T and / or the change reason information 605T, and a label indicating high reliability is attached to the corresponding part of the document data 502T corresponding to the part with a low change frequency. Then, in self-training, the LLM or VLM is trained to generate deficiency suggestion information 603B when the document data 502 is input, using the labeled document data 502T and the control information 603T. In this case, since the reliability indicated by the label functions as the weighting during training, the LLM or VLM can learn the relationship between the deficiencies in the document data 502T and the change pattern of the control information 603T corresponding to the deficiencies more precisely.

[0227] Next, steps S4 and S5 are executed. Steps S4 and S5 are the same as steps S4 and S5 in Embodiment 1, respectively.

[0228] Next, in step S6, the learning unit 112 trains the LLM or VLM using the second learning algorithm (second training). The second learning algorithm is reinforcement learning. In this case, for example, the learning unit 112 numerically evaluates the quality of the defect suggestion information 603B generated by the LLM or VLM after the first training is completed, calculates a reward value based on the evaluation value that is the result of the numerical evaluation, and uses the reward value to perform reinforcement learning on the LLM or VLM by means of a reinforcement learning algorithm. The quality of the defect suggestion information 603B is determined based on at least one of, for example, the accuracy of the suggestion, the comprehensiveness of the suggestion (the degree of not missing potential defects), and the usefulness of the suggestion.

[0229] Next, step S7 is executed. Step S7 is the same as step S7 in Embodiment 2. Through steps S1 to S7, a learning model TM that generates defect suggestion information 603B by inputting document data 502 based on the LLM or VLM is constructed.

[0230] According to Embodiment 6, by including the change history information 604T of the control information 603T in the learning dataset TDX, the learning model TMB can learn the dynamic and time-series change information of the control information 603T. As a result, the learning model TMB can learn the judgment criteria and improvement trends in the process of changing the control information 603T. Therefore, the accuracy of the obtained learning model TM is further improved, and more reliable defect suggestion information 603B can be generated.

[0231] Furthermore, in Embodiment 6, it is preferable that the control-related information 602T includes change reason information 605T in addition to the control information 603T and the change history information 604T. According to this preferred example, the learning dataset TDX includes document data 502T, control information 603T, change history information 604T, and change reason information 605T. In this way, by learning the change reason information 605T, the obtained learning model TM can understand the intention and judgment criteria behind the change and generate more appropriate and explainable defect suggestion information 603B according to the situation.

[0232] For example, the learning model TM generated by learning the change reason information 605T can infer the intention of the change. Therefore, flexible judgment can be made even for new situations, and reliable defect suggestion information 603B can be generated for new situations.

[0233] For example, by learning the change reason information 605T, the learning model TM can generate explanatory information (hereinafter referred to as "explanatory information EY") that explains the defect suggestion information 603B. The explanatory information EY indicates the basis for the learning model TM to generate the defect suggestion information 603B. The explanatory information EY is shown in, for example, text format. For example, the explanatory information EY includes an explanation indicating "why it was generated in this way" for the defect suggestion information 603B. Therefore, the explanatory information EY of the defect suggestion information 603B can be provided to the user. As a result, it can be shown to the user that the defect suggestion information 603B is highly reliable. Thus, the user can accept the defect suggestion information 603B with high confidence. In this way, the learning model TM, which is an explainable AI (XAI), can be provided.

[0234] FIG. 16 is a diagram for explaining the operation of the information generation device 300 according to Embodiment 6. FIG. 17 is a flowchart showing an example of the information generation method according to Embodiment 6. The robot RB is the control target. The information generation method is executed by the information generation device 300. As shown in FIG. 17, the information generation method includes steps S71 to S75.

[0235] As shown in FIGS. 16 and 17, first, in step S71, the reception unit 311 receives the document data 502 for the robot RB that is the control target. This is the same as step S31 in FIG. 11.

[0236] Next, in step S72, the search unit 315 searches the control database device 600 for control-related information 602 related to the content of the document data 502 based on the search query generated based on the document data 502. This is the same as step S32 in FIG. 11.

[0237] Next, in step S73, the generation unit 312 inputs the document data 502 and the search result (control-related information 602) by the search unit 315 into the learning model TM, and causes the learning model TM to generate control-related information 602B (specifically, deficiency suggestion information 603B).

[0238] Next, in step S74, the generation unit 312 acquires the control-related information 602B from the learning model TM. The control-related information 602B includes the deficiency suggestion information 603B. The storage unit 350 stores the control-related information 602B.

[0239] Next, in step S75, the update unit 313 updates the document database 501 by storing the deficiency suggestion information 603B in association with the document data 502 in the document database 501. Then, the information generation method ends.

[0240] As described above with reference to FIGS. 16 and 17, according to the sixth embodiment, the learning model TM generates the deficiency suggestion information 603B. Therefore, the user can recognize the possibility of deficiencies in the document data 502 based on the deficiency suggestion information 603B. As a result, the user can improve the document data 502 based on the deficiency suggestion information 603B.

[0241] Also, in the sixth embodiment, the control-related information 602B (specifically, the deficiency suggestion information 603B) includes at least one of information indicating the possibility of deficiencies in the document data 502 and information indicating the possibility of changes in the content of the document data 502.

[0242] The user can perform correction or improvement of the document data 502 based on the information indicating the possibility of deficiencies in the document data 502. The information indicating the possibility of deficiencies in the document data 502 may include, for example, one or more of the location of the deficient part in the document data 502, improvement proposals for the deficient part, and alternative proposals for the deficient part.

[0243] In addition, based on information indicating the possibility of changes in the content of the document data 502, the user can consider future revisions or improvements to the document data 502. The information indicating the possibility of changes in the content of the document data 502 may include, for example, one or more pieces of information among the location of the part in the document data 502 where changes may occur in the future, improvement plans for the part where changes may occur in the future, and alternative plans for the part where changes may occur in the future.

[0244] In addition, in Embodiment 6, not only the document data 502 but also the control-related information 602 as the search result by the search unit 315 is input to the learning model TM. Therefore, the learning model TM can generate the defect suggestion information 603B based on the document data 502 with reference to the control-related information 602 stored in the control database device 600. Therefore, the learning model TM can incorporate the latest control-related information 602 (control information 603, change history information 604, and change reason information 605). In addition, since the search is executed by the search query based on the document data 502 for the robot RB to be controlled, the learning model TM can incorporate the control-related information 602 similar to or related to the required specifications indicated by the document data 502. As a result, the learning model TM can generate more reliable defect suggestion information 603B.

[0245] Note that in Embodiment 1 (including modifications), Embodiment 2 (including modifications), Embodiment 4 (including modifications), and Embodiment 5 (including modifications), the learning unit 112 may construct a learning model TM that generates control-related information 602 (specifically, defect suggestion information 603B) when the document data 502 is input based on the learning data set TD or the learning data set TD. These examples are also a kind of modification.

[0246] For example, in Embodiment 1, the learning unit 112 can construct a learning model TM that generates the defect suggestion information 603B when the document data 502 is input, based on the learning dataset TD (control information 603T and document data 502T). Therefore, in this case, the defect suggestion information 603B can be easily generated as compared with the case of creating the defect suggestion information 603B only by a person. As a result, for example, the labor of a person checking the document data 502 can be reduced. Also, for example, it is possible to suppress the fact that the amount of experience and knowledge of a person checking the defect suggestion information 603B affects the accuracy of the defect suggestion information 603B.

[0247] Also, for example, in Embodiment 1, the learning unit 112 can generate a learning model TMB that has the ability to detect defects in the document data 502T and the ability to generate the defect suggestion information 603B, by causing the learning model TMB that has pre-learned the environmental information 702 to learn the correspondence relationship (for example, a structural and / or semantic correspondence relationship) between the document data 502T and the control information 603T, using supervised learning with the learning dataset TDX (document data 502T and control information 603T).

[0248] Also, for example, in Embodiment 5, the learning unit 112 can construct a learning model TM that generates the defect suggestion information 603B when the document data 502 is input, by causing the learning model TMB that has pre-learned the environmental information 702 to learn the learning dataset TDX or the learning dataset TD. Therefore, in this case, the learning model TM can generate the defect suggestion information 603B that reflects the industry standard information or legal regulation information to which the robot RB to be controlled should conform. This also applies to the case of causing the learning model TM to generate the defect suggestion information 603B in Embodiment 4.

[0249] The preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings. However, the technical scope of the present disclosure is not limited to such examples. It is obvious that those with ordinary knowledge in the technical field of the present disclosure can come up with various modification examples or correction examples within the scope of the technical idea described in the claims, and these are naturally understood to belong to the technical scope of the present disclosure.

[0250] The apparatus described in this specification may be realized as a single apparatus, or may be realized by a plurality of apparatuses (such as cloud servers) partially or entirely connected by a network.

[0251] A series of processes by the apparatus described in this specification may be realized using any of software, hardware, and a combination of software and hardware. It is possible to create a computer program for realizing each function of the processing units 110 and 310 according to this embodiment and install it on a PC or the like. Also, a computer-readable recording medium storing such a computer program can be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above computer program may be distributed via a network, for example, without using a recording medium.

[0252] In FIGS. 4, 7, 11, 13, 15, and 17, the processing unit 110 or 310 executes each step included in the learning method or the information generation method by executing the computer program stored in the storage unit 150 or 350. In other words, the computer program causes the processing unit 110 or 310 to execute each step included in the learning method or the information generation method. The processing unit 110 or 310 corresponds to an example of the "computer" of the present disclosure. Further in other words, the computer program product realizes each step included in the learning method or the information generation method when the computer program is executed by the processing unit 110 or 310. Also, the learning model TM is constructed by learning the learning data set TD or TDX, and functions the computer so as to generate control related information 602A or 602B indicating information related to the control of the robot RB to be controlled. Specifically, the learning model TM functions the computer so as to input document data 502 indicating the operation content to be executed by the robot RB to be controlled and generate control related information 602A or 602B.

[0253] Also, the effects described in this specification are merely illustrative or exemplary and not limiting. That is, the technology according to the present disclosure may exhibit other effects obvious to those skilled in the art from the description of this specification, together with or instead of the above effects.

[0254] Note that the following configurations also belong to the technical scope of the present disclosure.

[0255] (Item 1) A generation unit that inputs document data indicating the operation content to be executed by the robot to be controlled to a learning model and causes the learning model to generate control related information indicating information related to the control of the robot to be controlled; A storage unit that stores the control related information, and The learning model is constructed by learning a learning data set, The learning dataset includes learning document data indicating the operation content to be executed by the learning robot and learning control-related information that is information related to the control of the learning robot. The learning control-related information includes control information of the learning robot created based on the learning document data, and is an information generation device.

[0256] (Item 2) The learning control-related information includes information indicating the change history of the control information of the learning robot, and is the information generation device according to Item 1.

[0257] (Item 3) The learning control-related information includes information related to the reason for the change indicated by the change history, and is the information generation device according to Item 2.

[0258] (Item 4) The control information of the learning robot includes control information created via an input device operated by an operator, and is the information generation device according to any one of Items 1 to 3.

[0259] (Item 5) The control-related information includes control information of the controlled robot, and is the information generation device according to any one of Items 1 to 4.

[0260] (Item 6) The control-related information includes information directly or indirectly indicating the possibility of incompleteness of the document data, and is the information generation device according to any one of Items 1 to 4.

[0261] (Item 7) The control-related information includes at least one of information indicating the possibility of incompleteness in the document data and information indicating the possibility of changes in the content of the document data, and is the information generation device according to any one of Items 1 to 4.

[0262] (Item 8) It further includes a search unit that searches a database device for reference control-related information related to the content of the document data. The reference control-related information is information related to the control of the robot. The generation unit inputs the document data and the search result by the search unit into the learning model, and causes the learning model to generate the control-related information. The information generation device according to any one of items 1 to 7.

[0263] (Item 9) The database device stores information indicating the change history of the control information of the robot. The information generation device according to item 8.

[0264] (Item 10) The control-related information includes the control information of the robot to be controlled. The information generation device according to item 9 further includes an update unit that stores the control information of the robot to be controlled generated by the learning model in the database device together with the information indicating the change history, thereby updating the database device.

[0265] (Item 11) The learning model has pre-learned industry standard information or legal regulation information related to the robot. The information generation device according to any one of items 1 to 10.

[0266] (Item 12) It further includes a search unit that searches a database device for industry standard information or legal regulation information related to the content of the document data. The generation unit inputs the document data and the search result by the search unit into the learning model, and causes the learning model to generate the control-related information. The information generation device according to any one of items 1 to 11.

[0267] (Item 13) The learning robot is an automated guided vehicle. The information generation device according to any one of Items 1 to 12, wherein the robot to be controlled is an automated guided vehicle (AGV).

[0268] (Item 14) The information generation device according to Item 13, wherein the automated guided vehicle (AGV) transports a work robot that drives while being placed on the automated guided vehicle (AGV).

[0269] (Item 15) inputting document data indicating the operation content to be executed by the robot to be controlled into the learning model, and causing the learning model to generate control-related information indicating information related to the control of the robot to be controlled; obtaining the control-related information; and wherein the learning model is constructed by learning a learning data set, wherein the learning data set includes learning document data indicating the operation content to be executed by the learning robot and learning control-related information that is information related to the control of the learning robot, wherein the learning control-related information includes control information of the learning robot created based on the learning document data, an information generation method.

[0270] (Item 16) causing a computer to input document data indicating the operation content to be executed by the robot to be controlled into the learning model, and causing the learning model to generate control-related information indicating information related to the control of the robot to be controlled; obtaining the control-related information; and wherein the learning model is constructed by learning a learning data set, wherein the learning data set includes learning document data indicating the operation content to be executed by the learning robot and learning control-related information that is information related to the control of the learning robot, wherein the learning control-related information includes control information of the learning robot created based on the learning document data, a computer program.

[0271] (Item 17) A learning model that causes a computer to function so as to generate control-related information indicating information regarding control of a robot to be controlled, which is constructed by learning a learning dataset. The computer is caused to function so as to input document data indicating the operation content to be executed by the robot to be controlled and generate the control-related information. The learning dataset includes learning document data indicating the operation content to be executed by a learning robot and learning control-related information that is information regarding control of the learning robot. The learning control-related information includes control information of the learning robot created based on the learning document data, and is a learning model. [Industrial Applicability]

[0272] The present disclosure provides an information generation device, an information generation method, a computer program, and a learning model, and has industrial applicability. [Description of Signs]

[0273] 100 Learning device, 111 Acquisition unit, 112 Learning unit, 200 Generation AI device, 300 Information generation device, 311 Reception unit, 312 Generation unit, 313 Update unit, 314 Preprocessing unit, 315 Search unit, 400 Robot management device, 500 Document database device, 600 Control database device (database device), 700 Environment database device (database device), TM Learning model, RB Robot, EQ Industrial equipment, MG Management system, NW Network

Claims

1. A generation unit that inputs document data indicating the operation content to be executed by the robot to be controlled into a learning model and causes the learning model to generate control-related information indicating information related to the control of the robot to be controlled; A storage unit that stores the control-related information; and The document data includes information on an API provided when the robot to be controlled cooperates with an external device; The control-related information includes control information of the robot to be controlled; The control information of the robot to be controlled includes information for using the API for the robot to be controlled to cooperate with the external device; The learning model is constructed by learning a learning data set; The learning data set includes learning document data indicating the operation content to be executed by a learning robot and learning control-related information that is information related to the control of the learning robot; The learning document data includes information on an API provided when the learning robot cooperates with an external device; The learning control-related information includes control information of the learning robot created based on the learning document data; The control information of the learning robot includes information for using the API for the learning robot to cooperate with the external device, an information generation device.

2. The learning model is constructed by reinforcement learning using a reward value calculated based on the execution result and / or quality evaluation result of control information generated by the learning model during learning. The information generation device according to claim 1.

3. The learning control-related information includes information indicating a change history when the control information of the learning robot is created via an input device operated by an operator. The information generation device according to claim 1 or claim 2.

4. The control information of the learning robot includes control information created via an input device operated by an operator. The information generation device according to claim 1 or claim 2.

5. The apparatus further includes a search unit that searches a database device for reference control-related information related to the content of the document data; The reference control-related information is information related to the control of a robot; The generation unit inputs the document data and the search result by the search unit into the learning model, and causes the learning model to generate the control-related information. The information generation device according to claim 1 or claim 2.

6. The database device stores information indicating a change history of the control information of the robot. The information generation device according to claim 5.

7. The information generation device according to claim 5, further comprising an update unit that updates the database device by storing the control information of the controlled robot generated by the learning model in the database device together with information indicating a change history.

8. The database device A control database that stores a plurality of pieces of the reference control-related information, A control vector database that stores a plurality of control-related vector information indicating each of the plurality of reference control-related information, and includes: The search unit calculates a similarity between the numerical vector information indicating the document data and each of the control-related vector information, selects control-related vector information from among the plurality of control-related vector information based on the similarity, and selects the selected control-related vector information. The reference control-related information corresponding to the information is acquired from the control database, and the acquired reference control-related information is output to the generation unit as the search result. The information generation device according to claim 5.

9. The learning model has previously learned environmental information related to the robot, The environmental information includes information related to the technical environment or legal environment of the robot. The information generation device according to claim 1 or claim 2.

10. The information generation device further includes a search unit that searches a database device for environmental information related to the content of the document data, The environmental information includes information related to the technical environment or legal environment of the robot, The generation unit inputs the document data and the search result by the search unit into the learning model, and causes the learning model to generate the control-related information. The information generation device according to claim 1 or claim 2.

11. The database device An environment database that stores a plurality of pieces of the environmental information, An environment vector database that stores a plurality of environment vector information indicating each of the plurality of environmental information, and includes: The search unit calculates a similarity between the numerical vector information indicating the document data and each of the environmental vector information, selects environmental vector information from among the plurality of environmental vector information based on the similarity, obtains the environmental information corresponding to the selected environmental vector information from the environmental database, and outputs the obtained environmental information to the generation unit as the search result. The information generation apparatus according to claim 10.

12. The learning robot is an unmanned transport robot, The controlled robot is an unmanned transport robot. The information generation apparatus according to claim 1 or claim 2.

13. The unmanned transport robot transports a work robot that drives in a state of being disposed on the unmanned transport robot. The information generation apparatus according to claim 12.

14. Inputting document data indicating the operation content to be executed by the controlled robot into a learning model, and causing the learning model to generate control-related information indicating information related to the control of the controlled robot; Obtaining the control-related information, The document data includes information on an API provided when the controlled robot cooperates with an external device, The control-related information includes control information of the controlled robot, The control information of the controlled robot includes information for using the API for the controlled robot to cooperate with the external device, The learning model is constructed by learning a learning data set, The learning data set includes learning document data indicating the operation content to be executed by the learning robot, and learning control-related information that is information related to the control of the learning robot, The learning document data includes information on an API provided when the learning robot cooperates with an external device, The learning control-related information includes control information of the learning robot created based on the learning document data, The control information of the learning robot includes information for using the API for the learning robot to cooperate with the external device. Information generation method.

15. To a computer, Inputting document data indicating the operation content to be executed by the controlled robot into a learning model, and causing the learning model to generate control-related information indicating information related to the control of the controlled robot; Cause the step of acquiring the control-related information to be executed, The document data includes information on APIs provided when the robot to be controlled cooperates with an external device. The control-related information includes control information of the robot to be controlled. The control information of the robot to be controlled includes information for using the APIs for the robot to be controlled to cooperate with the external device. The learning model is constructed by learning a learning data set. The learning data set includes learning document data indicating operation contents to be executed by a learning robot, and learning control-related information that is information related to the control of the learning robot. The learning document data includes information on APIs provided when the learning robot cooperates with an external device. The learning control-related information includes control information of the learning robot created based on the learning document data. The control information of the learning robot includes information for using the APIs for the learning robot to cooperate with the external device, a computer program.

Images