Work system

Abstract

To provide a technique capable of automatically moving a cart used for work according to work status.SOLUTION: A work system includes: one or more fixed-point cameras that capture an image of a work place from above; one or more autonomously driven carts each having an accommodation portion for accommodating a tool used for work and a component to be assembled to a work object and one or more cart-mounted cameras for capturing an image of the cart and a peripheral region of the cart; and a control device that controls operation of the cart and has an acquisition portion for acquiring the captured image, a determination portion for determining which of a plurality of work processes the work represented by the captured image is on the basis of a position and a posture of an object represented by the captured image on an image by inputting the captured image to a trained machine learning model, and an operation control portion for moving the cart to a position corresponding to the work process determined by the determination portion.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a work system. [Background technology]

[0002] Conventionally, a tracking vehicle that tracks a moving leader such as a person, a vehicle, a mobile robot, or an autonomous vehicle is known (Patent Document 1). In this technology, the tracking vehicle calculates the position of the leader using sensor data output from a sensor such as a camera, and travels to track the leader. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2023-537700 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional technology, a tracking vehicle calculates the position of the leader as needed without analyzing the behavior or location of objects other than the leader in the travel area or analyzing the surrounding conditions of the tracking vehicle over time. Therefore, it is difficult for the tracking vehicle to travel autonomously according to the surrounding conditions of the tracking vehicle or to predict the next movement of a moving object such as a worker in the travel area and travel ahead of the leader. This issue is not limited to tracking vehicles, but is also common to carts that store and transport tools and parts in work areas where work is performed on work objects. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] According to one embodiment of the present disclosure, a work system is provided. The work system includes one or more fixed cameras that capture bird's-eye images of a work site; one or more unmanned mobile carts, each having a storage unit that stores tools used in the work and parts to be assembled to a work object, and one or more cart cameras that capture images of the cart and the area surrounding the cart; and a control device that controls the operation of the cart, the control device including an acquisition unit that acquires captured images output from at least one of the fixed cameras and the cart cameras, a determination unit that inputs the captured images into a trained machine learning model and determines which of multiple work processes the work represented by the captured images is based on the position and orientation of the object in the image, and an operation control unit that moves the cart to a position corresponding to the work process determined by the determination unit. According to this embodiment, the control device can analyze the work situation at the work site in chronological order based on the movement and position of multiple objects present at the work site. This allows the control device to automatically move the cart to an appropriate position at the required timing depending on the work situation, or to predict the next action of a movable object such as a worker present at the work site and move the cart in advance. The present disclosure can be realized in various forms other than the above-described operation system, such as a method for manufacturing an operation system, a method for controlling an operation system, a computer program for implementing the control method, a non-transitory recording medium on which the computer program is recorded, etc. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram showing the configuration of a work system. [Figure 2] FIG. 2 is a block diagram showing the configuration of a central control device. [Figure 3] FIG. 10 is a diagram showing an example of an operation flow of the work system. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: FIG. 1 is a diagram illustrating the configuration of a work system 1. The work system 1 is used by a worker P at a work location A to perform work on work objects W1 and W2. In this embodiment, the work objects W1 and W2 are vehicles. The work location A is a factory that manufactures vehicles. The factory includes, for example, a work line L and a preparation area R. On the work line L, the work objects W1 and W2 are transported by transportation equipment such as a conveyor. The worker P uses predetermined tools to assemble predetermined parts at predetermined assembly positions on the work objects W1 and W2 transported on the work line L. In the preparation area R, at least one of the tools to be used for the work and the parts to be assembled on the work objects W1 and W2 is prepared on a cart 40. The worker P opens a shelf installed in the preparation area R that stores tools and parts, and grasps and removes the desired tool or part. Then, the worker P sets the tools and parts that he or she has taken out in predetermined positions on the cart 40 and closes the shelf. That is, in this embodiment, the work is performed by a work method in which the selection and assembly of parts are separated. The work method in which the selection and assembly of parts are separated is also called "SPS (Set Parts Supply)."

[0009] The work system 1 includes one or more fixed cameras 21 to 26, one or more carts 40, and a central control device 60.

[0010] The fixed cameras 21-26 capture images of the work place A from above. The fixed cameras 21-26 are equipped with communication devices (not shown) and can communicate with other devices such as the central control device 60 via wired or wireless communication. The fixed cameras 21-26 are fixed to support members such as the ceiling, walls, and pillars of the work place A, for example.

[0011] The dolly 40 has a start mode, a standby mode, an operation mode, and a stop mode. When a start command is received from the worker P, the dolly 40 is set to the start mode. In the start mode, the dolly 40 starts up. When the dolly 40 is set to the start mode and becomes operable, the dolly 40 is set to the standby mode. In the standby mode, the dolly 40 waits to receive an operation from the worker P. When an operation to switch to the operation mode is received from the worker P during a period when the dolly 40 is set to the standby mode, the dolly 40 is set to the operation mode. In the operation mode, the dolly 40 can move by unmanned operation. "Unmanned operation" means operation without operation by the worker P. The operation by the worker P means an operation related to at least one of "running," "turning," and "stopping" of the dolly 40. Unmanned operation is achieved by automatic or manual remote control using a device located outside the carriage 40, or by autonomous control of the carriage 40. In this embodiment, during a period when unmanned operation control is being executed, the carriage 40 moves under remote control from the central control device 60. During a period when the carriage 40 is set to the operation mode, if an operation of the carriage 40 from the worker P is received, the carriage 40 is set to a standby mode. As a result, the carriage 40 performs an operation according to the operation of the worker P. In other words, manual operation by the worker P takes priority over automatic operation under unmanned operation control. During a period when the carriage 40 is set to the operation mode, if an abnormality occurs in the work, the carriage 40 temporarily stops and is set to a standby mode. During a period when the carriage 40 is set to the standby mode, if an end command is received from the worker P, the carriage 40 is set to an end mode. In the end mode, the power of the carriage 40 is cut off, and the carriage 40 comes to a complete stop.

[0012] The carriage 40 includes a storage section 41 , one or more carriage cameras 43 , and a carriage control device 47 .

[0013] The storage section 41 stores at least one of tools used in the work and parts to be assembled to the work targets W1 and W2.

[0014] The dolly camera 43 captures images of the dolly 40 and the area surrounding the dolly 40. The dolly camera 43 can capture images of the environment surrounding the dolly 40, such as the state of tools and parts in the storage section 41, the movements of the worker P, and any interfering objects. The dolly camera 43 is equipped with a communication device (not shown) and can communicate with other devices such as the central control device 60 via wired or wireless communication. The dolly camera 43 is fixed to the outside of the dolly body 49, for example.

[0015] The bogie control device 47 controls a group of actuators (not shown) to move the bogie 40. In this embodiment, the bogie control device 47 controls the group of actuators using a control signal received from the central control device 60 to move the bogie 40. The control signal includes, for example, at least one of the acceleration and the velocity of the bogie 40 and the steering angle as parameters.

[0016] FIG. 2 is a block diagram showing the configuration of the central control device 60. The central control device 60 controls the operation of the cart 40. In this embodiment, the central control device 60 is a server installed in a factory. The central control device 60 includes a processor 601, a memory 602, an input / output interface 603, and a bus 604. The processor 601, the memory 602, and the input / output interface 603 are connected via the bus 604 to enable bidirectional communication. A communication device 605 is connected to the input / output interface 603 for communicating with various devices external to the central control device 60. The communication device 605 can communicate with the cart control device 47 and the cart camera 43 via wireless communication and with the fixed cameras 21 to 26 via wired or wireless communication. The processor 601 executes a program PG stored in the memory 602 to function as an acquisition unit 611, a determination unit 612, and an operation control unit 613.

[0017] The acquisition unit 611 acquires the captured images output from at least one of the fixed cameras 21 to 26 and the dolly camera 43.

[0018] The determination unit 612 determines which of a plurality of work processes is performed on one work object W1, W2, the work represented by the captured image is, using an acquired image that shows the position and posture of the object represented by the captured image on the image. In this embodiment, when the object represented by the captured image is at least one of the cart 40, a tool, a part, and the work object W1, W2, the acquired image is a segmentation image in which the areas constituting the captured image are separated by segmentation into the target object and non-target object. When the object represented by the captured image is a worker P, the acquired image is a keypoint image that shows the musculoskeletal system of the worker P detected by keypoint detection.

[0019] In this embodiment, the determination unit 612 acquires an acquired image by inputting the captured image acquired by the acquisition unit 611 to a trained first machine learning model M1. The first machine learning model M1 is a machine learning model that outputs an acquired image when a captured image is input. When the object represented by the captured image is at least one of the cart 40, a tool, a part, and the work targets W1 and W2, the first machine learning model M1 is trained to perform, for example, instance segmentation. When the object represented by the captured image is a worker P, the first machine learning model M1 is trained to perform keypoint detection. The first machine learning model M1 can be, for example, a convolutional neural network (hereinafter, CNN) trained by supervised learning using a first training dataset D1. The first training dataset D1 includes, for example, a plurality of training images including at least one of the cart 40, a tool, a part, the worker P, and the work targets W1 and W2, and correct labels according to the types of objects represented by the training images. In the first learning dataset D1, of the multiple training images, training images that represent at least one of the cart 40, the tool, the part, and the work objects W1 and W2 are associated with region labels as correct labels. The region correct labels are labels that indicate whether each region in the training image represents the target object or a region that represents something other than the target object. Of the multiple training images, training images that represent a worker P are associated with musculoskeletal labels as correct labels. The musculoskeletal labels are labels that indicate the musculoskeletal structure of the worker P. Furthermore, each of the multiple training images may be associated with a classification label for classifying the object represented by the training image. During CNN training, it is preferable to update the CNN parameters by backpropagation (error backpropagation) so as to reduce the error between the output result of the first machine learning model M1 and the label.

[0020] Furthermore, the determination unit 612 inputs acquired images output from the first machine learning model M1 that represent at least one of a tool, a part, and a worker P into the trained second machine learning model M2. As a result, the determination unit 612 acquires process identification information indicating which of multiple work processes the work represented by the captured image from which the acquired image was derived is. The determination unit 612 determines that the work process identified by the process identification information is the work process represented by the captured image. The second machine learning model M2 is a machine learning model that outputs process identification information indicating the work process represented by the acquired image when the acquired image is input. The second machine learning model M2 can be, for example, a CNN trained by supervised learning using a second training dataset D2. The second training dataset D2 includes, for example, multiple acquired images as training images and process identification information as correct labels. During CNN training, it is preferable to update the parameters of the CNN using backpropagation (error backpropagation) to reduce errors between the output results of the second machine learning model M2 and the labels.

[0021] The second training dataset D2 may include training images representing the positions and postures of the work objects W1 and W2 when work on each work object W1 and W2 begins. Specifically, the second training dataset D2 may include, as training images, segmentation images representing the work objects W1 and W2 when work on each work object W1 and W2 begins. The second training dataset D2 may include, as training images, segmentation images representing the cart 40 positioned at a preset initial position corresponding to the work on each work object W1 and W2. The second training dataset D2 may include, as training images, keypoint images representing the worker P waiting at a preset standby position corresponding to the work on each work object W1 and W2. Among the multiple training images, a training image representing the position and posture of an object when work begins is associated, for example, with process identification information indicating that the work is the first of multiple work processes performed on one work object W1 and W2 as a correct answer label. Among the plurality of training images, a training image showing the position and posture of an object at the start of a task may further be associated with object identification information as supplementary information. The object identification information is information indicating which of the plurality of work objects W1, W2 the task shown in the training image is being performed on.

[0022] The operation control unit 613 controls the operation of the cart 40. In this embodiment, the operation control unit 613 controls the operation of the cart 40 by generating a control signal for controlling the operation of the cart 40 and transmitting it to the cart 40. During a period in which the cart 40 is set to the standby mode, the operation control unit 613 controls the operation of the cart 40 so as to perform an operation in accordance with an operation by the worker P. During a period in which the cart 40 is set to the operation mode, the operation control unit 613 moves the cart 40 to a position in accordance with the work process determined by the determination unit 612.

[0023] Fig. 3 is a diagram showing an example of an operation flow of the work system 1. The flow shown in Fig. 3 starts, for example, when the power of the cart 40 is turned on.

[0024] In step S1, worker P signals to cameras 21-26, 43 to start the cart 40 in order to start work on the first work object W1, which is being transported as the Nth object on the work line L. The central control device 60 acquires captured images from fixed cameras 21-26 and the cart camera 43, which include worker P in their detection ranges. The central control device 60 acquires keypoint images by inputting the captured images representing worker P into a trained first machine learning model M1. When information representing a predetermined posture is acquired from the keypoint images, the central control device 60 detects that worker P has signaled to cameras 21-26, 43 to start the cart 40. This causes the central control device 60 to accept a start command from worker P for the cart 40. When the central control device 60 accepts a start command from worker P for the cart 40, it sets the cart 40 to a start mode. This causes the cart 40 to start. When the carriage 40 becomes operable, the central control unit 60 sets the carriage 40 to a standby mode.

[0025] In step S2, the worker P moves the cart 40 to a standby position corresponding to the work performed on the first work object W1 in order to move the cart 40 to an initial position corresponding to the work performed on the first work object W1. The central control device 60 detects that the worker P has moved to the standby position corresponding to the work performed on the first work object W1. When the central control device 60 detects that the worker P has moved to the standby position corresponding to the work performed on the first work object W1, the central control device 60 extracts, from the second training dataset D2, a segmentation image of the cart 40 associated with process identification information indicating that this is the work process to be performed first and object identification information indicating the first work object W1. Based on the position and orientation of the cart 40 on the extracted segmentation image, the central control device 60 generates a control signal for moving the cart 40 to an initial position corresponding to the work performed on the first work object W1 and transmits the generated control signal to the cart 40. The carriage control device 47 controls the actuator group using the control signal received from the central control device 60, thereby moving the carriage 40 to an initial position appropriate for the work on the first work object W1. The central control device 60 sets the carriage 40 to an operating mode. This causes the carriage control device 47 to start controlling unmanned operation. The worker P uses the carriage 40 to start work on the first work object W1.

[0026] In step S3, the central control device 60 acquires captured images from the dolly camera 43, whose detection range includes at least one of a tool used in the work on the first work object W1 and a part to be assembled to the first work object W1. The central control device 60 acquires captured images from the fixed cameras 21-26 and the dolly camera 43, whose detection range includes the worker P. The central control device 60 inputs the captured images representing at least one of the tool, part, and worker P into the trained first machine learning model M1 to acquire captured images representing at least one of the tool, part, and worker P. The central control device 60 inputs the captured images representing at least one of the tool, part, and worker P into the trained second machine learning model M2 to acquire process identification information. The central control device 60 determines that the work process identified by the process identification information is the work process represented by the captured images. The central control device 60 extracts a segmentation image of the dolly 40 associated with the acquired process identification information from the second training dataset D2. The central control device 60 determines the next target position to which the cart 40 should head, based on the position and orientation of the cart 40 on the extracted segmentation image. The central control device 60 generates a control signal for moving the cart 40 to the determined target position, and transmits the generated control signal to the cart 40. The cart control device 47 controls the actuator group using the control signal received from the central control device 60, thereby moving the cart 40 to a position appropriate for the work process.

[0027] In step S4, while the carriage 40 is set to the operation mode, the central control device 60 repeatedly identifies the work process, determines the target position according to the work process, generates control signals, and transmits the control signals at a predetermined cycle. The carriage control device 47 repeatedly receives control signals and controls the actuators at a predetermined cycle. This allows the work system 1 to successively correct the position of the carriage 40 even during work.

[0028] In step S5, while the dolly 40 is set to the operation mode, the worker P signals to the cameras 21-26, 43 to manually operate the dolly 40 as necessary. As in step S1, the central control device 60 detects the signal from the worker P through image processing and accepts an operation from the worker P regarding the dolly 40. When the central control device 60 accepts an operation from the worker P regarding the dolly 40, the central control device 60 identifies the operation requested by the worker P regarding the dolly 40 from the operation content of the worker P. The central control device 60 generates a control signal for executing the identified operation and transmits the generated control signal to the dolly 40. The dolly control device 47 controls the actuator group using the control signal received from the central control device 60 to execute the operation requested by the worker P. When the central control device 60 accepts an operation from the worker P regarding the dolly 40 while the dolly 40 is set to the operation mode, the central control device 60 sets the dolly 40 to the standby mode. Furthermore, if an abnormality occurs in the work while the carriage 40 is set to the operation mode, the central control device 60 generates a control signal for temporarily stopping the carriage 40 and transmits the generated control signal to the carriage 40. The carriage control device 47 temporarily stops the carriage 40 by controlling the actuator group using the control signal received from the central control device 60. If an abnormality occurs in the work while the carriage 40 is set to the operation mode, the central control device 60 sets the carriage 40 to a standby mode. Note that if an abnormality occurs in the work, the carriage 40 may be temporarily stopped by manual operation by the worker P.

[0029] In step S6, when the worker P wants to resume the unmanned movement of the cart 40, the worker P gives a predetermined signal toward the cameras 21-26, 43. As in step S1, the central control device 60 detects the signal from the worker P by image processing and accepts an operation to resume the unmanned movement. When the central control device 60 accepts an operation to resume the unmanned movement of the cart 40, the central control device 60 sets the cart 40 to the operating mode again. This causes the cart control device 47 to resume control of the unmanned operation. Note that when the cart 40 resumes its unmanned movement, the worker P may give a signal toward the cameras 21-26, 43 to cause the cart 40 to perform a desired operation.

[0030] In step S7, when all work processes for the first work object W1 are completed, the worker P moves to a standby position corresponding to the work for the second work object W2 to begin work on the second work object W2, which is being transported (N+1)th on the work line L. The central control device 60 detects that the worker P has moved to the standby position corresponding to the work for the second work object W2. When the central control device 60 detects that the worker P has moved to the standby position corresponding to the work for the second work object W2, it sets the cart 40 to an operating mode. Then, the central control device 60 extracts, from the second training data set D2, a segmentation image of the cart 40 associated with process identification information indicating that this is the work process to be performed first and object identification information indicating the second work object W2. Based on the position and orientation of the cart 40 on the extracted segmentation image, the central control device 60 generates a control signal for moving the cart 40 to an initial position corresponding to the work for the second work object W2 and transmits the generated control signal to the cart 40. The carriage control device 47 controls the actuator group using the control signal received from the central control device 60, thereby moving the carriage 40 to an initial position according to the work on the second work object W2. The worker P starts work on the second work object W2 using the carriage 40.

[0031] According to the above embodiment, the central control device 60 can acquire captured images from the fixed cameras 21-26 and the cart camera 43 to analyze the movements and positions of multiple objects present in the work area A, such as the cart 40, tools, parts, worker P, and work targets W1 and W2. The central control device 60 can then use machine learning to acquire captured images representing the positions and postures of objects represented in the captured images in order to analyze the work situation at the work area A, such as the positions of the cart 40 and work targets W1 and W2, the status of the tools and parts, and the movements of worker P. The central control device 60 can then use the captured images to determine which of multiple work processes the work represented in the captured images corresponds to. The central control device 60 can then move the cart 40 to a position corresponding to the work process determined as the work represented by the captured images. In other words, the central control device 60 can analyze the work situation at the work area A in chronological order based on the movements and positions of multiple objects present in the work area A. This allows the central control device 60 to automatically move the cart 40 to an appropriate position at the required timing depending on the work situation, or to predict the next action of a movable object such as a worker P present at the work site A and move the cart 40 in advance.

[0032] Furthermore, according to the above embodiment, the central control device 60 can use keypoint images representing the musculoskeletal system of the worker P to determine which of a plurality of work processes the work depicted in the captured image corresponds to, and can move the cart 40 accordingly. That is, the central control device 60 can move the cart 40 in accordance with the movements of the worker P. This reduces the possibility of the worker P having to wait, being required to move according to the position of the cart 40, or interfering with the worker P due to the cart 40 not being positioned in an appropriate position at the required time. Furthermore, the position of the cart 40 can be successively corrected in accordance with the movements of the worker P even during work. This reduces the possibility of the worker P being unable to reach the cart 40 when moving from the front to the rear of the work objects W1, W2 due to the large size of the work objects W1, W2 during work. Note that if the worker P depicted in the captured image is a beginner, the work system 1 may move the cart 40 to a position in accordance with the movements of an experienced worker. In this way, the work system 1 can make it easier for beginners to learn the standard movements of experts.

[0033] Furthermore, according to the above embodiment, the central control device 60 can move the cart 40 to a position according to the work process. In other words, the central control device 60 can adjust the distance between the worker P and the cart 40 for each work process.

[0034] Furthermore, according to the above embodiment, the central control device 60 can refer to the second training data set D2 that associates the work objects W1 and W2 with a segmentation image representing the cart 40 placed at an initial position corresponding to the work objects W1 and W2. This allows the central control device 60 to automatically move the cart 40 to an initial position corresponding to the work on the second work object W2 when all work processes for the first work object W1 are completed.

[0035] Furthermore, according to the above embodiment, the work system 1 can identify the position of the dolly 40 using the captured images output from the fixed cameras 21 to 26 and the captured images output from the dolly camera 43. In this way, even when the dolly 40 is surrounded on all four sides by the workers P, information can be supplemented by the captured images output from the fixed cameras 21 to 26. This reduces the possibility that it will be difficult to identify the position of the dolly 40. Furthermore, by combining the captured images output from the fixed cameras 21 to 26 and the captured image output from the dolly camera 43, it is possible to avoid a decrease in the accuracy of estimating the position of the dolly 40 as the distance between the fixed cameras 21 to 26 and the dolly 40 increases. Furthermore, by synchronizing the captured images output from the fixed cameras 21 to 26 and the captured image output from the dolly camera 43, the position of the dolly 40 can be identified using a movable object as a reference, not limited to a fixed object.

[0036] Furthermore, according to the above embodiment, the work system 1 can identify the state of the tools and parts in the storage unit 41 by using the captured image output from the dolly camera 43. In this way, it is possible to use the captured image of the dolly camera 43, which captures the tools and parts from a location closer than the fixed cameras 21 to 26. This reduces the possibility that the worker P overlaps with the tools or parts in the captured image, making it difficult to identify the state of the tools or parts.

[0037] Furthermore, according to the above embodiment, the work system 1 can identify the movements of the worker P by using the captured images output from the fixed cameras 21 to 26 and the captured images output from the dolly camera 43. This reduces the possibility that it will be difficult to identify the movements of the worker P due to other objects overlapping the worker P in the captured images or parts of the body of the worker P being outside the detection ranges of the cameras 21 to 26, 43.

[0038] Furthermore, according to the above embodiment, work is performed by an SPS that separates the selection and assembly of parts. In this case, the dolly 40 containing heavy tools and parts is repeatedly moved between the preparation area R and the work line L, which may place strain on the worker P's legs, lower back, and wrists. In contrast, in the above embodiment, the dolly 40 can be moved by unmanned operation. This reduces the burden on the worker P caused by physically operating the dolly 40.

[0039] Furthermore, in the above embodiment, the central control device 60 uses the cameras 21 to 26, 43 to track objects present in the work area A, and therefore can automatically determine the progress of the work. In this way, it is possible to avoid imposing additional work on the worker P other than operating the cart 40. Therefore, it is possible to avoid an increase in work time due to additional work other than operating the cart 40.

[0040] B. Other Embodiments: (B1) The work objects W1 and W2 may be other than vehicles. The work objects W1 and W2 may be moving bodies other than vehicles, or objects other than moving bodies. Furthermore, when the work objects W1 and W2 are moving bodies, the moving bodies may be capable of moving by unmanned operation. In this case, the moving bodies are transported on the work line L by utilizing the movement of the moving bodies by unmanned operation.

[0041] (B2) Work location A may be a location other than a factory that manufactures work objects W1 and W2. Work location A may be a factory that repairs work objects W1 and W2 after shipment, a factory that inspects work objects W1 and W2 after shipment, or a location other than a factory.

[0042] (B3) The carriage 40 may move under autonomous control. In this case, the function of the central control device 60 is realized by the carriage control device 47.

[0043] (B4) Each of the multiple training images may further be associated with type information indicating the type of work object W1, W2 as supplementary information. In the present embodiment, when the work objects W1, W2 are vehicles, the type information may be, for example, information indicating the vehicle class determined based on the overall length, width, and height, or the vehicle model classified based on the vehicle's external shape. The type information may also be information indicating factors that affect the work procedure, such as the drive system of the motor or engine, or the type and installation location of accessories such as a steering wheel. In this configuration, the central control device 60 can move the cart 40 to a position corresponding to the type of work object W1, W2.

[0044] (B5) Of the multiple training images, a training image representing a worker P may further be associated with worker information about the worker P as supplementary information. The worker information includes, for example, at least one of worker identification information that identifies the multiple workers P, physique information that indicates the physique of the worker P, and characteristic information that indicates the characteristics of the worker P, such as dominant hand. In this configuration, the central control device 60 can move the cart 40 to a position that corresponds to the physique and characteristics of the worker P.

[0045] (B6) The machine learning models M1 and M2 may be updated by re-learning after the work is completed. This can improve the accuracy of the machine learning models M1 and M2. Therefore, the central control device 60 can move the cart 40 to a more suitable position at the necessary timing depending on the work situation.

[0046] (B7) The central control device 60 may accept operations from the worker P on the cart 40 by recognizing that a button provided on the cart 40 or at the work location A, or a button displayed on a monitor mounted on the cart 40, has been operated.

[0047] (B8) At least some of the work performed in the preparation area R may be performed by a robot, or may be performed through collaboration between a worker P and a robot. In addition, the work may be performed using a work method other than SPS.

[0048] (B9) The determination unit 612 may input the captured image into a trained third machine learning model to acquire process identification information indicating which of multiple work processes the work represented by the captured image is, and determine that the work process identified by the process identification information is the work process represented by the captured image. The third machine learning model is a machine learning model that outputs process identification information indicating the work process represented by the captured image when the captured image is input. For example, the third machine learning model may be a CNN trained by supervised learning using a third training dataset. The third training dataset includes various information contained in the first training dataset D1 and the second training dataset D2. The third training dataset may include, for example, a plurality of training images including at least one of the cart 40, a tool, a part, a worker P, and work targets W1 and W2, and process identification information as a correct label. In this configuration, the determination unit 612 can determine the work process without acquiring the captured image.

[0049] The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features of the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted. [Explanation of symbols]

[0050] 1...work system, 21-26...fixed camera, 40...cart, 41...storage section, 43...cart camera, 47...cart control device, 49...cart body, 60...central control device, 601...processor, 602...memory, 603...input / output interface, 604...bus, 605...communication device, 611...acquisition section, 612...judgment section, 613...operation control section, A...work location, D1...first learning dataset, D2...second learning dataset, L...work line, M1...first machine learning model, M2...second machine learning model, P...worker, PG...program, R...preparation area, W1...first work object, W2...second work object

Claims

[Claim 1] 1. A work system comprising: One or more fixed cameras that capture an overhead image of the work place; One or more carriages that can be moved by unmanned operation, a storage section for storing tools used in the work and parts to be assembled to the work object; one or more dollies having one or more dolly cameras configured to capture images of the dolly and the area surrounding the dolly; A control device for controlling the operation of the carriage, an acquisition unit that acquires captured images output from at least one of the fixed camera and the dolly camera; a determination unit that determines which of a plurality of work processes the work represented by the captured image is based on the position and orientation of the object represented by the captured image on the image by inputting the captured image into a trained machine learning model; A work system comprising: a control device having an operation control unit that moves the cart to a position corresponding to the work process determined by the determination unit.

Images